Robotic arms are widely used in the automation industry to package and deliver classified objects. When the products are small objects with very similar shapes, such as screwdriver bits with slightly different threads, pointed tips, and thicknesses, object selection and assembly often lead to misjudgment. We have developed a practical robotic arm control system based on vision detection techniques for screwdriver bits’ placement. In addition to effectiveness, easy deployment and high flexibility in the field are also taken into account. The vision-based system consists of four processing stages in the following order: world coordinate conversion from image pixel coordinates, object detection, edge detection, and object orientation. In the first stage, a manual two-point marking method is proposed to easily configure the coordinate conversion for robot operating system (ROS)-based manipulators. For the following stages, we focus on the fine integration of state-of-the-art methods for the technical feasibility of the screwdriver bit placement. Such integration includes the selection between object detection methods and the data flow control among system stages. The experimental results show that (1) in detecting screwdriver bits, You Only Look Once (YOLO) v4 outperforms YOLOv7 and Single Shot MultiBox Detector at an accuracy rate of 99.51%; (2) in the edge detection, the object detection output can better illustrate the object contour than a whole image, achieving a mean absolute error of 0.86% in estimating the object angle; and (3) a successful real-time replacement rate of 96% is achieved for 12 screwdriver bits randomly scattered on a conveyor belt.