Retrofitting a Legacy Industrial Robot Through Monocular Computer Vision-Based Human-Arm Posture Tracking and 3-DoF Robot-Axis Control (A1–A3)

This paper presents a low-cost retrofitting pipeline for a legacy industrial robot that uses a single RGB webcam and monocular 2D keypoint tracking to estimate human-arm posture angles (Formula presented.) and map them to robot-axis joint targets (Formula presented.) for A1–A3 on a KUKA KR5-2 ARC HW, while keeping the wrist orientation (A4–A6) fixed. Rather than targeting full six-DoF manipulation, the main contribution is an experimental characterization of how far monocular 2D posture-to-axis mapping can be used reliably for coarse placement and safeguarded low-speed demonstrations on a legacy robot platform. Vision-side accuracy was evaluated per axis against goniometer-based reference angles (Formula presented.), showing low errors for A2–A3 within the tested range and larger errors for A1 due to monocular yaw/depth ambiguity and occlusions. The study also analyzes failure modes during simultaneous multi-joint motion, where performance degrades notably, especially for A2 and A3, and reports practical mitigation directions such as improved viewpoints, multi-view/depth sensing, and stricter dropout handling. Runtime behavior is additionally characterized through a loop timing budget, with an end-to-end latency of 185.44 ms and an effective loop frequency of 5.39 Hz, which is consistent with low-speed online operation within the demonstrated scope. The system was implemented in a fenced industrial cell with restricted access and emergency stop; no collaborative operation is claimed.