Interactive UR5 Demo: Hand Tracking
A UR5 robot arm that mirrors hand movements in real time using MediaPipe hand tracking. Built in ~6 hours; uses servoL (velocity mode) for smooth following. Next goal: make it fist-bump.
May 2026
A live demo where a UR5 robot arm mirrors hand movements captured by a camera in real time. The key insight was switching from position-mode (moveL) to velocity-mode (servoL), which made the motion dramatically smoother and faster to implement.
About 6 hours of work: 2 hrs configuring a safety plane in the UR5, 2 hrs on the hand-coordinate to robot-coordinate transform code, and 2 hrs taking videos and showing friends.
Stack: MediaPipe hand tracking, Python UR5 driver over URScript/serial, safety plane constraints.
Next steps being considered: accepting Z-axis input (currently limited to XY plane for safety), adding derivative control to prevent runaway oscillation when hands move too fast, making the robot high-five or fist-bump, and eventually — making it build sandwiches.
