Purpose: This project developed a robotic pick-and-place system using the OpenManipulator-X arm and MATLAB. The robot integrated forward and inverse kinematics, velocity control, computer vision, and trajectory planning to sort objects by color. Using the Denavit-Hartenberg convention, Jacobian analysis, and HSV-based object detection, the team implemented a fully automated pipeline for localization, grasping, and sorting of colored balls.

Objectives:

1. Implement forward and inverse position kinematics for precise control.

2. Derive and apply velocity kinematics using the Jacobian matrix to prevent singularities.

3. Perform intrinsic and extrinsic camera calibration for accurate mapping.

4. Develop an HSV-based computer vision pipeline for robust object detection.

5. Integrate trajectory planning to enable smooth, collision-free arm motions.

6. Demonstrate dynamic tracking and automated sorting of colored objects.

Kinematics

• Forward Kinematics: Implemented in MATLAB using DH parameters to compute the pose of the end effector.

• Inverse Kinematics: Derived closed-form solutions for four joint angles, validated with real and simulated poses.

• Velocity Kinematics: Symbolic Jacobian matrix computed in MATLAB; used for mapping joint to task-space velocities.

Camera Calibration

• Intrinsic Calibration: Performed using MATLAB Camera Calibration app with 40 checkerboard images; achieved reprojection error < 1 pixel.

• Extrinsic Calibration: Mapped camera frame to robot base frame using getCameraPose() and pointsToWorld().

Object Detection

• Used HSV color segmentation for reliable detection under varying lighting.

• Located object centroid in image space, converted to real-world coordinates using calibration results.

• Applied radius correction for accurate 3D localization.

Sorting

• Implemented an automated pipeline: 

detect → localize → compute joint angles → grasp → sort into bins.

• Designed a state machine in MATLAB for continuous sorting.

• Extended system to handle moving and non-spherical objects with dynamic tracking.