Camera Pose Calibration

Many applications require knowing the precise location of a camera with respect to the robot base for performing perception tasks. Using a calibration procedure ensures repeatable accuracy of the camera pose without manual measuring and positioning.

This how-to guide will walk through an example camera calibration Objective that is used to find the pose of the camera mounted to the servicer craft in our space_satellite_sim config.

Launch MoveIt Pro

We assume you have already installed MoveIt Pro to the default install location. Launch the application using:

moveit_pro run -c space_satellite_sim_camera_cal

1. Run the Calibrate Servicer Camera Objective

To get an understanding of how to calibrate a camera, review and run the Calibrate Servicer Camera Objective. This Objective calculates the servicer craft camera's pose by referencing the known pose of the AprilTags on the calibration tool. The calibration is performed at four waypoints, and the calibrations are averaged to produce a final Calibrated Optical Pose and Calibrated Camera Mounting Pose, which is then saved to a file. Review these poses using the Markers dropdown menu.

2. Set up the config for calibration

The space_satellite_sim_camera_cal config is inherited from space_satellite_sim for the purpose of demonstrating the camera calibration procedure. You do not need a separate config to run calibration in your own config. Instead, you only need to add the calibration tool:

For Simulation:

• Add the calibration tool to the robot for the MuJoCo simulation: assets.xml, body.xml.
• Add the calibration tool to the URDF. This isn't strictly necessary, but will allow for collision checking when moving between waypoints.

For Hardware:

• Mount the physical calibration tool to the robot end effector.
• Add the calibration tool to the URDF. This will allow for collision checking when moving between waypoints.
  • Make sure the URDF matches the tool's location in the physical robot end effector.

(Optional) Use an external AprilTag detection to publish the tags' TFs

The default camera calibration objective uses the MoveIt Pro AprilTag detection Behavior. Our flexible framework also allows using an external tag TF publisher, similar to what we use here. The Calibrate Servicer Camera - Tag TFs Objective demonstrates this approach.

3. Create calibration waypoints

Create calibration waypoints about the workspace with all three AprilTags on the tool in view of the camera. Calibrating with a single waypoint can provide high local accuracy, while calibrating with multiple waypoints can provide more consistent accuracy throughout the workspace. See the Camera Calibration "X" waypoints in the space_satellite_sim_camera_cal config for examples.

4. Create a runnable calibration Objective

Create a runnable calibration Objective for a specific camera (like Calibrate Servicer Camera) by passing the appropriate inputs into the Calibrate Camera Subtree. You will also need to modify Calibrate Camera to use your calibration waypoints.

note

If using the node detection, make sure the optical_frame in the objective matches the optical frame in the 36h11.launch.

5. Interpret the calibration results

The calibration results are visualized with the calibrated optical and mount pose markers. The Objective will save the calibrated camera mounting pose to the input file name. If you are satisfied with this calibration, you can copy the file contents into the URDF to replace the existing camera mounting pose. Note that you will need to restart MoveIt Pro for this change to take effect.

These calibrated poses can also be compared to the Current URDF Optical and mount pose markers. If you rerun the calibration Objective after implementing the calibration and restarting MoveIt Pro, these markers should be nearly coincident.

note

Camera mounting pose and camera optical frame are not the same. Our calibration Objective handles the transform for you, and saves the camera mounting pose to file, which should be a direct replacement for the existing camera mounting pose in the URDF. This works for imported camera xacro models, see the RealSense in our example config.

Additional calibration details are provided by the remaining markers. The single-Image Optical Pose marker live-updates as the calibration runs on multiple images at each waypoint, and its consistency corresponds to tag detection accuracy. Variation in the camera_optical_frame markers at each of the calibration waypoints corresponds to the calibration consistency about the image and workspace.

Limitations and Troubleshooting

Calibration accuracy is fundamentally limited by the tag-to-pixel ratio, which is determined by the AprilTag size, the camera distance from the tags, and the camera resolution. The Calibrate Scene Camera Objective demonstrates this limitation. Notice how the tags are very small in the scene camera image. The result is that the calibration is much less accurate than the servicer camera calibration.

For detailed introspection of the calibration process for troubleshooting, overlay the camera image with the detected tags and robot model in RViz. Also remember that both algorithmic and real-world considerations can affect the calibration accuracy.

tip

Common causes of significant calibration error:

• tag-to-pixel ratio
• calibration tool misaligned in end effector
• camera intrinsic/distortion parameters
• lighting conditions
• tag detection parameters
• descrepancy between physical robot and URDF