Using AprilTags in MoveIt Pro

MoveIt Pro provides an example objective to detect an AprilTag and pick it up. You can view the XML here or run the Pick April Tag Labeled Object objective in the lab_sim configuration using moveit_pro run -c lab_sim. The objective will move to a predefined location where AprilTags can be seen, then capture a pointcloud using the wrist camera. If an AprilTag is detected, it will move to a closer position and refine the pose estimate of the block. It will then plan a path to grasp the block, and execute it.

Working with AprilTags

AprilTags are a type of fiducial marker widely used in applications of robotics.

They enable the use of a single RGB camera to detect 3D poses by using high-contrast, easy-to-detect, markers of known size and unique identifiers.

To experiment with the AprilTag detector behavior and visually inspect the accuracy, you can run the Loop Detect April Tag Objective in the lab_sim robot configuration. This objective will continually try to detect AprilTag markers, and place a moveit_studio_vision_msgs::msg::ObjectDetectionArray with the detected markers and their locations on the blackboard. It will also add the location of the markers to the tf tree. The GetDetectionPose behavior can be used to extract a marker's PoseStamped from the ObjectDetectionArray.

The april tag detection parameters can be edited at any time, including runtime. To make changes and see their impact on the DetectAprilTags behavior, you can use RViz for visualization:

Launch MoveIt Pro, and start the Loop Detect April Tag Objective. Then, bring up an RViz window using:

moveit_pro rviz

1. While the Objective is running you can edit the 51mm_apriltag_detection_config.yaml file in the your configuration package's objectives folder and see the effect live.
2. This configuration sets the AprilTags to be from the 36h11 family and have a size 0.08 (8 cm which is the size of the tags in the simulation).
3. You can create your own YAML parameter file by specifying a different file name and adding it to the objectives folder of your configuration package.

As shown below, the pose estimation returned from the AprilTag detector can have significant error or noise when observed at larger distances from the camera. For example here is a 5 cm tag being rotated at a distance of approximately 0.3 m from the RGB camera.

If you are experiencing noisy measurements, MoveIt Pro includes a Subtree (Sample April Tag) that can sample the pose a specified number of times and return the average. Averaging multiple estimates can help increase the repeatability of the observed pose. If you are experiencing large offsets between the pose and depth cloud produced by the 3D camera you should ensure you have the correct settings in your AprilTag configuration yaml (see this guide guide for tuning the detector parameters). You can also try to increase the camera resolution of the RGB image or increase the AprilTag size to reduce the error. For example you can see here a more accurate pose estimation when the tag is 10 cm square and at the same distance of around 0.3 m.

If you need to create or test tags of different sizes/encodings, refer to this repository for a set of pre-generated tags.