1. Basic Usage
Tutorial Overview
This tutorial will teach you how to create, edit, and execute objectives using MoveIt Pro.
An objective is a high-level function or application that can be performed by the robot, for example picking up an object from a table. Objectives are created using Behavior Trees, which are similar to state machines and are composed of behaviors, which are the low-level nodes or skills. An example of a behavior is opening a gripper or moving to a pose. To learn more about behaviors, see Built-in Behaviors. MoveIt Pro contains a wide variety of pre-made objectives and behaviors, so you can start commanding a robot right away.
Start MoveIt Pro
We assume you have already installed MoveIt Pro. Launch the application using:
moveit_pro run -c lab_sim
Run View
MoveIt Pro has two view tabs, a Build view, which is used when creating or editing objectives, and a Run view, for running objectives. Click on the Run tab.
The Run view shows the current visualization of the robot, the current objective's Behavior Tree, the Favorites toolbar (center), the collapsible More Objectives sidebar panel (left), and a pane selection drop-down menu (right).
Panes Overview
By default, you’ll see there are four panes displayed, the Visualization pane on the top left, the Behavior Tree pane on the top right, the Color Camera pane on the bottom left, and the Depth camera pane on the bottom right. All panes can be changed using the drop-down lists in the top left corner of each pane.
Various Panes
The Visualization pane displays a rendering of what the robot understands of the word, similar to the common Rviz visualizer in ROS. In this Visualization, you should see a robot arm and a table. The table is a simplified model of the lab bench that is being simulated. This is also known as the Planning Scene, and is used to avoid collision with the robot and the world. The view of the scene can be adjusted by clicking within the pane and dragging the mouse around. The left mouse button rotates the scene, and the right mouse button drags the scene.
The Color Camera and Depth Camera panes show the camera feeds from the underlying simulator. If you were connected to hardware, these cameras would show the real world and not simulations. By default, these are set to the scene cameras, which are simulated third-person views of the robot, both in color, and depth respectively.
In the Behavior Tree pane, the most recently run objective is shown, if any. While running an objective, this pane will highlight which behavior is currently executing, which is useful for debugging and introspection.
Use the drop-down list to change the bottom right pane to /wrist_camera/color
. This will change the camera view to the camera mounted on the robot’s wrist.
Setting Favorites
The Favorites toolbar can be customized for your needs. To add/remove objectives to/from the toolbar, click on the three dots next to an Objective in the Objectives sidebar to bring up a drop-down menu, and then star/unstar the Objective.
Now that you understand the Run view, you’re ready to run your first objective.
Run your first objective
Now, to run an objective in the Run view, choose Move Beakers to Burners
from the toolbar.
You should see the robot arm pick up the three flasks on the left side of the workspace and move them to the burner on the right. When finished, the objective status will update to Objective complete!
Now run the Push Button
objective and the robot will push a button to turn on the burners. This objective uses admittance control to push the button with a known force.
Try out more example objectives
Before diving in deeper, let’s see some more objectives in action.
The Objectives are listed in the collapsible Objectives sidebar on the left side of the Run mode screen.
The Take scene camera snapshot
Objective under the Application - Basic Examples section will take a depth camera snapshot of the table, which will appear in the Visualization pane. They are captured and displayed as point clouds in the MoveIt Pro. Run the Take scene camera snapshot
objective and you will see a point cloud appear in the visualization pane.
Run the Clear Snapshot
Objective under the Perception - 3D Point Cloud section to clear the point cloud from the visualization pane.
Now, run the 3 Waypoints Pick and Place
Objective under the Application - Basic Examples section. This objective runs a continuous loop of picking an object using hard-coded waypoints. It is a basic example of an application in MoveIt Pro.
Use the Stop Motion button on the top right to stop the loop when you’ve seen it run completely.
You’ll see the objective status change from Executing to Objective canceled
Select the Look at Table
Objective under the Application - Basic Examples section to reset the robot pose back to its original pose.
Teleoperate the Robot
MoveIt Pro provides several forms of manual control for when a robot needs to be set up, diagnosed, or recovered.
On the top right of the toolbar, you’ll see the Teleoperate button. Use that to switch into an interactive mode that allows for several types of teleoperation and human-in-the-loop control.
In the top left of the menu bar you should see four available Teleoperation modes:
- Waypoints
- IMarker (Interactive Marker)
- Pose Jog
- Joints Jog
Move to Waypoint
By default, Teleoperation starts in Waypoint mode. Waypoints are end-effector poses that you can save and return to either manually or using objectives. To access this control mode, select Waypoints tab after enabling the teleoperation panel.
The toolbar provides some quick waypoints to which the robot can move to. The Waypoints sidebar to the left contains all of the saved waypoints.
Try running a few waypoints:
- Above Pick Cube
- Pick Cube
- Above Place Cube
- Place Cube
- Look at table
All the saved waypoints can be accessed by clicking the More button which will open a collapsible sidebar.
Using the Waypoints sidebar, you can:
- Mark a waypoint as favorite by using the star (favoriting a waypoint adds it to the Move to Pose hotbar).
- Move to a waypoint by using the play button.
- Add a waypoint by using the + button.
- Remove, edit, or replace a waypoint by using the three dots next to the play button.
Later in this tutorial, you’ll learn how to create new waypoints.
Using Interactive Markers to Move the End Effector
The Interactive Marker (IMarker) teleoperation mode allows users to move the robot's end effector using arrows for translation, and discs for rotation.
The marker can be used to move the end effector to a desired pose. This can be done by clicking and dragging the marker's arrows and discs. Dragging an arrow will translate the marker along that direction while turning a disc will rotate the marker in that direction.
Once the marker is in a desired and valid pose, a motion will be planned that brings the end effector from the current pose to the goal pose. A preview of the trajectory will be visualized, and if it looks safe and desirable, you can click the green check button to start the motion. The motion planning algorithm used to generate the trajectory will also be listed underneath the approval prompt.
If no preview is shown, it means that there is no valid inverse kinematics solution for the desired pose. You may have dragged the IMarker beyond the robot’s physical reach.
If the marker is in an undesirable state, the Reset Marker button will reset the marker back to the current end effector pose.
Cartesian Pose Jog Mode
The Pose Jog mode enables the user to translate or rotate the end-effector along different planes or axes, and open or close the gripper.
The Pose Jog mode can be selected on the top left under the toolbar. You’ll see the Visualization pane will now show control buttons for jogging and rotating the end effector pose in the x, y, and z directions.
Notice the buttons at the top center of the pane.
The arrow button will translate the end effector in the positive direction along its x-axis and the circular arrow will rotate the end effector in the positive direction on the x-axis plane. Similarly, the buttons at the bottom will move the opposite direction, and the buttons on the sides will translate and rotate the pose along the y-axis.
On the top right of the Visualization pane are some additional buttons that can be used to translate and rotate the end effector on the z-axis and z-axis plane.
Gripper Control in Pose Jog
On the bottom left of the Visualization pane are buttons that can be used to open and close the gripper.
Open the gripper using the left icon in the bottom left corner of the Visualization pane.
Jog collision checking and speed
On the toolbar you’ll see a settings icon, which allows you to change the jog speed, and turn off collision checking if needed.
The Jog Collision Checking toggle switch informs the simulation if it should be checking for collisions between different solid entities.
There can be situations where the robot collides with an object during an objective and is unable to be teleoperated because the beginning of the trajectory is in a collision. In that case, Jog Collision Checking can be turned off so that the robot can be teleoperated. It’s recommended to keep collision checking on unless you move the robot into a collision and need full manual control to get it back into a safe state. After it is moved back to a non-collision state, it is recommended to turn collision checking back on for safety reasons.
The Jog Speed slider adjusts the normalized speed at which the end effector/joints move, meaning at 100% each joint is moving at its max speed, as dictated by the specified joint limits.
The Jog Collision Checking and Jog Speed parameters are only used when jogging a joint via the +/- buttons in the Joints Jog view or when using the endpoint jog buttons around the Visualization pane in the Pose Jog view. This is because those two methods use MoveIt Servo (and the respective servo parameters in the robot configuration package), whereas the other modes (such as the slider in Joints Jog view and Interactive Marker in IMarker view) use a regular motion planner.
To make it easy to control your jogging during this tutorial, change the Jog Speed to 30% using the slider.
Create a Waypoint
Now that you know how to use pose jog, you can create a new waypoint.
Choose Pose Jog mode from the Teleoperate menu, then use the buttons to move the gripper so that both fingers are along the sides of the right side cube, ready to grasp it.
To create a waypoint from your robot pose, switch to Waypoints mode from the Teleoperate menu, then press the +Waypoint button.
Name your new waypoint Pick right cube
.
Note that waypoints can be saved for an individual planning group or a combination of planning groups, such as when you want to specify between two arms or just one arm. This can be set by clicking the Select Joint Group button when adding a new waypoint.
Choose Create to finish creating your new waypoint. We’ll use this later when we create a new objective.
Joint Control
The Joints Jog mode can be used to move individual joints.
In the Visualization pane, there is an area that shows the current state and min/max limits of each joint, as well as +/- buttons to manipulate each joint. The joint states can be represented in degrees or radians and can also be copied to the clipboard.
Joint Movement
There are three ways to move a joint:
- Jogging using the +/- buttons. Clicking the + button will increase the joint angle and clicking on the - button will decrease the joint angle. When using the +/- buttons, a joint can be jogged up to the minimum or maximum limit of the joint or until a collision is detected.
- Trajectory movement by clicking and dragging the slider to move to the desired joint state.
- Trajectory movement by clicking the current joint state's numerical value and entering the desired value.
Gripper Control in Joint Jog
In the secondary navigation area at the top, there are the "Open Gripper" and "Close Gripper" buttons for controlling the end effector.
Teleoperating the gripper to joint values other than "Open" or "Close" is not currently supported.
Exit Teleop
When you’re finished, click the Stop Motion button or Cancel button to exit teleoperation mode.
Building and Editing Objectives
Building a new objective
Now that you have run objectives, know how to Teleop, and know how to save waypoints, you’re ready to begin building objectives. You’re going to build a waypoint-based pick and place application using the waypoint you created earlier called Pick right cube
.
To begin, select the Build tab on the top menu bar.
In the Build tab, select +Objective. This opens the New Objective dialog. Enter My Pick and Place
as the name, then Create.
Now you have a new objective. By default, all new objectives begin with the AlwaysSuccess
behavior.
Delete the AlwaysSuccess
behavior and replace it with a Move To Waypoint
behavior. Use the waypoint_name
port textbox to set the waypoint name to Look at Table
.
Add a Clear Snapshot
behavior, then add a Take Wrist Camera Snapshot
behavior to the tree. This will add the point cloud to the visualization pane when you run the objective.
Run the objective and you’ll see the Visualization updated to show the depth camera view of the objects on the table.
Now, edit your objective, select the Build tab, then Edit on the top right of the toolbar.
Continue to build your objective, adding the following behaviors to pick up the right cube and place it.
Move To Waypoint: Pick right cube
- This is the waypoint you added earlier during the Teleop tutorial
Close Gripper
Move To Waypoint: Above Place Cube
- This waypoint is used as a mid-point between pick and place
Move To Waypoint: Place Cube
Open Gripper
Move To Waypoint: Above Place Cube
To save time, you can use the blue Duplicate icon on the top right side of any behavior to make a copy of it.
Your completed objective should look like this:
Run the objective to make sure it works! You should see the robot pick up the right cube and place it in a different location on the table.
Modify your objective to add a recovery behavior
Sometimes you may encounter a condition that causes the robot to fail to run a behavior successfully. For example, it may not be able to move from one waypoint to another. In these situations, you can add a behavior to recover from that condition before proceeding with the objective. To illustrate how to do this, you’re going to modify your objective to add a Fallback
behavior and switch into Teleop mode for the user to move the robot.
To begin, select the Build tab on the top menu bar. Then, select the My Pick and Place
objective from the objectives list on the left side, and the Edit button on the top right.
Use the Search bar to find the AlwaysFailure
in the list of behaviors, and drag it into the objective. Connect the input of the AlwaysFailure
behavior to the bottom of the sequence.
Use the Run button on the top right toolbar to run the objective. You should see the objective run and then fail. This is expected since we added the AlwaysFail
behavior. You’ll see the status has changed to Objective failed and a pop-up will appear with an error message on the right side.
Next, you’re going to add a Fallback
behavior to the objective to recover from this failure. Fallback
behaviors allow you to execute a behavior when you encounter a failure. These behaviors are called recovery behaviors.
You’re going to use Request Teleoperation
as a recovery behavior. Choose Edit from the top right corner to edit the objective. Use the search bar to find the Fallback
behavior, then drag it to the editor and add it to the sequence.
Use the Delete key to delete the existing connection between the Sequence
node and the Always Failure
node. Then connect the Fallback
between those nodes.
Use the search bar again to find the Request Teleoperation
behavior, and add that below the AlwaysFailure
.
Set the enable_user_reaction
value to true
and add a user_interaction_prompt
for the user in the Request Teleoperation
behavior.
Now run the objective again. This time after the cube is placed, you should see that the Teleoperate menus appear and you can teleop the robot, as you learned in this tutorial. When you finish, use the Success button to end the objective.
Summary
You’ve now completed this tutorial. In this tutorial, you learned how to:
- Run built-in objectives
- Teleoperate a robot
- Create a waypoint
- Create a new objective
- Modify an objective
- Add a recovery behavior to an objective
Congratulations, you’re now ready to move to the next tutorial!