1. Basic Usage

Tutorial Overview

This tutorial will teach you how to create, edit, and execute objectives using MoveIt Pro.

An objective is a high-level function or application that can be performed by the robot, for example picking up an object from a table. Objectives are created using Behavior Trees, which are similar to state machines and are composed of behaviors, which are the low-level nodes or skills. An example of a behavior is opening a gripper or moving to a pose. To learn more about behaviors, see Built-in Behaviors. MoveIt Pro contains a wide variety of pre-made objectives and behaviors, so you can start commanding a robot right away.

Start MoveIt Pro

We assume you have already installed MoveIt Pro. Launch the application using:

moveit_pro run -c lab_sim

Run View

MoveIt Pro has two view tabs, a Build view, which is used when creating or editing objectives, and a Run view, for running objectives. Click on the Run tab.

The Run view shows the current visualization of the robot, the current objective's Behavior Tree, the Favorites toolbar (center), the collapsible More Objectives sidebar panel (left), and a pane selection drop-down menu (right).

Panes Overview

By default, you’ll see there are four panes displayed, the Visualization pane on the top left, the Behavior Tree pane on the top right, the Color Camera pane on the bottom left, and the Depth camera pane on the bottom right. All panes can be changed using the drop-down lists in the top left corner of each pane.

info

Various Panes
The Visualization pane displays a rendering of what the robot understands of the word, similar to the common Rviz visualizer in ROS. In this Visualization, you should see a robot arm and a table. The table is a simplified model of the lab bench that is being simulated. This is also known as the Planning Scene, and is used to avoid collision with the robot and the world. The view of the scene can be adjusted by clicking within the pane and dragging the mouse around. The left mouse button rotates the scene, and the right mouse button drags the scene.

The Color Camera and Depth Camera panes show the camera feeds from the underlying simulator. If you were connected to hardware, these cameras would show the real world and not simulations. By default, these are set to the scene cameras, which are simulated third-person views of the robot, both in color, and depth respectively.

In the Behavior Tree pane, the most recently run objective is shown, if any. While running an objective, this pane will highlight which behavior is currently executing, which is useful for debugging and introspection.

Use the drop-down list to change the bottom right pane to /wrist_camera/color. This will change the camera view to the camera mounted on the robot’s wrist.

Setting Favorites

The Favorites toolbar can be customized for your needs. To add/remove objectives to/from the toolbar, click on the More Objectives sidebar button to show the Objectives list, then star/unstar your favorite objectives.

Now that you understand the Run view, you’re ready to run your first objective.

Run your first objective

Now, to run an objective in the Run view, choose Pick April Tag Labeled Object from the toolbar.

You should see the robot arm move to a position with the camera centered above the table, pointing down, then move to the right cube and pick it up. When finished, the objective status will update to Objective complete!

Try out more example objectives

Before diving in deeper, let’s see some more objectives in action. From the Run mode screen, choose |-> More on the left side of the toolbar to see the full list.

The Wrist camera snapshot and Clear snapshot objectives will take a depth camera snapshot of the table, which will appear in the Visualization pane. They are captured and displayed as point clouds in the MoveIt Pro. Run the Scene camera snapshot objective, you will see a point cloud appear in the visualization pane.

Run the Clear Snapshot objective to clear the point cloud from the visualization pane.

Now, run the 3 Waypoints Pick and Place objective. The 3 Waypoints Pick and Place objective runs a continuous loop of picking an object using hard-coded waypoints. It is a basic example of an application in MoveIt Pro.

Use the Stop Motion button to stop the loop when you’ve seen it run completely.

You’ll see the objective status change from Executing to Objective canceled

Select Objective List then run the Look at Table objective to reset the robot pose back to its original pose.

Teleoperate the Robot

MoveIt Pro provides several forms of manual control for when a robot needs to be set up, diagnosed, or recovered.

On the top right of the toolbar, you’ll see the Teleoperate button. Use that to switch into an interactive mode that allows for several types of teleoperation and human-in-the-loop control.

In the top left of the menu bar you should see four available Teleoperation modes:

• Waypoints
• IMarker (Interactive Marker)
• Pose Jog
• Joints Jog

Move to Waypoint

By default, Teleoperation starts in Waypoint mode. Waypoints are end-effector poses that you can save and return to either manually or using objectives. To access this control mode, select Waypoints tab after enabling the teleoperation panel.

The toolbar provides some quick waypoints to which the robot can move to.

Try running a few waypoints:

• Above Pick Cube
• Pick Cube
• Above Place Cube
• Place Cube
• Look at table

All the saved waypoints can be accessed by clicking the More button which will open a collapsible sidebar.

Using the Waypoints sidebar, you can:

• Mark a waypoint as favorite by using the star (favoriting a waypoint adds it to the Move to Pose hotbar).
• Move to a waypoint by using the play button.
• Add a waypoint by using the + button.
• Remove a waypoint by using the three dots next to the play button.

Later in this tutorial, you’ll learn how to create new waypoints.

Using Interactive Markers to Move the End Effector

The Interactive Marker (IMarker) teleoperation mode allows users to move the robot's end effector using arrows for translation, and discs for rotation.

The marker can be used to move the end effector to a desired pose. This can be done by clicking and dragging the marker's arrows and discs. Dragging an arrow will translate the marker along that direction while turning a disc will rotate the marker in that direction.

Once the marker is in a desired and valid pose, a motion will be planned that brings the end effector from the current pose to the goal pose. A preview of the trajectory will be visualized, and if it looks safe and desirable, you can click the green check button to start the motion.

If no preview is shown, it means that there is no valid inverse kinematics solution for the desired pose. You may have dragged the IMarker beyond the robot’s physical reach.

If the marker is in an undesirable state, the Reset Marker button will reset the marker back to the current end effector pose.

Cartesian Pose Jog Mode

The Pose Jog mode enables the user to translate or rotate the end-effector along different planes or axes, and open or close the gripper.

The Pose Jog mode can be selected on the top left under the toolbar. You’ll see the Visualization pane will now show control buttons for jogging and rotating the end effector pose in the x, y, and z directions.

Notice the buttons at the top center of the pane.

The arrow button will translate the end effector in the positive direction along its x-axis and the circular arrow will rotate the end effector in the positive direction on the x-axis plane. Similarly, the buttons at the bottom will move the opposite direction, and the buttons on the sides will translate and rotate the pose along the y-axis.

On the top right of the Visualization pane are some additional buttons that can be used to translate and rotate the end effector on the z-axis and z-axis plane.

Gripper Control in Pose Jog

On the bottom left of the Visualization pane are buttons that can be used to open and close the gripper.

Open the gripper using the left icon in the bottom left corner of the Visualization pane.

Jog collision checking and speed

On the toolbar you’ll see a settings icon, which allows you to change the jog speed, and turn off collision checking if needed.

The Jog Collision Checking toggle switch informs the simulation if it should be checking for collisions between different solid entities.

note

There can be situations where the robot collides with an object during an objective and is unable to be teleoperated because the beginning of the trajectory is in a collision. In that case, Jog Collision Checking can be turned off so that the robot can be teleoperated. It’s recommended to keep collision checking on unless you move the robot into a collision and need full manual control to get it back into a safe state. After it is moved back to a non-collision state, it is recommended to turn collision checking back on for safety reasons.

The Jog Speed slider adjusts the normalized speed at which the end effector/joints move, meaning at 100% each joint is moving at its max speed, as dictated by the specified joint limits.

note

The Jog Collision Checking and Jog Speed parameters are only used when jogging a joint via the +/- buttons in the Joints Jog view or when using the endpoint jog buttons around the Visualization pane in the Pose Jog view. This is because those two methods use MoveIt Servo (and the respective servo parameters in the robot configuration package), whereas the other modes (such as the slider in Joints Jog view and Interactive Marker in IMarker view) use a regular motion planner.

To make it easy to control your jogging during this tutorial, change the Jog Speed to 30% using the slider.

Create a Waypoint

Now that you know how to use pose jog, you can create a new waypoint. Choose Pose Jog mode from the Teleoperate menu, then use the buttons to move the gripper so that both fingers are along the sides of the right side cube, ready to grasp it.

To create a waypoint for your robot pose, switch Waypoints mode from the Teleoperate menu. Then choose More from the left side of the menu, then +Waypoint.

Name your new waypoint Pick right cube.

Note that waypoints can be saved for an individual planning group or a combination of planning groups, such as when you want to specify between two arms or just one arm. This can be set by clicking the Select Joint Group button when adding a new waypoint.

Choose Create to finish creating your new waypoint. We’ll use this later when we create a new objective.

Joint Control

The Joints Jog mode can be used to move individual joints.

In the Visualization pane, there is an area that shows the current state and min/max limits of each joint, as well as +/- buttons to manipulate each joint. The joint states can be represented in degrees or radians and can also be copied to the clipboard.

Joint Movement

There are three ways to move a joint:

1. Jogging using the +/- buttons. Clicking the + button will increase the joint angle and clicking on the - button will decrease the joint angle. When using the +/- buttons, a joint can be jogged up to the minimum or maximum limit of the joint or until a collision is detected.
2. Trajectory movement by clicking and dragging the slider to move to the desired joint state.
3. Trajectory movement by clicking the current joint state's numerical value and entering the desired value.

Gripper Control in Joint Jog​

In the secondary navigation area at the top, there are the "Open Gripper" and "Close Gripper" buttons for controlling the end effector.

note

Teleoperating the gripper to joint values other than "Open" or "Close" is not currently supported.

Exit Teleop

When you’re finished, click the Stop Motion button or Objective List button to exit teleoperation mode.

Building and Editing Objectives

Building a new objective

Now that you have run objectives, know how to Teleop, and know how to save waypoints, you’re ready to begin building objectives. You’re going to build a waypoint-based pick and place application using the waypoint you created earlier called Pick right cube.

To begin, select the Build tab on the top menu bar.

In the Build tab, select +Objective. This opens the New Objective dialog. Enter My Pick and Place as the name, then Create.

Now you have a new objective. By default, all new objectives begin with the AlwaysSuccess behavior.

Delete the AlwaysSuccess behavior and replace it with a Move To Waypoint behavior. Use the search bar to find Set the waypoint name to Look at Table.

Add a Clear Snapshot behavior, then add a Take Wrist Camera Snapshot behavior to the tree. This will add the point cloud to the visualization pane when you run the objective.

Run the objective and you’ll see the Visualization updated to show the depth camera view of the objects on the table.

Now, edit your objective, select the Build tab, then Edit on the top right of the toolbar.

Continue to build your objective, adding the following behaviors to pick up the right cube and place it.

• Move To Waypoint: Pick right cube
  • This is the waypoint you added earlier during the Teleop tutorial
• Close Gripper
• Move To Waypoint: Above Place Cube
  • This waypoint is used as a mid-point between pick and place
• Move To Waypoint: Place Cube
• Open Gripper
• Move To Waypoint: Above Place Cube

note

To save time, you can use the blue Duplicate icon on the top right side of any behavior to make a copy of it.

Your completed objective should look like this:

Run the objective to make sure it works! You should see the robot pick up the right cube and place it in a different location on the table.

Modify your objective to add a recovery behavior

Sometimes you may encounter a condition that causes the robot to fail to run a behavior successfully. For example, it may not be able to move from one waypoint to another. In these situations, you can add a Recovery behavior to recover from that condition before proceeding with the objective. To illustrate how to do this, you’re going to modify your objective to add a Fallback behavior and switch into Teleop mode for the user to move the robot.

To begin, select the Build tab on the top menu bar. Then, select the My Pick and Place objective from the objectives list on the left side, and the Edit button on the top right.

Use the Search bar to find the AlwaysFailure in the list of behaviors, and drag it into the objective. Connect the input of the AlwaysFailure behavior to the bottom of the sequence.

Use the Run button on the top right toolbar to run the objective. You should see the objective run and then fail. This is expected since we added the AlwaysFail behavior. You’ll see the status has changed to Objective failed and a pop-up will appear with an error message on the right side.

Next, you’re going to add a Fallback behavior to the objective to recover from this failure. Fallback behaviors allow you to execute a behavior when you encounter a failure. These behaviors are called recovery behaviors.

You’re going to use Request Teleoperation as a recovery behavior. Choose Edit from the top right corner to edit the objective. Use the search bar to find the Fallback behavior, then drag it to the editor and add it to the sequence.

Use the Delete key to delete the existing connection between the Sequence node and the Always Failure node. Then connect the Fallback between those nodes.

Use the search bar again to find the Request Teleoperation behavior, and add that below the AlwaysFailure.

Set the enable_user_reaction value to true and add a user_interaction_prompt for the user in the Request Teleoperation behavior.

Now run the objective again. This time after the cube is placed, you should see that the Teleoperate menus appear and you can teleop the robot, as you learned in this tutorial. When you finish, use the Success button to end the objective.

Summary

You’ve now completed this tutorial. In this tutorial, you learned how to:

1. Run built-in objectives
2. Teleoperate a robot
3. Create a waypoint
4. Create a new objective
5. Modify an objective
6. Add a recovery behavior to an objective

Congratulations, you’re now ready to move to the next tutorial!