Tutorial 1 - Intro & Basic Usage
π Duration: 1-2 hours
πͺ Level: Beginner
Tutorial Series Overviewβ
Welcome to MoveIt Pro's tutorial series for learning how to use MoveIt Pro. We recommend taking these in our monthly live training courses, where two PickNik engineers are available to answer questions and help you get unstuck if anything comes up.
Training Goalsβ
Tutorials 1β4 build on each other. Tutorial 5 teaches the manual development workflow as an alternative to Tutorial 2:
- Tutorial 1 gives you a general beginner's overview of the platform using our optional, yet comprehensive, user interface.
- Tutorial 2 teaches the recommended development approach, using an AI agent and the MoveIt Pro MCP server to create a robot configuration package, build a Behavior plugin, author an Objective, and run it through MCP.
- Tutorial 3 teaches more advanced Behavior Tree concepts with Subtrees and using more computer vision and machine learning.
- Tutorial 4 teaches advanced motion planning and control concepts, including the many types of dynamic trajectory generation, collision checking, and Cartesian control.
- Tutorial 5 teaches the manual approach to robot configuration packages, custom C++ Behaviors, and Runtime SDK integration using manual CLI and code-editing steps without MCP or an AI agent.
Tutorial 1 Basic Usage - Overviewβ
This tutorial is designed to be your first hands-on introduction to the platform, giving you a general overview of MoveIt Pro, including how to:
- Run and edit robot applications (Objectives) built with Behavior Trees
- Use teleoperation to manually control the robot and create waypoints
- Build your own Objective from scratch
- Set up keep-out zones for motion planning
- Use AprilTag-based perception for object detection and grasping
An Objective is basically an application in MoveIt Pro, though it can also be a lower-level function (a Subtree) that is then combined into a higher-level Objective. Objectives are created using Behavior Trees, which are similar to state machines. Behavior Trees are composed of Behaviors, which are the low-level nodes or skills. An example of a Behavior is opening a gripper or moving to a pose. To learn more about Behaviors, see the Behaviors Concepts Page.
Pre-reqsβ
For a local Runtime, install MoveIt Pro before continuing. That guide covers both the desktop app and the Runtime, and walks through connecting the two. We assume you have some familiarity with robotic arms.
Start MoveIt Proβ
No launch command or desktop app is necessary. Open the application using the browser link provided for your Cloud deployment, then continue below. Cloud runs in simulation only and cannot be connected to robot hardware; it is intended for testing and training.
For a self-hosted Runtime:
-
On the Runtime computer, start the Runtime if you haven't already:
moveit_pro run -c lab_sim -
Open the MoveIt Pro desktop app and select Set Up on the discovered Runtime.
-
Enter the active frontend key if prompted, then select Save & Connect.
If the Runtime is not discovered, select Add Connection and enter its hostname or IP address.
Once MoveIt Pro is loaded, you should see a UI similar to the following, though the exact view pane layout may vary as it is customizable:

In this beginner tutorial we are showing you how to do everything from the UI. However, MoveIt Pro ships with a powerful API that allows you to start, stop, and monitor Objectives without the UI.
Exploring the User Interfaceβ
Workspace Modesβ
The Mode menu in the top left is how you move between the four workspaces MoveIt Pro provides. Every task in this tutorial starts by picking one, so open the menu now to see them.
Operate holds the two modes you drive the robot from:
- Objectives β browse, run, and edit Objectives. This is where you start, and where most of this tutorial happens.
- Teleoperation β manually jog the robot and save waypoints.
Tools holds two workspaces for inspecting and shaping what the robot plans against:
- Planning Scene Editor β add and edit collision objects such as keep-out zones.
- Task Constructor Debugger β inspect why an MTC planning pipeline chose the solution it did. See Debug MTC Pipelines.
Press M to open this menu from anywhere. The mode you are in is also part of the URL, so it survives a page refresh.
Customizing the Panesβ
The pane dropdown selects its content, Settings contains the layout presets, and the pane's right-click menu splits or removes it:



Try it yourself:
- Use the dropdown controls in the top left of a view pane to change its content and settings.
- Open the dropdown menu in the top left corner of the bottom right pane and switch the pane to
/wrist_camera/color. This will allow you to see the camera view of the camera mounted on the robot's wrist.
Next: Change the number of panes by opening Settings from the top navigation bar. In the View section, choose a preset under Layout. The layouts run from one pane up to six, ordered by pane count β pick the three-pane layout.
Right-click any view pane to split it horizontally or vertically, or remove it. The presets are starting points for custom layouts.
To learn more about each pane, check out the About the User Interface page.
Run your first Objective: Scanning the Environmentβ
The 3D Visualizer already shows the wrist camera's live point-cloud stream in lab_sim. Run the Scan Scene - Multiple Point Clouds Objective to capture several views, merge them, and display a frozen point-cloud snapshot covering more of the table. Neither this snapshot nor the live stream adds collision geometry by itself; planning-scene update Behaviors make sensor data available to the motion planner. There are two ways to find the scan Objective:
- Scroll the Objectives sidebar on the left. Objectives are grouped by category.
- Or type the name into the search box at the top of that sidebar, which is faster once you know what you are looking for.
You can also run it without leaving the keyboard. While the search box has focus, press ArrowDown or ArrowUp to highlight a result, then Enter to ask to run it. If the search found only one Objective, Enter asks to run it right away. Enter only opens a confirmation; press Y or click Run Objective to start it, E to edit it instead, or N or Escape to cancel.
Once you find it, click the play icon next to Scan Scene - Multiple Point Clouds to run the Objective, and you should see:
To inspect the captured point-cloud snapshot by itself, open View in the 3D Visualizer and look under Display. Keep Point Cloud Snapshot enabled and disable Sensor Streaming. To hide only the wrist camera, click the settings icon next to Sensor Streaming and turn off that source in the sidebar. Re-enable Sensor Streaming when you want the live sensor view back.
The planning scene is a representation of the robot's environment, including the robot itself, obstacles, and other objects. It is used by motion planners to determine valid paths and trajectories for the robot.
Obstacles can be added manually from the UI (keep-out zones), in Behavior Trees via planning scene manipulation Behaviors, or automatically from perception data. Tutorial 3's perception overview introduces the available approaches.
Running an example lab applicationβ
Open the left Objectives sidebar if it's not already open, then find the Move Flasks to Burners Objective β searching for it is quickest.
Close and reopen the sidebar with the Toggle Sidebar button at the far left of the toolbar β useful on a small screen. By default the sidebar closes when an Objective starts and reopens when it ends. To keep it where you leave it, open Settings and turn off Automatically open/close the sidebar when an Objective starts/ends under Preferences.
The pin icon next to Favorites is a separate control: it pins the Favorites section to the top of the sidebar so it stays visible while the rest of the list scrolls.
Click its play icon to run the Objective. You should see the robot arm pick up both flasks on the left side of the workspace and move them to the burner on the right.
Clearing the Planning Sceneβ
The captured snapshot is now out of date because we moved the flasks. The live point-cloud stream continues to update, but the frozen snapshot does not. Run Clear Snapshot to remove the displayed snapshots. It also clears any Octomap collision geometry previously added by another Objective; live streaming is unchanged.
Run Scan Scene - Multiple Point Clouds again to capture the flasks in their new locations and refresh the displayed snapshot.
Stop looping Objectivesβ
Some applications will run forever, until you tell it to stop. The 3 Waypoints Pick and Place Objective is a simple example like that - Run it now.
You should see it pick a small medicine bottle using hard-coded waypoints, repeating in a loop. This is open-loop, dead-reckoning motion: the waypoints are fixed and nothing re-checks where the bottle actually is, so any error in one pass carries straight into the next. Within the first few passes, the gripper will lose its grip and drop the bottle.
Let it run until that happens. Notice the loop keeps going with an empty gripper β nothing told it the pick failed. Then use the Stop Objective button on the top left to stop it.
That's expected, not a malfunction β hard-coded waypoints can't tell whether a pick succeeded. You'll see how detection addresses this later in this tutorial, and build your own in Tutorial 3.
In MoveIt Pro 10.2 and newer, optionally search the sidebar for 3 Waypoints Pick and Place with AprilTags. This Objective re-detects the bottle before every pick to avoid accumulating position errors between picks. On 10.0 or 10.1, skip this optional example and continue with the tutorial.
If you stopped the Objective while the robot was still holding the medicine bottle, simply run the Open Gripper Objective to release it.
Resetting the Simulationβ
What if we want to start over? Between the last few Objectives the robot has ended up in a different pose, the flasks are on the burners instead of their original positions, and the medicine bottle may not be where it started. This is a common situation during development when you want to iterate on an Objective multiple times.
Simulation reset is built into MoveIt Pro - simply click the gear icon menu in the top right and select Reset Simulation. This will restore all objects to their original positions.
To move the robot back to its original pose without resetting the entire simulation, you could instead run the Look at Table Objective.
This is an example of a very simple Behavior Tree that commands the robot to a named waypoint.
Using the Wrist Depth Cameraβ
Many robots have multiple cameras, and MoveIt Pro can visualize all of them. The camera feed from the wrist camera can be used for close up manipulation tasks, and is a common tool used in many Objectives.
Run the Take Wrist Camera Snapshot Objective. The live stream follows the wrist camera continuously, while the snapshot freezes one observation for display, so a much smaller captured area appears alongside the live stream. This Objective does not update the planner's collision geometry.


Adding Keep-out Zonesβ
Let's make it harder, or maybe impossible, for the robot to plan an automatic trajectory between waypoints. We can do that by adding collision objects, or keep-out zones.
In motion planning, a collision object represents a physical item in the environment that the robot must avoid during motion. These objects are typically defined by their shape, size, and position in the planning scene and can include things like tables, walls, tools, or even other robots. Collision objects are used by the motion planner to ensure that generated paths are free of collisions, allowing the robot to move safely and efficiently through its workspace.
A keep-out zone is a virtual collision object you add to the planning scene through the UI. The motion planner treats it as a real obstacle, even though nothing physical is there. They're great for forcing failures in tutorials, defining no-go regions around the robot, or modeling fixtures the robot doesn't have a sensor for. See the Keep-Out Zones guide for the full reference.
Keep-out zones live in their own workspace. Open the Mode menu in the top left and select Planning Scene Editor.
Click the + Keep-Out Zone button in the sidebar. You'll see a green wall appear in front of the robot with its interactive marker active. The wall has an outline because it is currently selected. Click anywhere outside the wall in the 3D Visualizer to deselect it, and the outline will disappear.
Re-run the Move Flasks to Burners Objective and observe the robot planning around the green wall. This is collision-aware motion planning! The robot automatically finds a collision-free path to reach its goal while avoiding the keep-out zone.
Next, let's make it impossible. Open the Mode menu and select Planning Scene Editor again, then click the wall to select it and show its interactive marker:
You could drag the arrows of the interactive marker to move the wall, but for a repeatable result use the Size, Position, and Rotation controls in the sidebar, which appear while the wall is selected. Each value has a drag control and an editable number field. Enter these values to lay the wall flat as a lid over the flasks:
| Field | Value |
|---|---|
| Pitch (degrees) | 90 |
| X | -0.5 |
| Y | 0.5 |
| Z | 0.65 |
Leave the size and the other fields at their defaults. The zone should now hover just above the flasks:
Re-run Move Flasks to Burners. The grasp pose for the first flask now sits inside the zone, so the Objective fails before the arm moves, and the 3D Visualizer shows the robot in the colliding pose with red spheres on the contact points. They stay until the next run (with the default Auto-Clear on Run setting); to remove them sooner, open the View menu in the 3D Visualizer and select Clear All Markers. Leave the zone in place: the next two sections use this failure.
Alerts & Error Handlingβ
With the zone lying over the flasks, Move Flasks to Burners fails with a PlanToJointGoal Error: the goal joint positions for the first pick collide with the keep-out zone, so the planner refuses them.
When errors occur, MoveIt Pro logs them to the Alert Sidebar. You can open it by clicking the bell icon in the top right of the toolbar:
This opens a sidebar showing a history of all Objective runs, including successes and failures with timestamps:
You can also filter alerts by severity and source (Runtime, Editor) using the Alert Settings icon in the top right of the sidebar. Severity is set separately for each surface: Toasts decides which alerts appear as a toast message in the corner of the screen, and Alert Sidebar decides which ones this list shows. Toasts are limited to Errors by default, and Debug is off on both. Both sources are on by default, so the Alert Sidebar lists Runtime and Editor alerts together. Whenever a severity the sidebar lists does not pop up, the bell icon carries an unread count instead, which turns red when any unread alert is an error. The count is independent of these filters β it counts every alert that arrived since you last opened the Alert Sidebar β and opening the sidebar clears it.
The bottom of the Alert Sidebar shows the directory on the Runtime host where the Runtime saves its logs, with a button to copy the path. It is hidden when the Runtime does not report one.
The Alert Sidebar is a useful tool for monitoring the status of your Objectives and diagnosing issues during development.
Finding the Failure in the Behavior Treeβ
The Alert Sidebar tells you what went wrong. The Behavior Tree tells you where.
With the failed run still on screen, click the Edit button in the top left of the menu bar. The editor opens on the branch that failed: every node on the path from the root down to the failed node gets a red border, the failed node itself pulses, and the editor opens centered on the node you were looking at in Run, at the same zoom, so you do not have to hunt through a large tree.
Here the outline runs from the Sequence to the first Constrained Pick and Place Subtree (Flask 1). The pick failed inside that Subtree, so the outline initially stops at the deepest visible node. Click the Expand button (the arrows icon) on the Subtree node, and the outline extends through the newly visible branch until it reaches the failed Behavior. The pencil icon next to it opens the Subtree as its own Objective, which you will use later in this tutorial.
The highlight stays on while you edit, also after you leave and reopen the editor, so you can fix the tree with the failure still in view. It clears when you run the Objective again; click Clear status at the top of the canvas to remove it sooner.
This also works for Objectives you did not write. When a shipped example fails on your robot, run it, click Edit, and follow the red border straight to the Behavior that needs attention.
Now remove the zone. Open the Mode menu and select Planning Scene Editor, click the zone to select it, and remove it by pressing Delete on your keyboard, or with the Delete button in the sidebar, which only appears while an object is selected. Then switch the Mode menu back to Objectives.
Editing the lab applicationβ
Behavior Tree Basicsβ
A Behavior Tree is built from nodes. Leaf Behaviors perform actions or evaluate conditions, while control and decorator nodes manage how other nodes run. Combining them hierarchically creates modular, reusable robot applications. See the Behavior Tree concepts page for a deeper reference.
A Behavior Tree is a hierarchical, modular way to model decision-making logic. MoveIt Pro provides a visual, node-based editor, stores the tree as XML, and implements the underlying Behaviors as customizable C++ plugins in ROS packages.
Node Typesβ
Behavior Trees have two broad node families: internal nodes that decide how their children run and leaf nodes that do the work. The editor uses an icon and color for each node type:
Internal nodes have one or more children and control how those children run:
- Control nodes run their children in a set pattern. A Sequence runs them one after another, while a Parallel runs several at the same time.
- Decorator nodes have exactly one child and change how it behaves. For example, an Inverter flips its child's result, while a Delay waits before running its child.
Leaf nodes sit at the bottom of the tree and do the work:
- Action nodes perform a task, such as moving the arm or opening the gripper.
- Condition nodes check something, such as whether the gripper is holding an object, and answer yes or no.
Every time the tree runs, a signal called a "tick" travels down from the root. A ticked node reports one of these statuses:
- SUCCESS
- FAILURE
- RUNNING β the task has started but needs more time to finish.
Condition nodes only report SUCCESS or FAILURE because a check is answered immediately.
Adding a Breakpointβ
Now let's learn how to pause an Objective at a specific point in its Behavior Tree using a breakpoint. This is useful for debugging or stepping through an Objective one Behavior at a time.
First, we need to open the Move Flasks to Burners Objective in edit mode.
Open the left Objectives sidebar if it's not already open, then select the Objective and click the Edit button on the top left of the menu bar.
You are now in edit mode, indicated by the blue bar at the top of the screen showing the Objective name.
You should see the full Behavior Tree for Move Flasks to Burners:
Select the second Constrained Pick and Place node (Flask 2), immediately after Flask 1. Open More Actions in the action buttons above it and click Add Breakpoint.
The node now shows a Breakpoint label under its title, and execution will pause after Flask 1, just before Flask 2 runs.
In MoveIt Pro 10.3 and newer, the Runtime holds the breakpoint, so the Behavior Tree and the Objective XML do not change: no Sequence or other node is added, and there is nothing to save. The breakpoint stays set across runs until you remove it or the Runtime restarts.
In MoveIt Pro 10.2, the Breakpoint node action wraps the selected node in a Sequence with a BreakpointSubscriber Behavior in front of it. That breakpoint changes the Objective XML and is saved with the Objective. In MoveIt Pro 10.1, the action is not available; add a BreakpointSubscriber Behavior where you want execution to pause instead.
Now run the Objective. Execution stops before Flask 2, the node's label changes to a yellow Paused here, and a Breakpoint prompt appears naming the held node. Resume carries on with that node, Fail Node makes it report FAILURE without running, and Stop cancels the Objective. A short beep plays so you notice the pause even when you are looking elsewhere.
With the Objective paused you can inspect the robot, the planning scene, and the blackboard before deciding which button to press.
A breakpoint holds the Objective, not the robot. A trajectory already sent to the controller runs to its end. Use Stop to halt the robot.
When the run is over, click Edit to open the Objective in edit mode again, select Flask 2, and click Remove Breakpoint in More Actions, so later runs in this tutorial do not pause there. Do not use Remove Breakpoint to continue a paused run: it releases the held node as a failure. Use Resume instead.
To make a pause part of the Objective itself, such as an operator check that everyone who runs it should hit, add the BreakpointSubscriber Behavior to the tree instead. See the Breakpoints guide for both kinds of breakpoint.
Turning on the Burnerβ
Now let's build towards a full lab automation application. After moving the flasks to the burners, we also want to turn on the burner. Rather than building everything from scratch, we can reuse existing Objectives as Subtrees within our Behavior Tree.
Subtrees are modular components within a larger Behavior Tree that encapsulate a set of actions or decision logic. By allowing one Behavior Tree to reference another, Subtrees enable greater reusability, organization, and maintainability in complex robotic applications. They help abstract away lower-level details, reduce duplication of nodes, and make it easier to manage common sequences or Behaviors across different Objectives. In MoveIt Pro, Subtrees are especially useful for structuring sophisticated task plans in a clean and scalable way.
From the left Objective sidebar, run the Push Button Objective to see the robot attempt to push the button on the burner.
This Objective uses admittance/force control to push the button with an exact force.
Now let's add this Push Button Objective as a Subtree into our flasks application.
Go back to editing the Move Flasks to Burners Objective.
In MoveIt Pro, the recommended way to add Behaviors is using the blue plus button on Control and Decorator nodes in the Behavior Tree. Clicking it opens a search dialog where you can quickly find and add any Behavior.
In MoveIt Pro 10.3 and newer, the button is a dashed Add Behavior placeholder shown below the node's last child, exactly where the new Behavior will be added; on a node without children it takes the place of the first child. A Decorator accepts a single child, so its placeholder disappears once it has one; an empty Objective shows one next to its root.
You can also scroll through the full library of Behaviors on the left sidebar, or use the search bar at the top. However, the blue plus button is the fastest and most common workflow.
Click the Add Behavior placeholder at the end of the Sequence node:
Search for Push Button and click the result to add it.
Congratulations β you've just built a longer, more capable lab automation application that moves flasks and turns on the burner!
Disabling/Skipping Behaviorsβ
As you iterate on an application, you often want to skip parts that are slow or unrelated to what you're currently testing. MoveIt Pro lets you disable nodes in your Behavior Tree so they're skipped at runtime, without deleting them.
The Push Button Subtree we just added uses admittance control to press the burner with a precise force β accurate, but slow to watch on every run.
Let's skip it while we iterate on the rest of the application:
- Make sure you are still editing the
Move Flasks to BurnersObjective. - Click on the
Push ButtonSubtree node and find the icons that appear on top of the box. - Click on the Comment out (skip execution) icon.
You should see the node change colors to indicate it's disabled.
The node also gains a _skipIf port set to true, which is what gets saved to the Objective's XML.
Now run the Objective β the flasks still move to the burners, but the button-push step is skipped, letting you iterate on the flasks portion much faster.
Renaming the Objectiveβ
Now that we've customized our lab application, let's rename it to something more personal:
- Open the
Move Flasks to BurnersObjective in edit mode. - Click on the Objective name in the blue bar at the top.
- Select Details to open the Edit Objective dialog, and rename it to something like
My Lab Automation. - Click Save.
Editing the Multi-Threaded Push Button Subtreeβ
One of the powerful features of MoveIt Pro's Behavior Tree engine is its support for multi-threaded parallel execution. This means multiple Behaviors can run simultaneously, which is essential for real-world robotics where you often need to perform an action while monitoring safety conditions at the same time.
Let's explore the Push Button Subtree we added earlier and modify how hard it pushes the burner button by changing the force threshold parameter.
To start, click on the pencil icon on the Push Button Subtree node.
You should see a Parallel node near the core of the tree.
The Parallel node ticks all of its children concurrently. It can be configured with different success/failure policies β for example, succeeding when all children succeed, or stopping when any child fails. This is different from a Sequence, which runs children one at a time.
In this example, the Parallel node is running two things simultaneously:
- A velocity command that pushes the robot's end effector forward along the z-axis toward the burner button.
- A force monitor that continuously checks the force sensor and stops the motion if the detected force exceeds a threshold (20 N by default).
A Timeout decorator bounds this parallel push to 10 seconds. If the force threshold is not reached, it halts both branches and the Objective fails. The timeout applies to the push, not the approach or return motion.
This pattern is common in contact-rich tasks like pushing buttons, inserting pegs, or polishing surfaces β anywhere you need the robot to move while respecting force limits. Without parallelism, you would have to alternate between moving and checking, which is slower and less safe.
Try it yourself:
- Find the
force_thresholdparameter in theForceExceedsThresholdBehavior and increase it to 1000 N.
- Run the
Push ButtonObjective again. At 1000 N, the burner can move without the measured force reaching the threshold, so the push ends with a timeout failure. - Observe the burner moving while the robot pushes. Increasing a force threshold does not guarantee that contact will produce that force; the timeout keeps the robot from pushing indefinitely.
- Set the value back to 20 N, reset the simulation, and run
Push Buttonagain. The force monitor should now stop the push before the timeout, allowing the Objective to complete successfully.
MoveIt Pro's entire runtime is built to support multi-threading, so you can safely use Parallel nodes throughout your Objectives without worrying about thread safety issues in the built-in Behaviors.
There are different schools of thought on how to structure Behavior Trees. In this tutorial, we use many small, atomic Behaviors β each performing a single action like opening a gripper or moving to a waypoint. This granular approach makes it easy to learn, debug, and reuse individual skills. However, MoveIt Pro also supports building larger, self-contained subsystems with fewer nodes overall, which can be more practical for production applications. Both approaches are valid, and you can mix them as needed.
Perception-Based Picking with AprilTagsβ
MoveIt Pro supports multiple perception approaches for autonomous manipulation. In this section, we'll use AprilTag fiducial markers for precise 6-DOF pose estimation to autonomously pick medicine bottles from the workspace.
What are AprilTags?β
AprilTags are a type of visual fiducial marker β small, printed patterns that can be detected by a camera to estimate the precise 3D position and orientation (6 degrees of freedom) of a tagged object. They are widely used in robotics because they are fast to detect, robust to lighting changes, and provide accurate pose estimates.
In our lab simulation, each medicine bottle has an AprilTag attached to its cap. The robot uses its wrist camera to detect these tags, compute a grasp pose, and pick up the bottles.
Run the Pick All Bottles with AprilTags Objectiveβ
Open the left Objectives sidebar and search for Pick All Bottles with AprilTags. Click the play icon to run it.
This Objective continuously loops, detecting and picking one bottle per cycle until no more AprilTags are found in the workspace.
How It Worksβ
Open the Objective in edit mode to see its Behavior Tree structure. Each cycle detects AprilTags and checks detection_count before picking. Successful detection with zero tags completes the Objective; detection or motion failures fail it. Each pick-and-place cycle is composed of these main steps:
Detect AprilTags on TableSubtree β Opens the gripper, moves to a top-down view of the table, captures a wrist camera image, and returns the AprilTag detections and their count.Pick from AprilTag DetectionsSubtree β When the count is positive, computes a grasp pose and uses MoveIt Task Constructor (MTC) to plan and execute the approach and close the gripper.- Retract β Plans and executes a Cartesian retract path away from the grasp location.
- Place in Tray β Moves to a precomputed drop-off pose above the tray, then uses velocity/force control to lower until contact.
- Release and Retract β Opens the gripper and retracts upward before the next cycle.
When the count is zero, the final Script sets bottles_remaining to false, ending the loop successfully.
Open this Objective in edit mode and click through each Subtree to see exactly which Behaviors are used. This is a great example of combining perception with dynamic motion planning, rather than relying on hard-coded waypoints.
Data Sharing through Ports and the Blackboardβ
Behaviors share data through the blackboard, a key/value store. Input ports read values, output ports write them, and bidirectional ports do both. A port can hold a direct value or refer to a blackboard entry with braces, such as {grasp_pose}. An input that reads an entry needs another Behavior or an Objective input to supply its value first.
The Blackboard pane shows these values while an Objective runs. Tutorial 3's blackboard inspection exercise follows a detected bottle pose through a Subtree and explains how port remapping shares it with the parent Objective.
Building your own Objectiveβ
Create New Objectiveβ
Now we're ready to build our own custom Objective.
We're going to build a simple pick demo using a custom waypoint we'll create called Pick right bottle, along with some existing waypoints that ship with lab_sim.
To begin, select the + button to the left and click New Objective. This opens the Create New Objective dialog.
Tutorial 2 shows how an AI agent creates and tests a custom C++ Behavior through MCP.
Enter My Pick and Place as the name.
For the category, create a new one called My Tutorials.
Categories are yours to define β on a real project you would typically use your company or organization's name. We use My Tutorials throughout these tutorials so the Objectives you create stay together, separate from the ones that ship with MoveIt Pro.
You can also provide an optional description, which we recommend. Try A simple pick demo using hardcoded waypoints.
The dialog has two more fields you can leave alone: Robot Config Package, already filled in with
the config you are running (lab_sim), and a Mark as favorite checkbox.
Leave the checkbox Subtree-only Objective unchecked. It specifies if an Objective can be executed. If an Objective is marked for usage as a Subtree only, then it is non-runnable, but can instead be used inside another Objective in the same way we use Behaviors.
Click the Create button. You should see a new, mostly empty Objective.
Adding Behaviorsβ
New Objectives are instantiated with a simple Behavior Tree using the AlwaysSuccess Behavior. Without this AlwaysSuccess Behavior, it would be considered an invalid / malformed tree and would throw warnings.
Delete the AlwaysSuccess Behavior by clicking on it and either selecting the popup delete icon, or by pressing the Delete button on your keyboard.
Open Settings and select Keyboard Shortcuts for the full in-app reference.
The MoveIt Pro Behavior Tree editor includes an Undo/Redo button that can protect you from accidental deletions or other mistakes. You can find it in the icon bar centered along the bottom of the editor canvas. Try using this feature to undo and redo the deletion.
Now we are going to add our first Behavior.
Using the blue plus button, add the Move to Named Waypoint Behavior to our new tree.
Click on the Behavior and scroll to the bottom of the parameter sidebar to find the waypoint_name port dropdown selector.
Use the dropdown list to set the waypoint name to Look at Table.
Each Behavior in an Objective has a name attribute that can be useful for identification. Let's set this Behavior's name to Look at Table.
By default, the Behavior Tree editor shows two ports per node, to keep nodes compact. The button in the bottom icon bar opens a Port Visibility menu that sets how many to show: No ports, 1 port through 5 ports, or All ports. The screenshot above uses All ports.
In MoveIt Pro 10.3 and newer, nodes list input ports first, then input/output ports, then output ports. Click the star next to a port in the node's edit menu to mark it as a favorite. Favorite ports are listed first and stay visible at every Port Visibility setting, on every instance of that Behavior. Favorites are saved in port_favorites.xml at the root of the config package you are running (the directory that holds package.xml), and apply on top of the favorites of the packages that provide its Objective libraries. Favorites you set therefore never change another config that shares those packages. The editor generates this file, so comments and manual formatting in it are not preserved. MoveIt Pro ships default favorites for its core motion Subtrees, such as waypoint_name on Move to Named Waypoint and target_pose on Move to Cartesian Pose. To remove a default favorite, click its star: your config stores the change, and the shipped defaults stay unchanged.
Next, add a Clear Snapshot Behavior, then add a Take Wrist Camera Snapshot Behavior to the tree.
These Subtrees demonstrate the snapshot workflow: clear previously displayed snapshots and any Octomap, then capture the wrist camera's current depth data for display. The live point-cloud stream remains independent, and a snapshot does not become collision geometry until a planning-scene update Behavior adds it.
MoveIt Pro ships with 200+ Behaviors from the Behaviors Hub for all domains of robotics: motion planning, machine learning, inverse kinematics, Cartesian planning, real-time control, grasping, task planning, human in the loop teleop, and more. You are also encouraged to build your own custom Behaviors and plugins, which allows you to incorporate other ROS packages or third party capabilities with MoveIt Pro for unlimited potential.
We are going to add a few more Behaviors to complete the Objective. Below is a complete list of steps, including the ones we've already added in the above.
Full Behavior Sequenceβ
Full sequence β each top-level bullet is a Behavior, with the port values to set indented below it:
Move to Named Waypointwaypoint_name:Look at Table
Clear SnapshotTake Wrist Camera SnapshotOpen Gripper- Make sure the gripper is open before we pick.
Move to Named Waypointwaypoint_name:Pick right bottleβ a custom waypoint we will create below.
Close GripperMove to Named Waypointwaypoint_name:Homeβ returns the robot to a safe home position while holding the medicine bottle.
Creating a Waypoint with Teleoperationβ
To complete our Objective, we need a waypoint for where the robot should pick up the medicine bottle. We'll create this using MoveIt Pro's teleoperation β manual control of the robot through the UI.
MoveIt Pro provides many types of manual control for when a robot needs to be set up, diagnosed, or recovered. Teleoperation is also a key tool for AI data collection β by recording human demonstrations, you can generate training data for machine learning models.
| Method | Best For | Precision | Setup Required |
|---|---|---|---|
| Interactive Marker | Visual positioning, trajectory preview | Medium | None |
| Pose Teleop | Cartesian movements, end-effector positioning | High | None |
| Joint Teleop | Joint-specific movements, singularity avoidance | High | None |
| Waypoints | Returning to a pose you already saved | Exact | A saved waypoint |
| Mobile Base | Driving a mobile base forward, back, and around | Medium | moveit_params.base_jog |
| Game Controllers | Familiar interface to many users | Medium | Plug in USB |
| Meta Quest VR | Intuitive hardware-based teleop | Medium | Setup Guide |
For more details on each mode, see the UI-Based Teleoperation Guide. For hardware-based teleop, see the Gamepad Setup Guide or Meta Quest Setup Guide.
Open the Mode menu in the top left and select Teleoperation.
At the top of the left sidebar you'll find four tabs β Pose, Joint, IMarker, and Waypoints β one per manual control method. Click through them to see how the controls change. The first three correspond to the Pose, Joint, and Interactive Marker rows in the table above; Waypoints drives the robot to a waypoint you have already saved rather than jogging it freehand.
Two controls in this sidebar are worth knowing about before you start jogging the arm:
- Require Approval stages your motion on a translucent "ghost" robot first and waits for you to click Approve before the real arm moves. Available in the Joint and IMarker tabs.
- Game Controllers, at the bottom of the Pose and IMarker tabs, is where a USB gamepad or a Meta Quest is connected.
Waypoints are saved joint states that can be reused later in Objectives or standalone.
Stop any running Objective, reset the simulation to return the bottles to their initial positions, and run Look at Table. Then return to Teleoperation to position the gripper and save a waypoint. Use the rightmost of the three square medicine bottles, highlighted below:
- Set the largest view pane to the
/wrist_camera/colorcamera view. - Open the gripper using the bottom buttons.
- Use Pose teleoperation to position the open fingers on opposite sides of the bottle's cap, rather than around the square body. Approach from above with a slight tilt and keep the cap centered between the finger pads.
For a repeatable starting pose in the reset lab_sim scene, select Joint, choose Degrees, and enable Require Approval. Enter each value below and press Enter after each field. The Approve Trajectory? prompt appears after the first field; keep entering the remaining values, inspect the preview, then click Approve. These values are specific to this simulated robot and scene, not a hardware grasp recipe.
| Joint | Target |
|---|---|
elbow_joint | 85.8779Β° |
linear_rail_joint | -0.4519 m |
shoulder_lift_joint | -45.1888Β° |
shoulder_pan_joint | 42.6687Β° |
wrist_1_joint | -145.0379Β° |
wrist_2_joint | -85.5786Β° |
wrist_3_joint | -106.7807Β° |
Keep the gripper open while saving Pick right bottle; the Objective closes it after moving to the waypoint.
Once the cap is between the fingers, save the robot's state by creating a waypoint:
- Open the + menu in the top left and select New Waypoint. The menu offers New Objective, New Waypoint, and New Behavior in every mode.
The dialog will open.
-
Name your new waypoint
Pick right bottle.tipYou can adjust which parts of the robot to save as a waypoint, by clicking the Change Planning Group button when adding a new waypoint. However, this is an advanced feature you do not need to worry about for now.
-
Click Create to finish creating your new waypoint.
-
Click the Stop Objective button in the top toolbar to stop the
TeleoperateObjective. The workspace stays in Teleoperation, so switch the Mode menu back to Objectives to return to the default view.
Finishing the Objectiveβ
Now go back to editing your My Pick and Place Objective and finish adding the remaining Behaviors from the sequence above.
Make sure to set the waypoint_name for each Move to Named Waypoint Behavior, including your newly created Pick right bottle waypoint.
To save time, you can copy a Behavior instead of adding it again. Select the Behavior, then click the blue Duplicate icon in the toolbar above the node.

Take advantage of the build navigation buttons located on the bottom of the pane. The tree re-lays out automatically as you drag Behaviors into place, keeping it clean and organized.
Your completed Objective should look similar to this:
Run the Objective to make sure it works.
We should see the robot pick up the right medicine bottle! We realize this is a very basic example, but this is our intro tutorial. MoveIt Pro can do much more advanced applications that involve computer vision, reasoning, and machine learning.
Grouping Behaviors under a Sequenceβ
Your Objective is a flat list of seven Behaviors, and it is already hard to see where the pick starts and ends. The editor can wrap neighboring Behaviors in a new Sequence for you, so the tree gains structure without rebuilding anything.
- Click
Open Gripper, then hold Ctrl (Cmd on macOS) and click theMove to Named WaypointforPick right bottleandClose Gripper. All three are now selected, and a Group Under Sequence button appears above the tree.
- Press Ctrl+G (Cmd+G on macOS), or click that button. The editor inserts a new
Sequencewhere the three Behaviors were and moves them under it, in the same order. - The new
Sequenceis selected, so name it now: typePickin the Name / Comment field of the sidebar and press Tab.
Run the Objective again. It behaves exactly as before, because a Sequence runs its children one after another, which is what the top-level Sequence was already doing. What changed is that the tree now says what those three Behaviors are for, and you can collapse, duplicate, or comment out the pick as one unit.
Grouping needs two or more nodes that share the same parent. If the button does not appear, check that everything you selected sits under the same Sequence. The sidebar also offers Convert to Subtree for a selected Sequence, which turns it into an Objective of its own. Tutorial 3 covers Subtrees in depth.
You can also edit Objectives using your favorite IDE / code editor, as all Behavior Trees are saved to file in plain text XML format.
The Behavior Tree you just created is saved in the Runtime's workspace. To list every Objective in lab_sim, open a view pane's dropdown, choose Docker Container under Terminal (beta), and run:
ls -1 ~/user_ws/src/lab_sim/objectives
On a self-hosted Runtime, the same files are on the host under the workspace you configured, which is ~/moveit_pro/moveit_pro_example_ws by default.
Example contents of objectives folder
3_waypoint_pick_and_place.xml
add_point_cloud_to_vector.xml
add_poses_to_mtc_task.xml
add_waypoints_to_mtc_task.xml
apriltag_pick_object.xml
clear_previous_obstacles.xml
close_gripper.xml
constrained_pick_and_place_subtree.xml
create_pose_vector.xml
cycle_between_waypoints.xml
force_relaxation.xml
fuse_multiple_views_subtree.xml
And many more.
Great job on creating your first Objective!
Using SAM3 AI Modelβ
Reset the simulation so the bottles return to their original positions. Click the gear icon in the top right and select Reset Simulation, as you did earlier in this tutorial.
Run the ML Find Bottles on Table Objectiveβ
This Objective runs an ML model that requires a decent GPU and uses a significant amount of VRAM. It may not work on all machines, or may run very slowly on systems without a dedicated GPU.
It also needs the moveit_pro_sam3 package, which is only built once you accept Meta's SAM License. If you declined it, or installed without a terminal to prompt you, the Objective fails to load the model β see Accepting the SAM3 model license.
To see an example:
- Use the dropdown menu in a secondary view pane, such as the bottom right pane in the three-pane layout, to switch it to
/bboxes_visualization. This pane shows the image exemplar and its bounding box, and it only receives them while the Objective runs. Keep it separate from the primary pane, which the Objective switches to the masks view. The topic is not listed until the Objective has published to it once, so on a fresh Runtime run the Objective once first, then select the topic and run it again. - Open the left Objectives sidebar and search for
ML Find Bottles on Table from Image Exemplar. Click the play icon to run it. - Wait for the Objective to complete. It switches the primary view pane to
/masks_visualization, where the segmented medicine bottles appear with colored masks and confidence scores:
In Tutorial 3, we dive deeper into MoveIt Pro's ML and perception capabilities, including how to integrate your own custom ONNX models and build full ML-powered pick and place applications.
Saving your workβ
This section does not apply to MoveIt Pro Cloud users, where your work cannot be saved once the cloud session expires.
Wrapping up this tutorial, you may be curious how all these changes are stored and saved.
In this beginner tutorial we are working out of our basic moveit_pro_example_ws, which is stored on your computer at ~/moveit_pro/moveit_pro_example_ws.
Your changes are auto-saved to file, but you can also push it into the cloud using standard Git code repository practices.
To do this:
- Open a terminal explore the changes you have made:
cd ~/moveit_pro/moveit_pro_example_ws
git status # see the files you have changed
tree # explore the file structure of this repo
- Fork the moveit_pro_example_ws repo on Github, to your own account
- Connect your local workspace to your new fork
git remote set-url origin <url_of_your_new_fork>
- If you want to now save the changes:
git checkout -b my_pick_demo
git add -A
git commit -m "Adding My Pick and Place Objective"
git push origin my_pick_demo
Exploring More Robot Configsβ
Before continuing on to the next tutorial, feel free to explore other example robot configs provided in MoveIt Pro beyond lab_sim.
These example worlds showcase other hardware and various reference applications.
MoveIt Pro Cloud runs in simulation and cannot connect to robot hardware. The commands below apply to self-hosted Runtimes. On Cloud, open your instance's page in the MoveIt Pro Cloud dashboard and click Run on another configuration under Simulation; MoveIt Pro restarts with that robot configuration.
To do this:
- Bring down your current instance of MoveIt Pro with
Ctrl-C. - See all available configuration packages with the command
moveit_pro run --list. - Choose one, such as
moveit_pro run -c grinding_sim. - Within the UI, try out the various example Objectives from the sidebar.
Summaryβ
By completing this tutorial, you've built a strong foundation in using MoveIt Proβfrom running Objectives and teleoperating the robot, to creating your own pick-and-place application with keep-out zones, breakpoints, and AprilTag-based perception. You explored how to configure the user interface, use Behavior Trees for task logic, diagnose failures from the Alert Sidebar and the failed-branch highlight in the Behavior Tree editor, and integrate ML-based perception. In the next tutorial, you'll learn how to build on MoveIt Pro as a developer β including the runtime SDK, Python integrations, and how to build your own custom Behavior plugins.
Just completed our live training? We would love to hear your feedback on this course!
π Congratulations, we're now ready to move to the next tutorial!