Connect an AI Assistant with the MCP Server
The MoveIt Pro MCP server connects an AI assistant to a Runtime and its workspace. A local or SSH session can also create and validate Objectives, build and test robot configuration packages, launch the Runtime, and start or cancel existing Objectives.
Choose where the MCP client runs
| Where the AI client runs | Connection | Access |
|---|---|---|
| On the Runtime computer | Local stdio | Workspace, build, test, Runtime launch, and Objective execution. |
| On another computer, with SSH access | Stdio over SSH | The same tools, running as the SSH account. |
| On another computer, with an MCP key | Authenticated HTTPS | Inspection of Objectives, Behaviors, Waypoints, robot state, cameras, and logs. |
MoveIt Pro 10.2.0: if stdio connects with no tools, check the startup workaround.
Local and SSH sessions have the workspace owner's authority. Configure your AI client's tool approvals before allowing Objective execution: clients may auto-approve calls, and the server adds no confirmation prompt. An optional policy limits a session's tools. The shared HTTPS credential grants inspection only.
orchestrate is opt-in, because it lets an agent compose motions and run any Behavior in the catalog, not only existing Objectives. Add --orchestrate to moveit_pro mcp serve or moveit_pro mcp install to offer it and save_objective for requested program saves. That session leaves out general authoring and host lifecycle tools, get_capabilities, and run_objective, whose work an objective step does. See MoveIt Pro Orchestrator.
Each session binds to one workspace, robot configuration package, and Runtime target. Change a fixed target with mcp install or by editing the client entry, then reconnect. Tools remain visible when a provider is unavailable; get_capabilities reports what is ready and how to enable it.
Same machine: install on the Runtime computer
Run the following commands as the workspace-owning account on the computer with MoveIt Pro and Docker installed. The Runtime can be stopped when you connect. Run moveit_pro mcp serve without sudo; the launcher rejects root.
Install a client entry with an absolute workspace and an explicit robot configuration package:
moveit_pro mcp install \
--client claude-desktop \
--workspace /absolute/path/to/user_ws \
--config-package my_robot_config
Restart the AI client after registration. It starts moveit_pro mcp serve for each connection; a separate server process is unnecessary. Installation preserves unrelated client entries.
Claude Code, Codex, and Pi can register that same Runtime-local bootstrap directly.
Register it without naming a target:
claude mcp add --scope user moveit-pro -- moveit_pro mcp serve
codex mcp add moveit-pro -- moveit_pro mcp serve
pi mcp add moveit-pro -- moveit_pro mcp serve
Each session binds its target when it starts. Without target options, moveit_pro mcp serve binds the Runtime your account is running. When several are running, it binds the one whose workspace contains the directory where you started the client; when that does not pick exactly one, it fails and lists them. Pass --instance <name>, or --instance default for the unnamed deployment, to choose one. When no Runtime is running, it binds the workspace and robot configuration package the MoveIt Pro CLI is configured for. Pass --workspace and --config-package to bind a fixed target instead. Before 10.2, moveit_pro mcp serve requires --workspace.
Your AI client controls approvals; see Claude Code permissions and Codex MCP configuration.
Approve tools automatically
MoveIt Pro 10.3 adds these shortcuts for a registered server:
moveit_pro mcp auto-approve --client claude-code moveit-pro
moveit_pro mcp auto-approve --client codex moveit-pro
These commands configure automatic tool approval. run_objective can then move the robot without a prompt.
The Desktop App's Don't ask before MoveIt Pro tool calls option also replaces the same-name registration, including custom --policy options.
Remote client: connect from another computer
The HTTP server runs with the REST backend on the Runtime computer. The AI client runs on the client computer and connects with:
- URL:
https://<runtime-host>:<rest-port>/mcp - Header:
Authorization: Bearer <MOVEIT_MCP_HTTP_KEY> - TLS verification for the Runtime certificate or its issuing CA
MOVEIT_MCP_HTTP_KEY is the 64-character lowercase hexadecimal value printed by
moveit_pro mcp http-key on the Runtime computer. It is derived from, but is not
the same as, the frontend key. Do not give an AI client the frontend key: that
credential also authenticates the broader REST backend.
Streamable HTTP is the preferred transport for a remote client. It uses the
deployment's existing TLS certificate and requires
Authorization: Bearer <MOVEIT_MCP_HTTP_KEY> on every MCP request; browser
CORS preflight requests remain unauthenticated. Install or trust the MoveIt Pro
Runtime certificate as described in
Endpoint Security.
HTTP sessions have the read-only deployment tools. They do not receive local
workspace, build, or Runtime-lifecycle authority.
Start the server on the Runtime computer
There is no separate HTTP MCP process to install or start. The /mcp endpoint
starts automatically with the authenticated REST backend.
Start MoveIt Pro normally. The CLI creates and stores the deployment frontend key automatically. Print its domain-separated MCP credential on the Runtime computer when configuring an AI client:
moveit_pro mcp http-key
Rotating the frontend key also rotates this derived MCP key, so update clients after a rotation. For a direct remote connection, allow the exact Host header that the client will send before starting the Runtime:
export MOVEIT_MCP_ALLOWED_HOSTS="robot.example.com:3200"
moveit_pro run -c my_robot_config
The default REST port is 3200; named instances use their assigned REST port.
The discovery service on port 3204 reports those ports. A native MCP client
normally omits Origin. If a browser-based client sends one, also set
MOVEIT_MCP_ALLOWED_ORIGINS to its exact origin before starting the Runtime.
Configure TLS and authentication on the client computer
Verify TLS before allowing the client to send the MCP HTTP key:
-
For a direct hostname or IP address, use a certificate signed by a CA the client trusts and whose subject alternative names cover the MCP URL.
-
The generated Runtime certificate covers loopback names. For a private or tunneled development connection, copy
cert.pemto the client through an authenticated channel such as SSH, trust it only for this connection, and connect through loopback. The generated certificate is at~/.local/share/moveit_pro/tls/cert.pemby default, or underMOVEIT_TLS_CERT_DIRwhen that variable is set. For example:scp <runtime-user>@<runtime-host>:~/.local/share/moveit_pro/tls/cert.pem \/owner-only/path/runtime-cert.pemssh -N -L 13200:127.0.0.1:3200 <runtime-user>@<runtime-host>Keep the tunnel open and use
https://localhost:13200/mcp. Loopback Host values are allowed withoutMOVEIT_MCP_ALLOWED_HOSTS.
The MoveIt Pro Desktop App performs frontend-key-authenticated certificate pairing automatically for its own Electron gateway. That paired certificate, stored key, and local gateway are deliberately not exported to other processes. An MCP client on the same computer is configured separately with its own TLS trust and MCP-only bearer key. The desktop profile's Alternate hostname or IP address also applies only to Electron; set the MCP URL or SSH tunnel to the route that the MCP client should use.
Native HTTP clients must support MoveIt Pro's sessionless MCP 2026-07-28
protocol. The mcp-remote bridge shown below adapts stdio clients to that HTTP
protocol.
Generate Claude Desktop's remote entry instead of writing it by hand:
moveit_pro mcp install --client claude-desktop --url https://localhost:13200/mcp
The command prints the JSON with a <MOVEIT_MCP_HTTP_KEY> placeholder plus
the remaining setup steps, including what a read-only remote session can and
cannot do. It never writes a client file or embeds the key. Paste the output
into the client configuration on the client computer.
Use the client's credential or environment-variable support instead of storing the key in plaintext when available. If the client config contains the key, make the file owner-only.
Claude Desktop's
remote connector flow
is for internet-reachable servers using authless or OAuth authentication. It
does not use the client computer's LAN, VPN, Tailscale route, or SSH tunnel, and
Claude Desktop does not load a remote HTTP server directly from
claude_desktop_config.json. For a private Runtime protected by a MoveIt Pro
MCP bearer key, configure a local stdio-to-HTTP bridge in that file. This
example uses mcp-remote; pin and review the bridge version your organization
approves:
{
"mcpServers": {
"moveit-pro": {
"command": "npx",
"args": [
"-y",
"mcp-remote@0.1.38",
"https://localhost:13200/mcp",
"--transport",
"http-only",
"--header",
"Authorization:${MOVEIT_MCP_AUTHORIZATION}"
],
"env": {
"MOVEIT_MCP_AUTHORIZATION": "Bearer <MOVEIT_MCP_HTTP_KEY>",
"NODE_EXTRA_CA_CERTS": "/absolute/path/to/copied-runtime-cert.pem"
}
}
}
}
mcp-remote@0.1.38 expands ${MOVEIT_MCP_AUTHORIZATION} in the header
argument from the bridge environment. Set it to Bearer followed by the
MoveIt Pro MCP HTTP key, as shown above; Claude Desktop only supplies the
environment to the local bridge.
This bridge runs on the client computer, so it can use the client's private
network and tunnel. Start the SSH tunnel before Claude Desktop. If the tunnel
closes, restore it and restart Claude Desktop so the app respawns the bridge. A
Server disconnected error with ECONNREFUSED on the forwarded local port
means that no tunnel is listening there. Protect the configuration as an
owner-only file and restart Claude Desktop after editing it.
On macOS or WSL, Claude Code can import the same client-local bridge instead of duplicating it:
claude mcp add-from-claude-desktop --scope user
Claude Code applies its own permission policy after connecting. An interactive
session can approve the read tools when first used. A non-interactive session
running with --permission-mode dontAsk must explicitly allow the desired
mcp__moveit-pro__... tools with --allowedTools or in Claude Code settings;
otherwise Claude Code rejects the calls without asking even though the MCP
server marks them read-only.
Codex CLI and the Codex desktop app use the same local MCP configuration. Add
the client-local bridge to the owner-only ~/.codex/config.toml file so the key
is not placed in shell history or the codex mcp add process arguments:
[mcp_servers.moveit-pro]
command = "npx"
args = [
"-y",
"mcp-remote@0.1.38",
"https://localhost:13200/mcp",
"--transport",
"http-only",
"--header",
"Authorization:${MOVEIT_MCP_AUTHORIZATION}",
]
[mcp_servers.moveit-pro.env]
MOVEIT_MCP_AUTHORIZATION = "Bearer <MOVEIT_MCP_HTTP_KEY>"
NODE_EXTRA_CA_CERTS = "/absolute/path/to/copied-runtime-cert.pem"
Here too, mcp-remote expands the authorization value from its environment;
Codex does not need to interpolate the placeholder itself.
Codex also supports env_vars when a separately managed parent environment
provides these values. After changing either desktop client's configuration,
restart the app and open a new conversation so it discovers the new tool
catalog.
Pi connects to the HTTP endpoint directly, without a bridge. Register it so Pi reads the key from its environment instead of ~/.pi/agent/mcp.json:
pi mcp add moveit-pro --url https://localhost:13200/mcp --bearer-token-env-var MOVEIT_MCP_HTTP_KEY
Start Pi with MOVEIT_MCP_HTTP_KEY set to the MoveIt Pro MCP HTTP key and NODE_EXTRA_CA_CERTS set to the absolute path of the copied Runtime certificate, then run pi mcp list to confirm that moveit-pro is connected.
Remote host-user session over SSH
SSH runs moveit_pro mcp serve on a remote Runtime under the SSH account. The
AI client starts and stops it with the MCP connection; no persistent MCP
service is needed.
Within MCP, this identity gets every tool but orchestrate and save_objective by default: inspect, validate, create a configuration package, build, test, launch the Runtime, start an existing Objective, and cancel it. Add --orchestrate to the mcp install command to offer orchestrator programs; that session leaves out general authoring and host lifecycle tools, get_capabilities and run_objective. An optional policy narrows those scopes, can set a session expiry, and can list runtime:orchestrate in place of the flag. The SSH credential separately grants the account's normal shell access.
Prepare the SSH connection
Use a normal, non-root account on the remote Runtime host that owns the
workspace and can access Docker without sudo. Configure SSH to accept the
matching public key for runtime-user. Keep the private key on the AI client.
Verify the Runtime host's fingerprint through a separate trusted channel. Do
not disable host-key verification or store a password in the MCP configuration.
Before adding the strict MCP alias below, enroll the host key interactively:
ssh \
-o StrictHostKeyChecking=ask \
-o PreferredAuthentications=publickey \
-o PasswordAuthentication=no \
-o KbdInteractiveAuthentication=no \
-o IdentitiesOnly=yes \
-i /absolute/path/to/id_ed25519 \
runtime-user@robot.example.com true
Compare the displayed fingerprint with the independently obtained fingerprint.
Only accept an exact match. OpenSSH then records the key in ~/.ssh/known_hosts.
An SSH alias keeps client configurations readable. For example, add this to
the AI client's owner-only ~/.ssh/config:
Host moveit-pro-runtime
HostName robot.example.com
User runtime-user
IdentityFile /absolute/path/to/id_ed25519
IdentitiesOnly yes
StrictHostKeyChecking yes
BatchMode yes
RequestTTY no
ClearAllForwardings yes
ForwardAgent no
ForwardX11 no
After enrollment, verify that the strict, non-interactive alias reaches the expected account:
ssh moveit-pro-runtime /usr/bin/moveit_pro mcp serve --help
Replace /usr/bin/moveit_pro below if command -v moveit_pro on the remote
Runtime host reports a different absolute path.
A private overlay network such as Tailscale needs no other change: point
HostName at the tailnet address. If the Runtime host uses Tailscale SSH, its
ACL rule for this account must not set a check period, because the AI client
starts the command non-interactively.
Configure the MCP client
Save the workspace, robot configuration package, instance, and optional policy as an owner-controlled target profile on the remote Runtime:
moveit_pro mcp target add lab-robot \
--workspace /home/runtime-user/moveit_pro_example_ws \
--config-package my_robot_config
Add --instance <name> for a named deployment. To narrow what the session may
do, create a policy and pass
--policy /home/runtime-user/mcp-policy.json. Reusing a target name replaces
its definition; reconnect to use the replacement.
Generate the client entry without putting target paths in the SSH command:
ssh moveit-pro-runtime /usr/bin/moveit_pro mcp install \
--client claude-desktop \
--target lab-robot \
--ssh-host moveit-pro-runtime \
--print
Use --client-id <name> to label the connecting computer in the audit log; the
default is the client name. Copy the generated moveit-pro entry into the AI
client's owner-only MCP configuration. --ssh-host requires --print because
the remote Runtime cannot edit that configuration.
The generated entry is compact and contains no workspace or policy paths. The
command is the client computer's ssh; the last argument runs through the
remote account's shell, so it is one quoted string:
{
"mcpServers": {
"moveit-pro": {
"command": "/usr/bin/ssh",
"args": [
"-o",
"BatchMode=yes",
"-T",
"moveit-pro-runtime",
"/usr/bin/moveit_pro mcp serve --target lab-robot --client-id claude-desktop"
]
}
}
}
Codex CLI and the Codex desktop app can use the same command in the AI client's
owner-only ~/.codex/config.toml:
[mcp_servers.moveit-pro]
command = "/usr/bin/ssh"
args = [
"-o",
"BatchMode=yes",
"-T",
"moveit-pro-runtime",
"/usr/bin/moveit_pro mcp serve --target lab-robot --client-id codex-over-ssh",
]
Pi registers the same command with pi mcp add, which writes the user's ~/.pi/agent/mcp.json:
pi mcp add moveit-pro -- /usr/bin/ssh -o BatchMode=yes -T moveit-pro-runtime \
"/usr/bin/moveit_pro mcp serve --target lab-robot --client-id pi-over-ssh"
The target profile and its paths live on the remote Runtime, in the
owner-only registry ~/.local/state/moveit_pro/mcp/targets.json. MoveIt Pro
validates the policy at session startup and rejects mixing a saved target with
direct target options. Keep the
SSH alias and remote command owner-controlled.
Restart the MCP client after changing its configuration. The client starts a new SSH process and MCP container for each connection. Runtime reads become available when the matching Runtime is running, but workspace reads and host lifecycle tools do not require the REST backend to be running.
Keep the private key on the client computer; agent forwarding is unnecessary. Administrators may further restrict a dedicated SSH key with a forced wrapper and disabled forwarding, but that wrapper must preserve MCP stdout and forward diagnostics on stderr. Non-interactive remote shell startup files must not print banners or other text to stdout, because stdout is reserved for MCP JSON-RPC.
Restrict a host-user session
A host-user session holds every tool but orchestrate and save_objective by default, whether the client starts the
command directly or over SSH. To narrow what one client may do, or to make its
authority expire, add an owner-only, target-bound policy. The session then
keeps the read tools plus only the scopes the policy lists. For example, a
policy that allows building and testing but not launching or starting
Objectives:
{
"version": 1,
"policy_id": "local-authoring",
"resource_audience": "moveit-pro-mcp",
"workspace": "/absolute/path/to/user_ws",
"config_package": "my_robot_config",
"instance": null,
"granted_scopes": [
"workspace:write",
"validation:read",
"runtime:config",
"runtime:build",
"runtime:test"
],
"expires_at": "REPLACE_WITH_TIMEZONE_QUALIFIED_ISO_8601"
}
expires_at is optional: set it to a timezone-qualified future timestamp, the latest time a session under the policy may run, or leave it out. MoveIt Pro 10.2 requires it. List only the scopes the client should keep (runtime:launch, runtime:execute, and runtime:orchestrate are the others). --orchestrate cannot be combined with a policy: to offer orchestrate under one, list runtime:orchestrate with runtime:execute. A session that holds runtime:orchestrate leaves out general authoring and host lifecycle tools, get_capabilities and run_objective, whatever else the policy lists. Use "granted_scopes": [] to retain only read tools. Without a policy expiry, a session keeps its authority until its client disconnects. After expiry, reconnect before making tool calls; renew an expired policy first. Expiry does not stop an Objective already running. Use a new authorized session to cancel it, or use the Runtime's own stop controls. Create the file under the workspace owner's account with an owner-only umask before editing it:
umask 077
touch /absolute/path/to/mcp-policy.json
chmod 600 /absolute/path/to/mcp-policy.json
$EDITOR /absolute/path/to/mcp-policy.json
Then include it when installing the desktop entry:
moveit_pro mcp install \
--client claude-desktop \
--workspace /absolute/path/to/user_ws \
--config-package my_robot_config \
--policy /absolute/path/to/mcp-policy.json
For Claude Code, Codex, or Pi, append --policy to a moveit_pro mcp serve command whose target matches the policy's: pass --workspace and --config-package, plus --instance for a named deployment. A policy is bound to one target. For SSH, save --policy with moveit_pro mcp target add and reconnect after replacing the profile. Rerun mcp install after changing an installed same-host entry. A confirmation key in older policy files is accepted and ignored.
create_config, build_workspace, and test_workspace run the workspace's package generators, build files, and tests as the local OS user, and run_objective and orchestrate can move the robot. Configure your MCP client's tool approvals and keep the operator present or use a simulation target when an agent starts Objectives. cancel_objective is best effort, not an emergency stop. Neither MCP cancellation nor the Runtime's software stop controls replace physical emergency-stop controls.
By default, Pi does not ask for approval before calling a tool. A policy is the limit MoveIt Pro enforces on a session. Pi's --tools option additionally limits which tools Pi offers the model, including from codemode scripts, for example pi --tools 'read,codemode,mcp__moveit_pro__get_*,mcp__moveit_pro__list_*'. Pi names each tool mcp__<server>__<tool> with - replaced by _, so use your registered server name, such as mcp__moveit_pro_lab_test__get_* for moveit-pro-lab-test.
To print the configuration instead of writing it:
moveit_pro mcp install \
--client claude-desktop \
--workspace /absolute/path/to/user_ws \
--config-package my_robot_config \
--print
If the workspace contains exactly one discoverable robot configuration package,
--config-package may be omitted. MoveIt Pro refuses to guess when there are
zero or multiple candidates. Add --instance <name> to bind a named
deployment.
For a named instance, stdio Runtime reads never bind to the host DDS graph.
They are proxied over the session's authenticated host channel to the exact
labeled Agent container of that instance, so get_execution_status
and the URDF resource work over stdio just as they do over
the backend's authenticated HTTP endpoint. RUNTIME remains unavailable until
that instance's containers are running.
The generated command runs moveit_pro mcp serve. Each client connection:
- creates an immutable OS-user principal and target binding;
- creates isolated config and private session state;
- starts one
docker compose run --rm --no-deps -Tcontainer; - reserves stdout for MCP JSON-RPC and sends setup diagnostics to stderr;
- verifies the invoking account is non-root, maps that account into the container, and drops entrypoint privileges before the MCP server starts; and
- removes its container, credential, and socket on disconnect.
Containers and runtime directories carry a session ID. The next serve
removes orphaned sessions left by abnormal termination, and a policy that sets
expires_at also gives session credentials an absolute expiry. Security audit logs are retained separately from
the ephemeral credential directory. Authorized host-control intent is recorded
before any effect and followed by its outcome. Read-only polling stays quiet;
repeated read-side target failures are retained once per session, provider, and
reason instead of growing durable storage with request volume.
List MCP Sessions
Run this on the computer that runs moveit_pro mcp serve, as the account that owns the sessions. For an SSH connection, run it on the SSH host.
moveit_pro mcp session list
moveit_pro mcp session list --json
Each session shows its UUID, state, client, instance, and target; --json prints records with session_id, state, pid, client_id, instance, and target_id, with null for a value the session did not record. An active session's launcher is still running. An orphaned session's launcher stopped without cleaning up, for example after SIGKILL, and the next moveit_pro mcp serve on that host removes it. To end a session, disable the server in your AI client: Claude Code, for example, starts a new session on its next tool call after one ends.
What the assistant can see
Tools
| Tool | Returns |
|---|---|
get_capabilities | Which capabilities the session's tools can use right now, plus its principal_type, granted_scopes, and upgrade_paths: the operator steps that unlock more tools. |
list_objectives | Every Objective in the selected robot configuration, optionally filtered by words in its ID, description, or category. |
get_objective | One current Objective's normalized XML and metadata, by ID. With source: true, returns the selected package's original source XML and edit revision instead. |
get_objective_authoring_guide | Customer-shipped Markdown guidance, matched to the installed image. Defaults to path: "create-objective/SKILL.md"; linked guides are available by path. |
get_behavior_authoring_guide | Customer-shipped Behavior guidance, matched to the installed image. Defaults to path: "create-behavior/SKILL.md". |
list_behaviors | Compact search over the matching Runtime's loaded Behaviors and current source Subtrees; accepts query, offset, and limit. Requires the Runtime. |
get_behavior_details | One Behavior's metadata and loaded ports, by ID. Requires the matching Runtime. |
list_waypoints | Every saved Waypoint's name and description. |
get_recent_logs | A bounded tail of ROS node logs or launch stdout, ordered by last write. Optional query filters lines; modified_at reports file freshness and output_truncated reports omitted older output. |
get_execution_status | The matching Objective Server's live state, running Objective, current_goal_id, and UUID-tagged recent_goals, including terminal statuses. |
get_robot_profile | Where each tool and any TF frame you name is now, each joint's position, and what an orchestrator program can do on this robot: each planning group's tool, trajectory controller, end-effector Objectives and force sensing, the default group, and in an --orchestrate session how to write each node this robot can use. See Run Agentic Programs. |
capture_observation | Camera image, acquisition timestamp, camera pose, and available frame context. |
locate_points | Measured 3D surface positions for selected image points in a captured frame. |
These are available to every authorized client, including over authenticated HTTP. find_objects joins them, returning optional Gemini labels, image points and an annotated image, on a server configured with MOVEIT_MCP_GEMINI_ENABLED=1 and GOOGLE_GEMINI_API_KEY. A host-user stdio session, started locally or over SSH, additionally exposes:
| Tool | Returns | Requires |
|---|---|---|
validate_objective | Validation of Objective XML by the same validator the Objective Server uses. level is full or structural_lint. | Local session with the canonical validator installed. full also needs a live matching Runtime manifest. |
create_objective | Validates and creates an Objective source file without overwriting an existing filename or Objective ID. Returns its revision. | Host-user session with workspace:write |
update_objective | Validates and replaces an Objective source file only when expected_revision matches. Returns its new revision. | Host-user session with workspace:write |
create_behavior | Creates one complete custom Behavior package from the standard template and supplied header, implementation body, and test source. Returns the registered ID, created files, and build/test handoffs. | Host-user session with workspace:write |
create_config | A newly generated package using the session's bound configuration name. | Host-user session |
build_workspace | A job handle for a build. Pass package_name to build one package and the workspace packages it depends on, or omit it to build the whole user workspace. | Host-user session |
test_workspace | A job handle for tests. Pass package_name to test one package, or omit it to test the whole workspace. Packages run sequentially, and test failures fail the job. | Host-user session |
get_job_status | State and new output for a build or test job. | Host-user session |
launch_runtime | Returns already_running, pid, and target_id. A new launch starts the Runtime supervisor; it does not guarantee that the Runtime is ready. Poll get_capabilities until RUNTIME is available, then call the relevant Runtime-backed tool, which rechecks availability. If startup fails, use get_recent_logs to inspect the retained launch output. | Host-user session |
run_objective | Runs an existing Objective on the bound Runtime and returns how it ended, with its goal id. A run longer than 50 seconds returns STILL_RUNNING, and get_execution_status reports it from there. It refuses when another Objective is reported as executing; that check is best effort, since a start from another client in the same instant still preempts. The robot may move, and keeps moving if the session ends. | Host-user session; approvals follow your client's configured permissions |
orchestrate | In beta. Runs the next step of a task as one program, without the model between its motions: planned moves, straight tool moves, picks and places, with retries and alternatives. See Run Agentic Programs. | Host-user session started with --orchestrate; approvals follow your client's configured permissions |
cancel_objective | Cancels exactly the supplied goal_id; a stale ID never cancels a replacement Objective. | Host-user session |
Pass the goal_id that run_objective or orchestrate returned with STILL_RUNNING or still_running to cancel_objective(goal_id). While either call is still waiting, the same ID is get_execution_status.current_goal_id. Compare it with get_execution_status.current_goal_id to identify the observed execution, including when two clients start the same Objective. Status is a snapshot, not a reservation: another client can change it immediately. Cancellation applies the UUID selector at the action server and echoes both the requested goal_id and the observed current_goal_id. Unknown or finished goals return cancelled: false; accepted cancellation returns cancelled: true with the Runtime state observed after a bounded wait. That state may describe a different goal. During a transition, status can ask you to retry instead of returning an ambiguous active ID.
Stop the Runtime through MoveIt Pro's terminal or application controls. MCP does not stop it because container shutdown cannot atomically prevent a new Objective from starting; exposing that operation would risk removing robot controllers during execution.
run_objective reports how the run ended in status and message. After STILL_RUNNING, match its goal_id in get_execution_status.recent_goals to observe SUCCEEDED, ABORTED, or CANCELED. The list contains at most 20 goals from the Runtime's retained action-status snapshot, not durable history. An absent goal has unknown outcome; IDLE alone never means success. For diagnostics, get_recent_logs(query="<Objective name>") searches the bounded tails of up to 20 recently modified node and launch logs. A successful Objective does not prove physical placement unless its tree includes that verification.
A full validation reports manifest_source: runtime because its plugin
manifest is read from the live, target-authenticated Runtime. It does not trust
a durable manifest cache: later configuration edits, product plugin upgrades,
or workspace builds could make one stale. A full check never silently falls
back to structural_lint; start the matching Runtime and retry when its
manifest is unavailable.
build_workspace and test_workspace return immediately with a job_id
rather than holding the request open for the length of a build. Poll
get_job_status(job_id, output_offset) and pass back the output_offset it
returns to read only what is new, so compiler errors are readable while the
build is still running.
When an assistant creates or changes one package, it can pass that package's
name to either tool. The job fails if colcon cannot find that package in the
bound workspace. Omitting package_name preserves the whole-workspace build and
test operation.
get_capabilities is advisory. Each tool rechecks its required provider and
capabilities immediately before execution. An authorized but unavailable tool
returns a structured result containing available: false, its missing
capabilities, a remedy, remedy_tool naming the authorized tool that resolves
it when one exists, and the immutable target_id.
The default host-network deployment uses a cooperative Agent identity handshake before any ROS-backed result is accepted. It detects a missing, stale, or wrong deployment, but plain DDS cannot prove the identity of a hostile local peer. Named instances use an isolated network namespace and managed DDS configuration. Closing the cooperative deployment's hostile-peer limit requires instance isolation, SROS2, or an authenticated remote Runtime provider.
Run Agentic Programs
MoveIt Pro Orchestrator lets an LLM sense, plan, act, and measure by compiling agentic programs into Objectives. See MoveIt Pro Orchestrator to enable this beta tool.
Follow the MCP Tutorial
Follow Tutorial 2 - Build with an AI Agent and MCP for short connection steps for Claude Code, Claude Desktop, Codex, and Pi, followed by prompts that prepare a Runtime and run an Objective with a simulated robot. The MCP server also works with physical robot configuration packages.
Observe the robot workspace
Camera tools require a running Runtime. They do not command motion. Camera capture and point measurements work without a Gemini key.
- Call
get_robot_profile()and select acamera_idfrom the cameras itssummarynames; it marks the ones that measure depth. - Call
capture_observation(camera_id, reference_frame)with a frame such asbase_linkorworld. A camera's namespace, such as/scene_camera, also works when it holds one color stream. The result contains an image the AI client can inspect, metadata, and anobservation_idfor follow-up queries. - Call
locate_points(observation_id, points, target_frame)with points such as[{"x": 0.5, "y": 0.5}]. Coordinates run from zero to one, with the origin at the image's top-left. Choosetarget_framefromavailable_target_frames.
Measurements use the captured depth and transforms even if the robot subsequently
moves. Each position is the visible surface at the selected pixel, in meters. Each point also estimates the object it lies on: width_m across the image row, a center in the middle of the object's top as depth shows it apart from the table or shelf it stands on, and the base where it stands, each null where the depth cannot show it. None of these is a grasp pose or a collision-checked motion goal.
Missing depth produces an error; an invalid depth pixel returns valid: false and
position: null.
Observations remain available for 120 seconds, with at most four captures retained per MCP server session. The oldest capture is evicted when the limit is reached. The HTTP endpoint shares its server session between clients authorized with the same deployment credential. Reconnects to stdio create a new observation store. Sensor data stays in server memory; camera tools do not save images to disk.
Supported streams use sensor_msgs/msg/Image: RGB/BGR, RGBA/BGRA, or 8-bit monochrome.
Metric lookup requires depth registered to the color image: matching optical frame,
image dimensions, and distortion-free pinhole CameraInfo with identity R and
matching K/P intrinsics. Depth must use 16UC1 millimeters or
32FC1 meters. RGB/depth timestamps must differ by at most 30 milliseconds.
Unregistered depth, nonzero distortion, ambiguous raw/rectified projection, binning,
and cropped calibration disable metric lookup. RGB capture remains available and
reports depth_unavailable_reason; it does not infer a transform or distance. Metadata distinguishes source_width/source_height from the returned JPEG’s
image_width/image_height; normalized coordinates apply to either size.
Source images are limited to 1920 × 1080 pixels;
previews are resized for the AI client while point coordinates remain normalized.
Enable optional Gemini inspection
For the best results, get a Gemini API key from Google AI Studio and enable inspection as below.
The connected vision-capable AI client can reason about capture images directly.
To additionally use find_objects(observation_id, query), set both
MOVEIT_MCP_GEMINI_ENABLED=1 and GOOGLE_GEMINI_API_KEY in the MCP server's
launch environment and restart that server. This sends the selected image, every other depth camera's current image
and the query to Google; an orchestrate pose that names its target in words sends each depth camera's image with those words. An existing Gemini key used by Behaviors does not enable MCP
inspection by itself. MOVEIT_MCP_GEMINI_MODEL optionally selects another Gemini Robotics-ER model;
the default is gemini-robotics-er-2-preview.
Inspection returns labeled normalized image points in the selected image as points and, where another camera's image shows the thing, those as other_views, each point with the observation_id of the image it lies in and its depth-backed measurement as measured. It also returns the selected image and each other image with a point, annotated with numbers matching points and then other_views. Labels and points are model
predictions. Pass selected points to locate_points for depth-backed measurements.
If Gemini is disabled, use the image from capture_observation directly.
Create and update Objectives
Ask the assistant to read get_objective_authoring_guide before editing. It serves the same skills bundle shipped to customers, starting with create-objective/SKILL.md. Follow its relative links by resolving them against the current document's path: ../debug-objective/SKILL.md from the overview becomes debug-objective/SKILL.md. This includes the SOP example and linked Behavior, debugging, CLI, and safety guidance. The reader cannot access arbitrary workspace files or execute skills. The assistant should inspect the installed Behavior catalog and existing Objectives, preserve unaffected XML and metadata, and verify Behavior IDs and ports.
create_objective(file_name, objective_xml) accepts a bare .xml filename and complete XML. It writes to the last Objective library owned by the session's selected robot configuration package, inside that package's workspace source directory. It does not write inherited packages or the installed snapshot. The package must declare an objective_library_paths entry and have been built.
For an edit, use the revision from your last write, or call get_objective(objective_id, source=true) for XML you did not just write. Pass its file_name and revision to update_objective(file_name, objective_xml, expected_revision). A stale revision leaves the file intact: read the current source and reapply the change. Updates keep the Objective ID and filename. MCP edits are serialized within that library; external editors do not participate in its advisory lock, so avoid simultaneous editing outside MCP.
To create a custom Behavior, describe what it should do, its inputs and outputs, and how it should be tested. The assistant follows guidance matched to your MoveIt Pro version and creates the package with its plugin registration and build and test wiring. It will not overwrite an existing package. Ask it to build and test the workspace, then launch or restart the Runtime before using the new Behavior.
Objective changes are fully validated against the running Runtime by default. Offline work can use structural validation, which does not verify installed plugins or motion safety. Saving an Objective does not build the workspace, launch the Runtime, or start execution.
The write result confirms the save. Both read modes discover current XML without a build or Runtime restart, and the open Desktop App receives file-change updates. source=true adds the original formatting and edit revision; the default returns normalized XML. Starting the Objective remains a separate tool call governed by your MCP client's approvals.
For node discovery, start with list_behaviors(query="sleep") or another short search. Results contain at most 20 summaries by default (up to 50 with limit), not every Behavior's ports. Use next_offset to continue a search and get_behavior_details(behavior_id) to inspect the few candidates you need. This keeps authoring inside MCP without loading the whole catalog into the conversation.
Resources
moveit-pro://catalog/behaviors— the Behavior catalog as JSON.moveit-pro://config/current— the selected robot configuration.moveit-pro://robot/urdf— the robot URDF; requires the matching Runtime.moveit-pro://objective/{objective_id}— one Objective's parsed JSON.
Workspace, capabilities, and host-user authority
The stdio session bind-mounts its canonical workspace read/write when its resolved scopes include workspace:write, otherwise read-only. It also mounts isolated materialized configuration, the target log directory, DDS inputs, and—when configured—the private typed provider files. It mounts no Docker socket and no devices. The
container has a reduced capability set only during setup, clears supplementary
groups, enables no-new-privileges, and runs the server as the invoking
host-mapped non-root user. Fixed Runtime probes also use that non-root UID/GID;
they never rely on the Runtime container's default root exec user.
Different workspaces, robot configuration packages, deployments, and instances
receive separate session state. Toolchain and lifecycle operations for the same
canonical workspace use a host advisory exclusion lock. A competing session
receives a structured TOOLCHAIN unavailable result naming the holder session
rather than running a concurrent build.
Host-shell access is already host-user authority; MCP does not turn an MCP HTTP key, ordinary network reachability, or socket reachability into that authority. An SSH stdio session has host-user authority because it authenticates as that host account, not because MCP elevates it. Confirmation of a consequential call is the MCP client's own tool approval; it is never a model-supplied tool argument.
Protocol version
The server speaks the sessionless MCP 2026-07-28 protocol on both transports.
Privacy considerations
Tool and resource results are sent to the connected AI client, which may be an
external service. Camera captures send robot workspace images to that client.
Enabling Gemini inspection additionally sends selected images and queries to Google.
In particular, get_recent_logs forwards log lines. Avoid
writing tokens, passwords, or private endpoint URLs to custom Behavior logs.
Troubleshooting
Stdio startup fails with Read-only file system
MoveIt Pro 10.2.0 can exit before tool discovery when ROS initialization tries to write to the MCP container's read-only Runtime log mount. The client then shows no tools, and moveit_pro mcp session list shows no active session after the process exits. Increasing the startup timeout does not resolve this error.
On the Runtime computer, edit the installed Compose file:
sudoedit /opt/moveit_pro/docker-compose.yaml
Add this entry to the existing services.mcp.environment list:
- ROS_LOG_DIR=/tmp/moveit-pro-mcp-ros-logs
Keep the existing Runtime log volume read-only. This setting gives the MCP process a separate writable ROS log directory inside its temporary container. It must be set in the container environment; adding it only to the AI client's host environment does not pass it through.
Restart the MCP client and check its tool list (/mcp in Codex). Tools such as get_capabilities and list_objectives should appear even before a Runtime is running. This workaround was verified against the 10.2.0 release image.
No stdio tools appear
Run the exact configured moveit_pro mcp serve or SSH command in a terminal and inspect stderr. A healthy stdio server waits for MCP requests; it does not print a tool list in the terminal. Check Docker access and the selected workspace and robot configuration package. Apply the workaround above if stderr ends with Read-only file system under .ros/log.
After changing client configuration, restart the client. Run moveit_pro mcp session list as the workspace owner on the Runtime computer while the client is connected. A stopped Runtime or an unbuilt robot configuration package should make individual capabilities unavailable, rather than hide all tools.
Other connection and capability errors
- HTTP returns
401 Unauthorized. Send the derived key frommoveit_pro mcp http-keyas a bearer token. - HTTP returns
421 Misdirected Request. Add the exact Host header, including its port, toMOVEIT_MCP_ALLOWED_HOSTS. - HTTP returns
503with "MCP endpoint unavailable". The MCP subsystem failed to start with the rest of the backend — the response body carries the reason (for example a misconfigured target identity, audit location, or transport allowlist). Fix the configuration and restart MoveIt Pro; this is not a transient outage. - HTTP returns
403 Invalid Origin header. Add the browser client's exact origin toMOVEIT_MCP_ALLOWED_ORIGINS; native clients should omit Origin. - HTTP returns
408 Request Timeout. The full request body must arrive within 30 seconds; check for a slow or unreliable network link between the client and the deployment. - HTTP returns
413 Request Entity Too Large. Keep a request at most 1 MiB. - HTTP returns
429 Too Many Requests. RespectRetry-After; one worker admits at most 16 concurrent MCP requests. - HTTP returns
504 Gateway Timeout. The server bounds how long a single MCP request may spend processing after upload. A healthy deployment answers well inside that bound; a 504 usually points at unresponsive storage backing the robot configuration or log directories. - TLS verification fails. Trust the backend certificate or its CA. Do not disable verification for a remote deployment.
- SSH authentication or host-key verification fails. Repeat host-key
enrollment, verify the fingerprint, and confirm public-key authentication
through the
moveit-pro-runtimealias. - SSH initialization disconnects. Run the configured SSH command in a
terminal and inspect stderr. Check the
moveit_pro, workspace, and policy paths; Docker access; and shell startup output. ACTIVE_CONFIGis unavailable. If the bound robot configuration package does not exist in the workspace, run thecreate_configtool to generate it, thenbuild_workspace. Otherwise, run thebuild_workspacetool to rebuild the bound robot configuration package. A successful job atomically refreshes the session's isolated robot configuration before the next read; a build run outside the session, such ascolcon buildin a terminal, does not. Changing the bound robot configuration package still requires a new session. If the refresh fails, run the generatedmoveit_pro mcp servecommand in a terminal and inspect stderr for the robot configuration package error.RUNTIMEis unavailable. Start the matching target. A Runtime answering with missing or different deployment identity is deliberately ignored. A session keeps its target until it ends, so after you start a different Runtime, reconnect the MCP server. In 10.2 and later, a server registered without--workspace,--config-package, or--targetthen binds the running Runtime, or the one--instancenames.- The URDF resource is unavailable. It requires the matching Runtime's
latched
/robot_description.