Skip to main content
Version: 11

Connect an AI Assistant with the MCP Server

The MoveIt Pro MCP server connects an AI assistant to a Runtime and its workspace. A local or SSH session can also create and validate Objectives, build and test robot configuration packages, launch the Runtime, and start or cancel existing Objectives.

Choose where the MCP client runs​

Where the AI client runsConnectionAccess
On the Runtime computerLocal stdioWorkspace, build, test, Runtime launch, and Objective execution.
On another computer, with SSH accessStdio over SSHThe same tools, running as the SSH account.
On another computer, with an MCP keyAuthenticated HTTPSInspection of Objectives, Behaviors, Waypoints, robot state, cameras, and logs.

MoveIt Pro 10.2.0: if stdio connects with no tools, check the startup workaround.

Local and SSH sessions have the workspace owner's authority. Configure your AI client's tool approvals before allowing Objective execution: clients may auto-approve calls, and the server adds no confirmation prompt. An optional policy limits a session's tools. The shared HTTPS credential grants inspection only.

orchestrate is opt-in, because it lets an agent compose motions and run any Behavior in the catalog, not only existing Objectives. Add --orchestrate to moveit_pro mcp serve or moveit_pro mcp install to offer it and save_objective for requested program saves. That session leaves out general authoring and host lifecycle tools, get_capabilities, and run_objective, whose work an objective step does. See MoveIt Pro Orchestrator.

Each session binds to one workspace, robot configuration package, and Runtime target. Change a fixed target with mcp install or by editing the client entry, then reconnect. Tools remain visible when a provider is unavailable; get_capabilities reports what is ready and how to enable it.

Same machine: install on the Runtime computer​

Run the following commands as the workspace-owning account on the computer with MoveIt Pro and Docker installed. The Runtime can be stopped when you connect. Run moveit_pro mcp serve without sudo; the launcher rejects root.

Install a client entry with an absolute workspace and an explicit robot configuration package:

moveit_pro mcp install \
--client claude-desktop \
--workspace /absolute/path/to/user_ws \
--config-package my_robot_config

Restart the AI client after registration. It starts moveit_pro mcp serve for each connection; a separate server process is unnecessary. Installation preserves unrelated client entries.

Claude Code, Codex, and Pi can register that same Runtime-local bootstrap directly.

Required Version
This feature requires MoveIt Pro version 10.2 or newer.

Register it without naming a target:

claude mcp add --scope user moveit-pro -- moveit_pro mcp serve
codex mcp add moveit-pro -- moveit_pro mcp serve
pi mcp add moveit-pro -- moveit_pro mcp serve

Each session binds its target when it starts. Without target options, moveit_pro mcp serve binds the Runtime your account is running. When several are running, it binds the one whose workspace contains the directory where you started the client; when that does not pick exactly one, it fails and lists them. Pass --instance <name>, or --instance default for the unnamed deployment, to choose one. When no Runtime is running, it binds the workspace and robot configuration package the MoveIt Pro CLI is configured for. Pass --workspace and --config-package to bind a fixed target instead. Before 10.2, moveit_pro mcp serve requires --workspace.

Your AI client controls approvals; see Claude Code permissions and Codex MCP configuration.

Approve tools automatically​

Required Version
This feature will not be released until MoveIt Pro version 10.3.

MoveIt Pro 10.3 adds these shortcuts for a registered server:

moveit_pro mcp auto-approve --client claude-code moveit-pro
moveit_pro mcp auto-approve --client codex moveit-pro

These commands configure automatic tool approval. run_objective can then move the robot without a prompt.

The Desktop App's Don't ask before MoveIt Pro tool calls option also replaces the same-name registration, including custom --policy options.

Remote client: connect from another computer​

The HTTP server runs with the REST backend on the Runtime computer. The AI client runs on the client computer and connects with:

  • URL: https://<runtime-host>:<rest-port>/mcp
  • Header: Authorization: Bearer <MOVEIT_MCP_HTTP_KEY>
  • TLS verification for the Runtime certificate or its issuing CA

MOVEIT_MCP_HTTP_KEY is the 64-character lowercase hexadecimal value printed by moveit_pro mcp http-key on the Runtime computer. It is derived from, but is not the same as, the frontend key. Do not give an AI client the frontend key: that credential also authenticates the broader REST backend.

Streamable HTTP is the preferred transport for a remote client. It uses the deployment's existing TLS certificate and requires Authorization: Bearer <MOVEIT_MCP_HTTP_KEY> on every MCP request; browser CORS preflight requests remain unauthenticated. Install or trust the MoveIt Pro Runtime certificate as described in Endpoint Security. HTTP sessions have the read-only deployment tools. They do not receive local workspace, build, or Runtime-lifecycle authority.

Start the server on the Runtime computer​

There is no separate HTTP MCP process to install or start. The /mcp endpoint starts automatically with the authenticated REST backend.

Start MoveIt Pro normally. The CLI creates and stores the deployment frontend key automatically. Print its domain-separated MCP credential on the Runtime computer when configuring an AI client:

moveit_pro mcp http-key

Rotating the frontend key also rotates this derived MCP key, so update clients after a rotation. For a direct remote connection, allow the exact Host header that the client will send before starting the Runtime:

export MOVEIT_MCP_ALLOWED_HOSTS="robot.example.com:3200"
moveit_pro run -c my_robot_config

The default REST port is 3200; named instances use their assigned REST port. The discovery service on port 3204 reports those ports. A native MCP client normally omits Origin. If a browser-based client sends one, also set MOVEIT_MCP_ALLOWED_ORIGINS to its exact origin before starting the Runtime.

Configure TLS and authentication on the client computer​

Verify TLS before allowing the client to send the MCP HTTP key:

  • For a direct hostname or IP address, use a certificate signed by a CA the client trusts and whose subject alternative names cover the MCP URL.

  • The generated Runtime certificate covers loopback names. For a private or tunneled development connection, copy cert.pem to the client through an authenticated channel such as SSH, trust it only for this connection, and connect through loopback. The generated certificate is at ~/.local/share/moveit_pro/tls/cert.pem by default, or under MOVEIT_TLS_CERT_DIR when that variable is set. For example:

    scp <runtime-user>@<runtime-host>:~/.local/share/moveit_pro/tls/cert.pem \
    /owner-only/path/runtime-cert.pem
    ssh -N -L 13200:127.0.0.1:3200 <runtime-user>@<runtime-host>

    Keep the tunnel open and use https://localhost:13200/mcp. Loopback Host values are allowed without MOVEIT_MCP_ALLOWED_HOSTS.

The MoveIt Pro Desktop App performs frontend-key-authenticated certificate pairing automatically for its own Electron gateway. That paired certificate, stored key, and local gateway are deliberately not exported to other processes. An MCP client on the same computer is configured separately with its own TLS trust and MCP-only bearer key. The desktop profile's Alternate hostname or IP address also applies only to Electron; set the MCP URL or SSH tunnel to the route that the MCP client should use.

Native HTTP clients must support MoveIt Pro's sessionless MCP 2026-07-28 protocol. The mcp-remote bridge shown below adapts stdio clients to that HTTP protocol.

Generate Claude Desktop's remote entry instead of writing it by hand:

moveit_pro mcp install --client claude-desktop --url https://localhost:13200/mcp

The command prints the JSON with a <MOVEIT_MCP_HTTP_KEY> placeholder plus the remaining setup steps, including what a read-only remote session can and cannot do. It never writes a client file or embeds the key. Paste the output into the client configuration on the client computer.

Use the client's credential or environment-variable support instead of storing the key in plaintext when available. If the client config contains the key, make the file owner-only.

Claude Desktop's remote connector flow is for internet-reachable servers using authless or OAuth authentication. It does not use the client computer's LAN, VPN, Tailscale route, or SSH tunnel, and Claude Desktop does not load a remote HTTP server directly from claude_desktop_config.json. For a private Runtime protected by a MoveIt Pro MCP bearer key, configure a local stdio-to-HTTP bridge in that file. This example uses mcp-remote; pin and review the bridge version your organization approves:

{
"mcpServers": {
"moveit-pro": {
"command": "npx",
"args": [
"-y",
"mcp-remote@0.1.38",
"https://localhost:13200/mcp",
"--transport",
"http-only",
"--header",
"Authorization:${MOVEIT_MCP_AUTHORIZATION}"
],
"env": {
"MOVEIT_MCP_AUTHORIZATION": "Bearer <MOVEIT_MCP_HTTP_KEY>",
"NODE_EXTRA_CA_CERTS": "/absolute/path/to/copied-runtime-cert.pem"
}
}
}
}

mcp-remote@0.1.38 expands ${MOVEIT_MCP_AUTHORIZATION} in the header argument from the bridge environment. Set it to Bearer followed by the MoveIt Pro MCP HTTP key, as shown above; Claude Desktop only supplies the environment to the local bridge.

This bridge runs on the client computer, so it can use the client's private network and tunnel. Start the SSH tunnel before Claude Desktop. If the tunnel closes, restore it and restart Claude Desktop so the app respawns the bridge. A Server disconnected error with ECONNREFUSED on the forwarded local port means that no tunnel is listening there. Protect the configuration as an owner-only file and restart Claude Desktop after editing it.

On macOS or WSL, Claude Code can import the same client-local bridge instead of duplicating it:

claude mcp add-from-claude-desktop --scope user

Claude Code applies its own permission policy after connecting. An interactive session can approve the read tools when first used. A non-interactive session running with --permission-mode dontAsk must explicitly allow the desired mcp__moveit-pro__... tools with --allowedTools or in Claude Code settings; otherwise Claude Code rejects the calls without asking even though the MCP server marks them read-only.

Codex CLI and the Codex desktop app use the same local MCP configuration. Add the client-local bridge to the owner-only ~/.codex/config.toml file so the key is not placed in shell history or the codex mcp add process arguments:

[mcp_servers.moveit-pro]
command = "npx"
args = [
"-y",
"mcp-remote@0.1.38",
"https://localhost:13200/mcp",
"--transport",
"http-only",
"--header",
"Authorization:${MOVEIT_MCP_AUTHORIZATION}",
]

[mcp_servers.moveit-pro.env]
MOVEIT_MCP_AUTHORIZATION = "Bearer <MOVEIT_MCP_HTTP_KEY>"
NODE_EXTRA_CA_CERTS = "/absolute/path/to/copied-runtime-cert.pem"

Here too, mcp-remote expands the authorization value from its environment; Codex does not need to interpolate the placeholder itself.

Codex also supports env_vars when a separately managed parent environment provides these values. After changing either desktop client's configuration, restart the app and open a new conversation so it discovers the new tool catalog.

Pi connects to the HTTP endpoint directly, without a bridge. Register it so Pi reads the key from its environment instead of ~/.pi/agent/mcp.json:

pi mcp add moveit-pro --url https://localhost:13200/mcp --bearer-token-env-var MOVEIT_MCP_HTTP_KEY

Start Pi with MOVEIT_MCP_HTTP_KEY set to the MoveIt Pro MCP HTTP key and NODE_EXTRA_CA_CERTS set to the absolute path of the copied Runtime certificate, then run pi mcp list to confirm that moveit-pro is connected.

Remote host-user session over SSH​

SSH runs moveit_pro mcp serve on a remote Runtime under the SSH account. The AI client starts and stops it with the MCP connection; no persistent MCP service is needed.

Within MCP, this identity gets every tool but orchestrate and save_objective by default: inspect, validate, create a configuration package, build, test, launch the Runtime, start an existing Objective, and cancel it. Add --orchestrate to the mcp install command to offer orchestrator programs; that session leaves out general authoring and host lifecycle tools, get_capabilities and run_objective. An optional policy narrows those scopes, can set a session expiry, and can list runtime:orchestrate in place of the flag. The SSH credential separately grants the account's normal shell access.

Prepare the SSH connection​

Use a normal, non-root account on the remote Runtime host that owns the workspace and can access Docker without sudo. Configure SSH to accept the matching public key for runtime-user. Keep the private key on the AI client. Verify the Runtime host's fingerprint through a separate trusted channel. Do not disable host-key verification or store a password in the MCP configuration.

Before adding the strict MCP alias below, enroll the host key interactively:

ssh \
-o StrictHostKeyChecking=ask \
-o PreferredAuthentications=publickey \
-o PasswordAuthentication=no \
-o KbdInteractiveAuthentication=no \
-o IdentitiesOnly=yes \
-i /absolute/path/to/id_ed25519 \
runtime-user@robot.example.com true

Compare the displayed fingerprint with the independently obtained fingerprint. Only accept an exact match. OpenSSH then records the key in ~/.ssh/known_hosts.

An SSH alias keeps client configurations readable. For example, add this to the AI client's owner-only ~/.ssh/config:

Host moveit-pro-runtime
HostName robot.example.com
User runtime-user
IdentityFile /absolute/path/to/id_ed25519
IdentitiesOnly yes
StrictHostKeyChecking yes
BatchMode yes
RequestTTY no
ClearAllForwardings yes
ForwardAgent no
ForwardX11 no

After enrollment, verify that the strict, non-interactive alias reaches the expected account:

ssh moveit-pro-runtime /usr/bin/moveit_pro mcp serve --help

Replace /usr/bin/moveit_pro below if command -v moveit_pro on the remote Runtime host reports a different absolute path.

A private overlay network such as Tailscale needs no other change: point HostName at the tailnet address. If the Runtime host uses Tailscale SSH, its ACL rule for this account must not set a check period, because the AI client starts the command non-interactively.

Configure the MCP client​

Save the workspace, robot configuration package, instance, and optional policy as an owner-controlled target profile on the remote Runtime:

moveit_pro mcp target add lab-robot \
--workspace /home/runtime-user/moveit_pro_example_ws \
--config-package my_robot_config

Add --instance <name> for a named deployment. To narrow what the session may do, create a policy and pass --policy /home/runtime-user/mcp-policy.json. Reusing a target name replaces its definition; reconnect to use the replacement.

Generate the client entry without putting target paths in the SSH command:

ssh moveit-pro-runtime /usr/bin/moveit_pro mcp install \
--client claude-desktop \
--target lab-robot \
--ssh-host moveit-pro-runtime \
--print

Use --client-id <name> to label the connecting computer in the audit log; the default is the client name. Copy the generated moveit-pro entry into the AI client's owner-only MCP configuration. --ssh-host requires --print because the remote Runtime cannot edit that configuration.

The generated entry is compact and contains no workspace or policy paths. The command is the client computer's ssh; the last argument runs through the remote account's shell, so it is one quoted string:

{
"mcpServers": {
"moveit-pro": {
"command": "/usr/bin/ssh",
"args": [
"-o",
"BatchMode=yes",
"-T",
"moveit-pro-runtime",
"/usr/bin/moveit_pro mcp serve --target lab-robot --client-id claude-desktop"
]
}
}
}

Codex CLI and the Codex desktop app can use the same command in the AI client's owner-only ~/.codex/config.toml:

[mcp_servers.moveit-pro]
command = "/usr/bin/ssh"
args = [
"-o",
"BatchMode=yes",
"-T",
"moveit-pro-runtime",
"/usr/bin/moveit_pro mcp serve --target lab-robot --client-id codex-over-ssh",
]

Pi registers the same command with pi mcp add, which writes the user's ~/.pi/agent/mcp.json:

pi mcp add moveit-pro -- /usr/bin/ssh -o BatchMode=yes -T moveit-pro-runtime \
"/usr/bin/moveit_pro mcp serve --target lab-robot --client-id pi-over-ssh"

The target profile and its paths live on the remote Runtime, in the owner-only registry ~/.local/state/moveit_pro/mcp/targets.json. MoveIt Pro validates the policy at session startup and rejects mixing a saved target with direct target options. Keep the SSH alias and remote command owner-controlled.

Restart the MCP client after changing its configuration. The client starts a new SSH process and MCP container for each connection. Runtime reads become available when the matching Runtime is running, but workspace reads and host lifecycle tools do not require the REST backend to be running.

Keep the private key on the client computer; agent forwarding is unnecessary. Administrators may further restrict a dedicated SSH key with a forced wrapper and disabled forwarding, but that wrapper must preserve MCP stdout and forward diagnostics on stderr. Non-interactive remote shell startup files must not print banners or other text to stdout, because stdout is reserved for MCP JSON-RPC.

Restrict a host-user session​

A host-user session holds every tool but orchestrate and save_objective by default, whether the client starts the command directly or over SSH. To narrow what one client may do, or to make its authority expire, add an owner-only, target-bound policy. The session then keeps the read tools plus only the scopes the policy lists. For example, a policy that allows building and testing but not launching or starting Objectives:

{
"version": 1,
"policy_id": "local-authoring",
"resource_audience": "moveit-pro-mcp",
"workspace": "/absolute/path/to/user_ws",
"config_package": "my_robot_config",
"instance": null,
"granted_scopes": [
"workspace:write",
"validation:read",
"runtime:config",
"runtime:build",
"runtime:test"
],
"expires_at": "REPLACE_WITH_TIMEZONE_QUALIFIED_ISO_8601"
}

expires_at is optional: set it to a timezone-qualified future timestamp, the latest time a session under the policy may run, or leave it out. MoveIt Pro 10.2 requires it. List only the scopes the client should keep (runtime:launch, runtime:execute, and runtime:orchestrate are the others). --orchestrate cannot be combined with a policy: to offer orchestrate under one, list runtime:orchestrate with runtime:execute. A session that holds runtime:orchestrate leaves out general authoring and host lifecycle tools, get_capabilities and run_objective, whatever else the policy lists. Use "granted_scopes": [] to retain only read tools. Without a policy expiry, a session keeps its authority until its client disconnects. After expiry, reconnect before making tool calls; renew an expired policy first. Expiry does not stop an Objective already running. Use a new authorized session to cancel it, or use the Runtime's own stop controls. Create the file under the workspace owner's account with an owner-only umask before editing it:

umask 077
touch /absolute/path/to/mcp-policy.json
chmod 600 /absolute/path/to/mcp-policy.json
$EDITOR /absolute/path/to/mcp-policy.json

Then include it when installing the desktop entry:

moveit_pro mcp install \
--client claude-desktop \
--workspace /absolute/path/to/user_ws \
--config-package my_robot_config \
--policy /absolute/path/to/mcp-policy.json

For Claude Code, Codex, or Pi, append --policy to a moveit_pro mcp serve command whose target matches the policy's: pass --workspace and --config-package, plus --instance for a named deployment. A policy is bound to one target. For SSH, save --policy with moveit_pro mcp target add and reconnect after replacing the profile. Rerun mcp install after changing an installed same-host entry. A confirmation key in older policy files is accepted and ignored.

create_config, build_workspace, and test_workspace run the workspace's package generators, build files, and tests as the local OS user, and run_objective and orchestrate can move the robot. Configure your MCP client's tool approvals and keep the operator present or use a simulation target when an agent starts Objectives. cancel_objective is best effort, not an emergency stop. Neither MCP cancellation nor the Runtime's software stop controls replace physical emergency-stop controls.

By default, Pi does not ask for approval before calling a tool. A policy is the limit MoveIt Pro enforces on a session. Pi's --tools option additionally limits which tools Pi offers the model, including from codemode scripts, for example pi --tools 'read,codemode,mcp__moveit_pro__get_*,mcp__moveit_pro__list_*'. Pi names each tool mcp__<server>__<tool> with - replaced by _, so use your registered server name, such as mcp__moveit_pro_lab_test__get_* for moveit-pro-lab-test.

To print the configuration instead of writing it:

moveit_pro mcp install \
--client claude-desktop \
--workspace /absolute/path/to/user_ws \
--config-package my_robot_config \
--print

If the workspace contains exactly one discoverable robot configuration package, --config-package may be omitted. MoveIt Pro refuses to guess when there are zero or multiple candidates. Add --instance <name> to bind a named deployment.

For a named instance, stdio Runtime reads never bind to the host DDS graph. They are proxied over the session's authenticated host channel to the exact labeled Agent container of that instance, so get_execution_status and the URDF resource work over stdio just as they do over the backend's authenticated HTTP endpoint. RUNTIME remains unavailable until that instance's containers are running.

The generated command runs moveit_pro mcp serve. Each client connection:

  1. creates an immutable OS-user principal and target binding;
  2. creates isolated config and private session state;
  3. starts one docker compose run --rm --no-deps -T container;
  4. reserves stdout for MCP JSON-RPC and sends setup diagnostics to stderr;
  5. verifies the invoking account is non-root, maps that account into the container, and drops entrypoint privileges before the MCP server starts; and
  6. removes its container, credential, and socket on disconnect.

Containers and runtime directories carry a session ID. The next serve removes orphaned sessions left by abnormal termination, and a policy that sets expires_at also gives session credentials an absolute expiry. Security audit logs are retained separately from the ephemeral credential directory. Authorized host-control intent is recorded before any effect and followed by its outcome. Read-only polling stays quiet; repeated read-side target failures are retained once per session, provider, and reason instead of growing durable storage with request volume.

List MCP Sessions​

Run this on the computer that runs moveit_pro mcp serve, as the account that owns the sessions. For an SSH connection, run it on the SSH host.

moveit_pro mcp session list
moveit_pro mcp session list --json

Each session shows its UUID, state, client, instance, and target; --json prints records with session_id, state, pid, client_id, instance, and target_id, with null for a value the session did not record. An active session's launcher is still running. An orphaned session's launcher stopped without cleaning up, for example after SIGKILL, and the next moveit_pro mcp serve on that host removes it. To end a session, disable the server in your AI client: Claude Code, for example, starts a new session on its next tool call after one ends.

What the assistant can see​

Tools​

ToolReturns
get_capabilitiesWhich capabilities the session's tools can use right now, plus its principal_type, granted_scopes, and upgrade_paths: the operator steps that unlock more tools.
list_objectivesEvery Objective in the selected robot configuration, optionally filtered by words in its ID, description, or category.
get_objectiveOne current Objective's normalized XML and metadata, by ID. With source: true, returns the selected package's original source XML and edit revision instead.
get_objective_authoring_guideCustomer-shipped Markdown guidance, matched to the installed image. Defaults to path: "create-objective/SKILL.md"; linked guides are available by path.
get_behavior_authoring_guideCustomer-shipped Behavior guidance, matched to the installed image. Defaults to path: "create-behavior/SKILL.md".
list_behaviorsCompact search over the matching Runtime's loaded Behaviors and current source Subtrees; accepts query, offset, and limit. Requires the Runtime.
get_behavior_detailsOne Behavior's metadata and loaded ports, by ID. Requires the matching Runtime.
list_waypointsEvery saved Waypoint's name and description.
get_recent_logsA bounded tail of ROS node logs or launch stdout, ordered by last write. Optional query filters lines; modified_at reports file freshness and output_truncated reports omitted older output.
get_execution_statusThe matching Objective Server's live state, running Objective, current_goal_id, and UUID-tagged recent_goals, including terminal statuses.
get_robot_profileWhere each tool and any TF frame you name is now, each joint's position, and what an orchestrator program can do on this robot: each planning group's tool, trajectory controller, end-effector Objectives and force sensing, the default group, and in an --orchestrate session how to write each node this robot can use. See Run Agentic Programs.
capture_observationCamera image, acquisition timestamp, camera pose, and available frame context.
locate_pointsMeasured 3D surface positions for selected image points in a captured frame.

These are available to every authorized client, including over authenticated HTTP. find_objects joins them, returning optional Gemini labels, image points and an annotated image, on a server configured with MOVEIT_MCP_GEMINI_ENABLED=1 and GOOGLE_GEMINI_API_KEY. A host-user stdio session, started locally or over SSH, additionally exposes:

ToolReturnsRequires
validate_objectiveValidation of Objective XML by the same validator the Objective Server uses. level is full or structural_lint.Local session with the canonical validator installed. full also needs a live matching Runtime manifest.
create_objectiveValidates and creates an Objective source file without overwriting an existing filename or Objective ID. Returns its revision.Host-user session with workspace:write
update_objectiveValidates and replaces an Objective source file only when expected_revision matches. Returns its new revision.Host-user session with workspace:write
create_behaviorCreates one complete custom Behavior package from the standard template and supplied header, implementation body, and test source. Returns the registered ID, created files, and build/test handoffs.Host-user session with workspace:write
create_configA newly generated package using the session's bound configuration name.Host-user session
build_workspaceA job handle for a build. Pass package_name to build one package and the workspace packages it depends on, or omit it to build the whole user workspace.Host-user session
test_workspaceA job handle for tests. Pass package_name to test one package, or omit it to test the whole workspace. Packages run sequentially, and test failures fail the job.Host-user session
get_job_statusState and new output for a build or test job.Host-user session
launch_runtimeReturns already_running, pid, and target_id. A new launch starts the Runtime supervisor; it does not guarantee that the Runtime is ready. Poll get_capabilities until RUNTIME is available, then call the relevant Runtime-backed tool, which rechecks availability. If startup fails, use get_recent_logs to inspect the retained launch output.Host-user session
run_objectiveRuns an existing Objective on the bound Runtime and returns how it ended, with its goal id. A run longer than 50 seconds returns STILL_RUNNING, and get_execution_status reports it from there. It refuses when another Objective is reported as executing; that check is best effort, since a start from another client in the same instant still preempts. The robot may move, and keeps moving if the session ends.Host-user session; approvals follow your client's configured permissions
orchestrateIn beta. Runs the next step of a task as one program, without the model between its motions: planned moves, straight tool moves, picks and places, with retries and alternatives. See Run Agentic Programs.Host-user session started with --orchestrate; approvals follow your client's configured permissions
cancel_objectiveCancels exactly the supplied goal_id; a stale ID never cancels a replacement Objective.Host-user session

Pass the goal_id that run_objective or orchestrate returned with STILL_RUNNING or still_running to cancel_objective(goal_id). While either call is still waiting, the same ID is get_execution_status.current_goal_id. Compare it with get_execution_status.current_goal_id to identify the observed execution, including when two clients start the same Objective. Status is a snapshot, not a reservation: another client can change it immediately. Cancellation applies the UUID selector at the action server and echoes both the requested goal_id and the observed current_goal_id. Unknown or finished goals return cancelled: false; accepted cancellation returns cancelled: true with the Runtime state observed after a bounded wait. That state may describe a different goal. During a transition, status can ask you to retry instead of returning an ambiguous active ID.

Stop the Runtime through MoveIt Pro's terminal or application controls. MCP does not stop it because container shutdown cannot atomically prevent a new Objective from starting; exposing that operation would risk removing robot controllers during execution.

run_objective reports how the run ended in status and message. After STILL_RUNNING, match its goal_id in get_execution_status.recent_goals to observe SUCCEEDED, ABORTED, or CANCELED. The list contains at most 20 goals from the Runtime's retained action-status snapshot, not durable history. An absent goal has unknown outcome; IDLE alone never means success. For diagnostics, get_recent_logs(query="<Objective name>") searches the bounded tails of up to 20 recently modified node and launch logs. A successful Objective does not prove physical placement unless its tree includes that verification.

A full validation reports manifest_source: runtime because its plugin manifest is read from the live, target-authenticated Runtime. It does not trust a durable manifest cache: later configuration edits, product plugin upgrades, or workspace builds could make one stale. A full check never silently falls back to structural_lint; start the matching Runtime and retry when its manifest is unavailable.

build_workspace and test_workspace return immediately with a job_id rather than holding the request open for the length of a build. Poll get_job_status(job_id, output_offset) and pass back the output_offset it returns to read only what is new, so compiler errors are readable while the build is still running.

When an assistant creates or changes one package, it can pass that package's name to either tool. The job fails if colcon cannot find that package in the bound workspace. Omitting package_name preserves the whole-workspace build and test operation.

get_capabilities is advisory. Each tool rechecks its required provider and capabilities immediately before execution. An authorized but unavailable tool returns a structured result containing available: false, its missing capabilities, a remedy, remedy_tool naming the authorized tool that resolves it when one exists, and the immutable target_id.

The default host-network deployment uses a cooperative Agent identity handshake before any ROS-backed result is accepted. It detects a missing, stale, or wrong deployment, but plain DDS cannot prove the identity of a hostile local peer. Named instances use an isolated network namespace and managed DDS configuration. Closing the cooperative deployment's hostile-peer limit requires instance isolation, SROS2, or an authenticated remote Runtime provider.

Run Agentic Programs​

Required Version
This feature will not be released until MoveIt Pro version 10.3.

MoveIt Pro Orchestrator lets an LLM sense, plan, act, and measure by compiling agentic programs into Objectives. See MoveIt Pro Orchestrator to enable this beta tool.

Follow the MCP Tutorial​

Required Version
This feature requires MoveIt Pro version 10.2 or newer.

Follow Tutorial 2 - Build with an AI Agent and MCP for short connection steps for Claude Code, Claude Desktop, Codex, and Pi, followed by prompts that prepare a Runtime and run an Objective with a simulated robot. The MCP server also works with physical robot configuration packages.

Observe the robot workspace​

Camera tools require a running Runtime. They do not command motion. Camera capture and point measurements work without a Gemini key.

  1. Call get_robot_profile() and select a camera_id from the cameras its summary names; it marks the ones that measure depth.
  2. Call capture_observation(camera_id, reference_frame) with a frame such as base_link or world. A camera's namespace, such as /scene_camera, also works when it holds one color stream. The result contains an image the AI client can inspect, metadata, and an observation_id for follow-up queries.
  3. Call locate_points(observation_id, points, target_frame) with points such as [{"x": 0.5, "y": 0.5}]. Coordinates run from zero to one, with the origin at the image's top-left. Choose target_frame from available_target_frames.

Measurements use the captured depth and transforms even if the robot subsequently moves. Each position is the visible surface at the selected pixel, in meters. Each point also estimates the object it lies on: width_m across the image row, a center in the middle of the object's top as depth shows it apart from the table or shelf it stands on, and the base where it stands, each null where the depth cannot show it. None of these is a grasp pose or a collision-checked motion goal. Missing depth produces an error; an invalid depth pixel returns valid: false and position: null.

Observations remain available for 120 seconds, with at most four captures retained per MCP server session. The oldest capture is evicted when the limit is reached. The HTTP endpoint shares its server session between clients authorized with the same deployment credential. Reconnects to stdio create a new observation store. Sensor data stays in server memory; camera tools do not save images to disk.

Supported streams use sensor_msgs/msg/Image: RGB/BGR, RGBA/BGRA, or 8-bit monochrome. Metric lookup requires depth registered to the color image: matching optical frame, image dimensions, and distortion-free pinhole CameraInfo with identity R and matching K/P intrinsics. Depth must use 16UC1 millimeters or 32FC1 meters. RGB/depth timestamps must differ by at most 30 milliseconds. Unregistered depth, nonzero distortion, ambiguous raw/rectified projection, binning, and cropped calibration disable metric lookup. RGB capture remains available and reports depth_unavailable_reason; it does not infer a transform or distance. Metadata distinguishes source_width/source_height from the returned JPEG’s image_width/image_height; normalized coordinates apply to either size. Source images are limited to 1920 × 1080 pixels; previews are resized for the AI client while point coordinates remain normalized.

Enable optional Gemini inspection​

For the best results, get a Gemini API key from Google AI Studio and enable inspection as below.

The connected vision-capable AI client can reason about capture images directly. To additionally use find_objects(observation_id, query), set both MOVEIT_MCP_GEMINI_ENABLED=1 and GOOGLE_GEMINI_API_KEY in the MCP server's launch environment and restart that server. This sends the selected image, every other depth camera's current image and the query to Google; an orchestrate pose that names its target in words sends each depth camera's image with those words. An existing Gemini key used by Behaviors does not enable MCP inspection by itself. MOVEIT_MCP_GEMINI_MODEL optionally selects another Gemini Robotics-ER model; the default is gemini-robotics-er-2-preview.

Inspection returns labeled normalized image points in the selected image as points and, where another camera's image shows the thing, those as other_views, each point with the observation_id of the image it lies in and its depth-backed measurement as measured. It also returns the selected image and each other image with a point, annotated with numbers matching points and then other_views. Labels and points are model predictions. Pass selected points to locate_points for depth-backed measurements. If Gemini is disabled, use the image from capture_observation directly.

Create and update Objectives​

Ask the assistant to read get_objective_authoring_guide before editing. It serves the same skills bundle shipped to customers, starting with create-objective/SKILL.md. Follow its relative links by resolving them against the current document's path: ../debug-objective/SKILL.md from the overview becomes debug-objective/SKILL.md. This includes the SOP example and linked Behavior, debugging, CLI, and safety guidance. The reader cannot access arbitrary workspace files or execute skills. The assistant should inspect the installed Behavior catalog and existing Objectives, preserve unaffected XML and metadata, and verify Behavior IDs and ports.

create_objective(file_name, objective_xml) accepts a bare .xml filename and complete XML. It writes to the last Objective library owned by the session's selected robot configuration package, inside that package's workspace source directory. It does not write inherited packages or the installed snapshot. The package must declare an objective_library_paths entry and have been built.

For an edit, use the revision from your last write, or call get_objective(objective_id, source=true) for XML you did not just write. Pass its file_name and revision to update_objective(file_name, objective_xml, expected_revision). A stale revision leaves the file intact: read the current source and reapply the change. Updates keep the Objective ID and filename. MCP edits are serialized within that library; external editors do not participate in its advisory lock, so avoid simultaneous editing outside MCP.

Required Version
This feature requires MoveIt Pro version 10.2 or newer.

To create a custom Behavior, describe what it should do, its inputs and outputs, and how it should be tested. The assistant follows guidance matched to your MoveIt Pro version and creates the package with its plugin registration and build and test wiring. It will not overwrite an existing package. Ask it to build and test the workspace, then launch or restart the Runtime before using the new Behavior.

Objective changes are fully validated against the running Runtime by default. Offline work can use structural validation, which does not verify installed plugins or motion safety. Saving an Objective does not build the workspace, launch the Runtime, or start execution.

The write result confirms the save. Both read modes discover current XML without a build or Runtime restart, and the open Desktop App receives file-change updates. source=true adds the original formatting and edit revision; the default returns normalized XML. Starting the Objective remains a separate tool call governed by your MCP client's approvals.

For node discovery, start with list_behaviors(query="sleep") or another short search. Results contain at most 20 summaries by default (up to 50 with limit), not every Behavior's ports. Use next_offset to continue a search and get_behavior_details(behavior_id) to inspect the few candidates you need. This keeps authoring inside MCP without loading the whole catalog into the conversation.

Resources​

  • moveit-pro://catalog/behaviors — the Behavior catalog as JSON.
  • moveit-pro://config/current — the selected robot configuration.
  • moveit-pro://robot/urdf — the robot URDF; requires the matching Runtime.
  • moveit-pro://objective/{objective_id} — one Objective's parsed JSON.

Workspace, capabilities, and host-user authority​

The stdio session bind-mounts its canonical workspace read/write when its resolved scopes include workspace:write, otherwise read-only. It also mounts isolated materialized configuration, the target log directory, DDS inputs, and—when configured—the private typed provider files. It mounts no Docker socket and no devices. The container has a reduced capability set only during setup, clears supplementary groups, enables no-new-privileges, and runs the server as the invoking host-mapped non-root user. Fixed Runtime probes also use that non-root UID/GID; they never rely on the Runtime container's default root exec user.

Different workspaces, robot configuration packages, deployments, and instances receive separate session state. Toolchain and lifecycle operations for the same canonical workspace use a host advisory exclusion lock. A competing session receives a structured TOOLCHAIN unavailable result naming the holder session rather than running a concurrent build.

Host-shell access is already host-user authority; MCP does not turn an MCP HTTP key, ordinary network reachability, or socket reachability into that authority. An SSH stdio session has host-user authority because it authenticates as that host account, not because MCP elevates it. Confirmation of a consequential call is the MCP client's own tool approval; it is never a model-supplied tool argument.

Protocol version​

The server speaks the sessionless MCP 2026-07-28 protocol on both transports.

Privacy considerations​

Tool and resource results are sent to the connected AI client, which may be an external service. Camera captures send robot workspace images to that client. Enabling Gemini inspection additionally sends selected images and queries to Google. In particular, get_recent_logs forwards log lines. Avoid writing tokens, passwords, or private endpoint URLs to custom Behavior logs.

Troubleshooting​

Stdio startup fails with Read-only file system​

MoveIt Pro 10.2.0 can exit before tool discovery when ROS initialization tries to write to the MCP container's read-only Runtime log mount. The client then shows no tools, and moveit_pro mcp session list shows no active session after the process exits. Increasing the startup timeout does not resolve this error.

On the Runtime computer, edit the installed Compose file:

sudoedit /opt/moveit_pro/docker-compose.yaml

Add this entry to the existing services.mcp.environment list:

- ROS_LOG_DIR=/tmp/moveit-pro-mcp-ros-logs

Keep the existing Runtime log volume read-only. This setting gives the MCP process a separate writable ROS log directory inside its temporary container. It must be set in the container environment; adding it only to the AI client's host environment does not pass it through.

Restart the MCP client and check its tool list (/mcp in Codex). Tools such as get_capabilities and list_objectives should appear even before a Runtime is running. This workaround was verified against the 10.2.0 release image.

No stdio tools appear​

Run the exact configured moveit_pro mcp serve or SSH command in a terminal and inspect stderr. A healthy stdio server waits for MCP requests; it does not print a tool list in the terminal. Check Docker access and the selected workspace and robot configuration package. Apply the workaround above if stderr ends with Read-only file system under .ros/log.

After changing client configuration, restart the client. Run moveit_pro mcp session list as the workspace owner on the Runtime computer while the client is connected. A stopped Runtime or an unbuilt robot configuration package should make individual capabilities unavailable, rather than hide all tools.

Other connection and capability errors​

  • HTTP returns 401 Unauthorized. Send the derived key from moveit_pro mcp http-key as a bearer token.
  • HTTP returns 421 Misdirected Request. Add the exact Host header, including its port, to MOVEIT_MCP_ALLOWED_HOSTS.
  • HTTP returns 503 with "MCP endpoint unavailable". The MCP subsystem failed to start with the rest of the backend — the response body carries the reason (for example a misconfigured target identity, audit location, or transport allowlist). Fix the configuration and restart MoveIt Pro; this is not a transient outage.
  • HTTP returns 403 Invalid Origin header. Add the browser client's exact origin to MOVEIT_MCP_ALLOWED_ORIGINS; native clients should omit Origin.
  • HTTP returns 408 Request Timeout. The full request body must arrive within 30 seconds; check for a slow or unreliable network link between the client and the deployment.
  • HTTP returns 413 Request Entity Too Large. Keep a request at most 1 MiB.
  • HTTP returns 429 Too Many Requests. Respect Retry-After; one worker admits at most 16 concurrent MCP requests.
  • HTTP returns 504 Gateway Timeout. The server bounds how long a single MCP request may spend processing after upload. A healthy deployment answers well inside that bound; a 504 usually points at unresponsive storage backing the robot configuration or log directories.
  • TLS verification fails. Trust the backend certificate or its CA. Do not disable verification for a remote deployment.
  • SSH authentication or host-key verification fails. Repeat host-key enrollment, verify the fingerprint, and confirm public-key authentication through the moveit-pro-runtime alias.
  • SSH initialization disconnects. Run the configured SSH command in a terminal and inspect stderr. Check the moveit_pro, workspace, and policy paths; Docker access; and shell startup output.
  • ACTIVE_CONFIG is unavailable. If the bound robot configuration package does not exist in the workspace, run the create_config tool to generate it, then build_workspace. Otherwise, run the build_workspace tool to rebuild the bound robot configuration package. A successful job atomically refreshes the session's isolated robot configuration before the next read; a build run outside the session, such as colcon build in a terminal, does not. Changing the bound robot configuration package still requires a new session. If the refresh fails, run the generated moveit_pro mcp serve command in a terminal and inspect stderr for the robot configuration package error.
  • RUNTIME is unavailable. Start the matching target. A Runtime answering with missing or different deployment identity is deliberately ignored. A session keeps its target until it ends, so after you start a different Runtime, reconnect the MCP server. In 10.2 and later, a server registered without --workspace, --config-package, or --target then binds the running Runtime, or the one --instance names.
  • The URDF resource is unavailable. It requires the matching Runtime's latched /robot_description.