Run a pi0.5 Policy in the Runtime

Required Version

This feature will not be released until MoveIt Pro version 10.3.

MoveIt Pro can run a policy on an inference server or inside the Runtime. An inference server runs the model in a separate process. A ROS adapter forwards camera images and robot state to the server and returns its predicted actions to ExecutePolicy. Use this out-of-process route to try a new model or run inference on another computer. See Connect a VLA Policy.

For pi0.5, LoadPi05Policy loads a LibTorch bundle directly into the Runtime. ExecutePolicy then calls the model in the same process. This avoids the inference service's serialization and transport latency, and removes the separate server process to start and monitor. Camera topics, robot state, and controller commands still use ROS.

This guide runs a task-trained pi0.5 policy in the Runtime using the shipped Stack Cubes with the VLA Policy Objective in vla_sim.

Before You Begin

You need MoveIt Pro 10.3, a workspace containing vla_sim, and the amd64 NVIDIA Runtime image, which includes LibTorch and LoadPi05Policy.

Start with a verified LibTorch bundle. For training, follow Train a VLA Policy with Trainer or Train a VLA Policy. For export, follow Export a pi0.5 Model Bundle and the installed export-model skill. Its reference covers checkpoint preparation, export dependencies, target compatibility, and verification.

For this example, export PickNikRobotics/pi05_kinova_gen3_cube_stack_sim with dataset metadata from PickNikRobotics/kinova_gen3_cube_stack_sim. Select --runtimes libtorch --device cuda --dtype bfloat16 and name the output directory cube_stack. Preserve the checkpoint's saved processors and camera mapping. Continue only after bundle verification passes.

Export for the GPU architecture and PyTorch release used by the Runtime. Use only bundles you trust, because they contain compiled code. Check available GPU memory before loading; this checkpoint's weights alone are about 8.3 GB, and inference and the Simulator need additional memory. The Runtime also needs temporary disk space to extract the bundle.

Make the Bundle Available

Keep the complete cube_stack directory outside the workspace so its weights stay out of Git and the Docker build context. It must contain model.yaml and all the files produced by the exporter.

Set VLA_POLICY_BUNDLE_ROOT to the parent directory of cube_stack in the shell that will start the Runtime. For example, if you exported to ~/vla-artifacts/policies/cube_stack:

export VLA_POLICY_BUNDLE_ROOT="$HOME/vla-artifacts/policies"
nvidia-smi

In the execution workspace's docker-compose.yaml, add this mount to the existing runtime service. Preserve its other settings and volumes:

services:
  runtime:
    volumes:
      - ${VLA_POLICY_BUNDLE_ROOT:?Set VLA_POLICY_BUNDLE_ROOT}:/vla_policies:ro

The manifest will be available inside the Runtime at /vla_policies/cube_stack/model.yaml.

Start the Runtime and Run the Objective

Launch the Runtime and Desktop App

We assume you have already installed MoveIt Pro to the default install location. Start the MoveIt Pro Runtime using:

moveit_pro run -c vla_sim

Then launch or connect the separately distributed MoveIt Pro Desktop App. The Runtime does not serve a bundled user interface.

Start without the inference-server flags. If another Runtime is already running on the host, append --instance vla-pi05 to the launch command and connect the Desktop App to that instance's advertised endpoint. See Run Multiple Instances.

Open Stack Cubes with the VLA Policy in edit mode in the Desktop App. In the Objective's input ports, set the default values below, then save the Objective. Saving writes the new defaults to the Objective's XML in your workspace.

Input	Default value
use_in_process_policy	true
model_bundle_manifest	/vla_policies/cube_stack/model.yaml

Keep the default prompt stack the blue cube on the green cube, then run the Objective. Confirm that LoadPi05Policy reports the intended manifest, runtime=libtorch, and the expected serving device.

These inputs select LoadPi05Policy instead of the inference service. Keep the shipped camera, joint, gripper, controller, and safety settings.

Check the Result

Watch whether the blue cube rests on the green cube after the gripper releases. An Objective success status means the bounded motion finished; it does not establish stacking success. Record the observed stack separately from the Objective result. For repeated task scoring, see Evaluate a VLA Policy.

Stop through the Desktop App and wait for the controller to finish stopping before another motion. Run Reset MuJoCo Sim between attempts. See Load a Policy Once for policy reuse and cancellation.

Troubleshoot

Failure	Check
Bundle cannot load	Check the mount, complete bundle, matching LibTorch/GPU target, free GPU memory, and temporary disk space. Use the export skill's verification and compatibility checks.
Camera or state rejected	Compare the bundle's observation contract with the Objective's camera feeds, joint names, and gripper. See Connect a VLA Policy.
Inference times out or motion pauses	Capture call latency and available resources. See Sizing the Committed Window.
Motion rejected or stopped	Read the joint-limit, collision, controller, or force-torque diagnostic. Keep these checks enabled while correcting the cause.
Objective succeeds without a stack	Record a failed task attempt and check the scene, prompt, checkpoint, and observation mapping before repeating.