Collect VLA Training Data with Trainer

MoveIt Pro Trainer turns robot demonstrations into a dataset you can use to train a vision-language-action (VLA) policy. The whole workflow stays inside MoveIt Pro, and the data stays on the Runtime computer unless you choose to move or upload it later.

This guide uses the vla_sim robot configuration package, the same block-stacking scene used in Run the VLA Cube-Stacking Example. You will record a Kinova Gen3 arm stacking a red cube on a blue cube, then convert the recording into a LeRobot dataset.

Stage	What it does
Training Config	Chooses the robot state, command, and camera topics to capture.
Dataset	Creates or selects the folder that will hold your demonstrations.
Record	Saves synchronized robot and camera data as episodes.
Playback	Lets you inspect an episode before using it.
Convert	Packages the selected episodes as a LeRobotDataset v3.0.

How Trainer is organized

Four pieces make up Trainer, each shown on its own tab. Here's how they relate:

• Training Config — the saved capture profile (cameras and topics). Trainer reads it at the start of every recording session, so before MoveIt Pro 10.0 whatever it says at that moment is what the new session captures. From 10.0 the session must also match the dataset's pinned options or it is refused before it starts — see the note in step 1. Its FPS is not a capture rate: recording always takes every topic at whatever rate it publishes at, and the FPS is applied later, at conversion.
• Recording session — one run of the recorder, started by Start recording and ended by Stop or by reaching the requested episode count. One recorder process backs the whole session, and it buffers everything in memory: nothing reaches disk until an episode is saved, either because you select Save or because a Fixed episode's timer expires and saves it for you.
• Episode — one saved demonstration. Each save flushes the buffered data and closes one file, and that file is the episode; conversion reads it as one episode by default. Selecting Stop instead of Save discards the buffer for that in-progress episode — nothing is written for it.
• Dataset — the folder that accumulates episodes. Every recording session you run into the same dataset name adds its saved episodes there; nothing is removed or reset between sessions.

Required Version

This feature requires MoveIt Pro version 10.0 or newer.

Conversion (step 5) reads from the dataset folder, not from whatever Training Config happens to be active at convert time — editing the config after recording does not change how already-recorded episodes convert.

Prerequisites

vla_sim ships in the example workspace, so check that workspace out and run moveit_pro build in it before launching.

Launch the Runtime and Desktop App

We assume you have already installed MoveIt Pro to the default install location. Start the MoveIt Pro Runtime using:

moveit_pro run -c vla_sim

Then launch or connect the separately distributed MoveIt Pro Desktop App. The Runtime does not serve a bundled user interface.

Confirm that the robot is publishing joint states and that each of the three cameras (overview, scene, wrist) is visible in a camera pane.

The cubes are camera-only in this scene

vla_sim renders the cubes for the camera topics but not in the 3D Visualizer, so the 3D pane always shows an empty table. This is expected: check a camera pane, not the 3D Visualizer, to see the cubes. Nothing is wrong with your simulation.

Open Trainer with the graduation-cap button in the upper-right corner. You can also press T.

1. Create a Training Config

Trainer opens on the Dataset tab. Select the gear icon next to Training to open Training Config.

Set:

• Robot type: a short name for the robot. This example uses gen3.
• Default FPS: the FPS shown when you start a recording, applied at conversion rather than at capture. This example uses 10, matching the Gen3 cube-stacking policy. A different Record-tab value is labeled as an override and is invalidated if this default changes.
• Dataset namespace: optional; leave it empty for a local, un-namespaced dataset.
• Camera topics: enable each view you want in the dataset and confirm its live ROS topic. This example enables all three vla_sim cameras: /overview_camera/color, /scene_camera/color, and /wrist_camera/color.

Open Advanced settings to check the joint-state and joint-command topics. Joint states become observation.state. Joint commands become the action labels. Conversion refuses an episode whose command topic carried no messages, or whose commands only mirror the recorded state, and names the reason instead of substituting labels you did not ask for; a robot with no command stream converts with Action source set to Next state.

Select Save Config.

A dataset locks in its recording options at creation time

A new dataset is pinned to the recording options in force when you create it: the enabled Camera topics and their order, the FPS, the joint-state and joint-command topics, and the robot type. Change any of them afterward and Trainer refuses further recordings into that dataset, naming the field that no longer matches; create a new dataset for the new settings. Episodes recorded under different camera sets or rates are not comparable training data, which is what the refusal protects.

This is also why a dataset can only be created once a Training Config is saved — the pin is resolved from it, so the create control stays unavailable until one is bound.

2. Create a Dataset

Open the Dataset tab. With no dataset yet, it opens with the Active dataset name field ready: enter a descriptive name, then select the checkmark to create it. Once a dataset exists, that field becomes a selector, and the + button beside it starts another one. Creating a dataset resets any Record-tab FPS override so its first recording matches the Training Config values pinned at creation.

This example uses vla_sim_block_stacking for episodes recorded while stacking the red cube on the blue cube.

Trainer makes the dataset active immediately.

Where Trainer data lives

Recordings and converted datasets are written under ~/.local/share/moveit_pro/trainer inside the Runtime container. That path is bind-mounted from the host directory MOVEIT_HOST_DATA_DIR (default ~/.local/share/moveit_pro). To put Trainer data elsewhere, set MOVEIT_HOST_DATA_DIR on the host before starting the Runtime; the Trainer subdirectory follows it. Container recreation never wipes this data.

Raw datasets are stored at:

~/.local/share/moveit_pro/trainer/recordings/<dataset-name>

You can return to this tab to switch datasets, rename one, record another episode, or delete an episode you do not want (read the caution in step 4 before deleting from a dataset you plan to convert).

3. Record Episodes

Open the Record tab and describe the task in plain language. This description is saved with the episode and later becomes the training task. For this example, use:

Stack the red cube on the blue cube

Choose how the episode ends:

• Until I Stop keeps recording until you decide the demonstration is complete.
• Fixed stops automatically after the number of seconds you enter.

Set the number of episodes, reset time between episodes, and FPS, then select Start recording. Trainer labels FPS when it overrides the Training Config default.

Wait until the status changes from Preparing to Recording before you move the robot. The preparing phase is not recorded, and Save and Stop stay unavailable until it ends. Once the status reads Recording, teleoperate the robot with any of MoveIt Pro's teleoperation modes to perform the task while Trainer captures joint states, joint commands, and every enabled camera stream.

The Record tab continues counting time while you jog the robot. Select Save when the useful part of the demonstration is complete; this keeps the episode. Select Stop instead to discard the in-progress episode without saving it.

The wrist camera confirms the cube is in view and being manipulated, even though the 3D Visualizer never shows it.

After saving, Trainer confirms the episode count and returns to the regular Record controls. Repeat the task with natural variation, such as different starting cube positions, to build a more useful training set.

4. Review and Play Back an Episode

Open the Dataset tab. Each episode row shows its duration, task, robot type, and two frame rates: cfg is the FPS the recording was configured for, and obs is the rate the enabled cameras actually delivered, measured from that episode's own recording. Compare them — an episode captured well below its configured rate produces training data labeled with timesteps it was never recorded at. obs is omitted when it could not be measured, and episodes recorded before MoveIt Pro measured it show only cfg.

Select the play button on an episode. Playback controls appear over the camera panes and 3D pane:

• Use the play/pause button to inspect a moment closely.
• Drag the timeline to scrub through the episode.
• Select Stop in the Playback mode card to return to the live robot view.

Playback is a review tool; it does not command the live robot. Deleting a poor episode is the only way to keep it out of the conversion — read the warning below first.

Deleting an episode can merge the rest of its recording session

When the recording session that saved an episode saved three or more, deleting one of them currently rewrites that session's survivors into a single file, and a later conversion reads that file as one training episode. The Dataset tab lists episodes without showing which session each came from, so the interface gives you no way to tell in advance whether a given delete lands in that case.

5. Convert to LeRobot

Open the Convert tab. Trainer shows the active MCAP dataset, the proposed output name, and the number of episodes it will convert.

The default settings work for most demonstrations. Open Advanced settings when you need to change:

• Output name and Repo ID for the local LeRobot metadata.
• Overwrite output when replacing a previous conversion.
• Trim idle frames to remove still frames before and after the task moves. Turn this off for a deliberately stationary test episode; otherwise every frame may be filtered out.
  • Idle scope chooses where trimming applies: Endpoints (the default) trims only the leading and trailing idle runs of each episode, leaving mid-episode pauses alone; All frames collapses every idle dwell anywhere in the episode, which can remove an intentional pause such as a grasp hold.
  • Idle eps is the motion threshold below which a frame counts as idle. Raise it to trim more aggressively; lower it if trimming cuts into slow, deliberate motion at the episode boundaries.
• Min frames and Drop first, both off at 0 and both counted in synced camera frames — one per message from the first enabled camera — not joint-state samples.
• Action source to choose command labels or state-derived actions.
• Episodes to keep each recording separate or apply another supported split strategy.

Select Convert to LeRobotDataset v3.0. Conversion runs in the background in a queue; monitor it until it reports completion and shows the local output path.

A "frame" is one message from the first enabled camera — the stream everything else is synced to — so FPS describes the camera rate, not the joint-state sampling rate.

Conversion skips episodes it cannot turn into sound training data and continues with the rest — for example a recording gap that dominates the episode's time span, or an FPS more than 25% above the rate that camera delivered in that episode. A conversion in which no episode survives fails outright, and some faults fail the whole job rather than skipping one episode. The Convert tab shows the converter log for a failed job, and that log names the reason.

The converted dataset is written next to the raw dataset:

~/.local/share/moveit_pro/trainer/recordings/<output-name>

It contains LeRobot data and meta folders. There is no videos folder: camera frames are stored as PNG images inside the data parquet files rather than as separate video files. Conversion does not upload data to any external service.

Plan for the extra disk space this costs. On a reference dataset, the image-mode output was roughly 11.65 times larger than the same data in the previous video-encoded format (469.9 MB of PNG frames versus 40.3 MB of video). Video-based storage is being evaluated for a future release.

Required Version

This feature requires MoveIt Pro version 10.0 or newer.

A completed job lists the skipped episodes under Skipped episodes on the Convert tab, one entry per reason. A job in which every episode was skipped fails instead of completing, so it has no such list; its failure log on the same tab carries the same reasons. Either way, the FPS reason reports both numbers: the frames per second the recording provides and the FPS the conversion requested. That reason is the only place that camera's own rate is shown. The obs value from step 4 is not guaranteed to match it, being a mean over the whole episode span across every enabled camera rather than that camera's median gap between frames — a single multi-second recording hiccup pulls the two apart on its own.

Conversion has no FPS setting of its own: it uses the FPS the dataset was pinned to and applies that one value to every episode in the folder. An episode skipped on FPS therefore cannot be rescued in place — record the take again into a new dataset created at an FPS no higher than the rate that entry reports, never raised to compensate for a slow camera. Episodes skipped over a recording gap or idle filtering have their own remedies, which their entries name.

Troubleshooting

• The Start recording button is unavailable. Create or select a dataset, enter a task description, and make sure the Training Config has been saved. If a warning icon appears next to the button, the dataset's pin does not match the current Training Config — hover or focus the icon for details.
• Conversion says the episode is empty. Confirm the configured camera and joint topics have live messages. If the robot did not move, turn off Trim idle frames and convert again.
• A camera is missing in playback or conversion. Open Training Config, correct that camera topic, then create a new dataset and record into it. The existing dataset is pinned to the old camera set and refuses the corrected recording. Existing MCAP files are not changed when you edit the Training Config.
• Recording is refused because the dataset was pinned with different options. The Start recording button is disabled and a warning icon appears beside it; hover or focus the icon to see which fields no longer match. Either restore those Training Config values (or enter the pinned rate as the labeled FPS override), or create a new dataset for the new settings.
• A folder under <TRAINER_DATA_ROOT>/recordings does not appear in the Dataset tab. Trainer lists a dataset only when it can read the rate its episodes were recorded at, from that dataset's own dataset.json or, for a converted dataset, its meta/info.json. A folder copied in by hand, or one whose metadata file is missing or damaged, is skipped and the Runtime log names it. Trainer never guesses a rate, because a wrong one mislabels every frame in the dataset. Record the demonstrations into a dataset created in Trainer.