Skip to main content
Version: 8

Set Up Third-Party Simulators

This guide provides instructions for connecting external simulators (other than MuJoCo) to MoveIt Pro. While we officially support MuJoCo as our primary simulation engine, some customers have successfully connected NVIDIA Isaac Sim and Gazebo following the approaches outlined below.

Official Support

We only officially support MuJoCo as our simulation engine. While this guide provides general guidance for connecting other simulators, these configurations may require additional troubleshooting and are not covered under standard support agreements.

Overview​

The key to connecting any external simulator to MoveIt Pro is understanding how ROS 2 and ros2_control integration works. This guide covers two main approaches:

  1. Direct ros2_control integration - For simulators with native ros2_control plugins (e.g., MoveIt Pro's simulator and Gazebo)
  2. Topic-based integration - For simulators without native ros2_control support (e.g., Isaac Sim)

Understanding MoveIt Pro's Communication Architecture​

MoveIt Pro uses different communication patterns for different types of hardware:

High-Bandwidth, Low-Latency Communication (Actuators)​

ros2_control is our framework of choice for reading and writing to actuated hardware when we need high-bandwidth (500Hz - 1KHz), low-latency communication, meaning no communication through middleware.

Standard ROS 2 Communication (Sensors)​

For other sensors, such as cameras, where high-frequency, low-latency communication is not needed, we use standard ROS 2 message definitions and middleware topics.

  • The MoveIt Pro UI automatically finds all camera topics
  • Behaviors that interact with cameras will need to be updated to reference the topic name your simulator is publishing the ROS message type to
  • We assume your simulator natively talks ROS 2 or provides a bridge to talk ROS 2 for camera-type sensors

Actuator Integration Approaches​

Option 1: Direct ros2_control Integration​

If your simulator (such as Gazebo) provides a direct, memory-sharing integration with ros2_control, you can use this approach:

  1. Update the hardware plugin configuration: Modify the <hardware> <plugin> tag in your robot description URDF/Xacro file referenced in your config.yaml's hardware.robot_description.urdf entry.

    For example, our MuJoCo plugin is called picknik_mujoco_ros/MujocoSystem. Your simulator would have its own equivalent plugin name.

    For detailed information about hardware configuration settings, see the config.yaml reference tutorial.

    For more details about ros2_control hardware plugins and controller setup, see our MuJoCo migration guide which demonstrates these concepts in the context of our officially supported simulator.

  2. Configure ros2_control controllers: Set up the ros2_control controllers configuration and point your config.yaml ros2_control.config entry to that file.

    For information about what controllers need to be set up for teleoperation, see the configure custom robot guide.

Option 2: Topic-Based Integration​

If your simulator does not provide a native ros2_control plugin (such as Isaac Sim), you can connect it via middleware using the topic_based_ros2_control package and its topic_based_ros2_control/TopicBasedSystem hardware plugin.

Latency Consideration

Note that this approach will add latency to the responsiveness of the control algorithms due to the middleware latency. This may impact performance for applications requiring high-frequency control loops.

Setting Up Topic-Based Integration​

  1. Install topic_based_ros2_control: Ensure the topic_based_ros2_control package is available in your ROS 2 workspace.

  2. Configure the hardware plugin: In your robot description URDF, use the topic_based_ros2_control/TopicBasedSystem plugin instead of a native simulator plugin.

  3. Set up topic mapping: Configure your simulator to publish joint states and subscribe to joint commands using standard ROS 2 topics that the topic-based system expects.

  4. Configure controllers: Set up your ros2_control controllers as with the direct integration approach above.

Sensor Integration​

For sensors like cameras, lidars, and IMUs:

  1. Ensure ROS 2 compatibility: Make sure your simulator publishes sensor data using standard ROS 2 message types (e.g., sensor_msgs/Image, sensor_msgs/PointCloud2, sensor_msgs/Imu).

  2. Topic discovery: The MoveIt Pro UI will automatically discover camera topics. Ensure your simulator publishes to appropriately named topics.

  3. Update Behaviors: Modify any Behaviors that interact with sensors to reference the correct topic names that your simulator publishes to.

  4. Frame configuration: Ensure proper TF frame relationships between your robot and sensor frames are published by your simulator.

Configuration Steps​

1. Update config.yaml​

Create or modify your robot configuration package's config.yaml file to work with your external simulator:

# Enable simulation mode
hardware:
  simulated: true
  
  # Set based on your simulator's requirements
  launch_control_node: true  # Set to false if simulator manages controller
  
  # Configure simulation launch file
  simulated_hardware_launch_file:
    package: "your_simulator_config"  
    path: "launch/your_simulator.launch.py"

# Enable simulation time if your simulator publishes /clock
ros_global_params:
  use_sim_time: true

# Point to your ros2_control configuration  
ros2_control:
  config:
    package: "your_config_package"
    path: "config/ros2_controllers.yaml"

2. Robot Description Updates​

Update your robot's URDF/XACRO files to include the appropriate ros2_control hardware plugin:

For direct integration:

<ros2_control name="your_robot" type="system">
  <hardware>
    <plugin>your_simulator/YourSimulatorSystem</plugin>
    <!-- Simulator-specific parameters -->
  </hardware>
  <!-- Joint configurations -->
</ros2_control>

For topic-based integration:

<ros2_control name="your_robot" type="system">
  <hardware>
    <plugin>topic_based_ros2_control/TopicBasedSystem</plugin>
    <param name="joint_commands_topic">/joint_commands</param>
    <param name="joint_states_topic">/joint_states</param>
  </hardware>
  <!-- Joint configurations -->
</ros2_control>

3. Controller Configuration​

Create a ros2_control controllers configuration file that matches your robot's joint interface requirements.

For more details on controller configuration, refer to the config.yaml reference documentation.

Testing Your Configuration​

  1. Start your external simulator with the robot model loaded

  2. Launch MoveIt Pro with your custom configuration:

    moveit_pro run -c your_simulator_config

  3. Verify connections:

    • Check that joint states are being received: ros2 topic echo /joint_states
    • Verify that camera topics are available: ros2 topic list | grep camera
    • Test basic robot motion through the MoveIt Pro interface

Common Issues and Troubleshooting​

Joint Interface Mismatches​

Ensure that your simulator's joint interfaces match what's configured in your ros2_control configuration. Common issues include:

  • Position vs. velocity command interfaces
  • Missing state interfaces
  • Incorrect joint names

Timing and Synchronization​

When using use_sim_time: true, ensure all nodes are properly synchronized with the simulator's clock.

Sensor Frame Configuration​

Verify that your sensor frames are properly defined in the robot description and that transforms are being published correctly.

Getting Additional Help​

This guide provides general principles for connecting external simulators to MoveIt Pro. Each simulator has its own specific requirements and configuration details that may need additional customization.

For more help on setting up external simulators, please contact support. While we do not guarantee support of third-party simulators, we may provide more detailed instructions for specific simulators like Isaac Sim in the future based on customer demand.

Additional Resources​