Skip to main content
Version: 9

Real-time Performance and Trajectory Jitter Troubleshooting

Arm movements have noticeable hitches and jitter in real-time setup​

Problem: When running MoveIt Pro drivers container in real-time on x86/Intel or ARM/Nvidia Jetson platforms, arm motions show noticeable hitches and jitter that can be seen and felt during movement, even with real-time kernel and priority settings configured.

Solution: This issue typically manifests in two different ways that require different solutions:

For Large Jerky Motions (Real-time Performance Issues)​

If you see large, irregular jerky motions, this indicates real-time performance problems. Try these Docker container CPU optimization settings in your docker-compose.yaml:

services:
  drivers:
    cap_add:
      - IPC_LOCK
      - SYS_NICE
      - SYS_RESOURCE
    # ... existing configuration ...
    # Ensures the drivers container has RT priority
    ulimits:
      rtprio: 99
      memlock:
        soft: -1
        hard: -1
    # CPU optimization options
    cpuset: "0-3"  # Pin to specific CPUs for isolation
    cpu_shares: 2048    # Increase CPU priority (>1024) for CFS processes

See the relevant Docker documentation for details on each of these optimizations. The ros2_control Controller Manager documentation also has relevant information on ros2_control recommended configuration.

For advanced real-time optimization, consider CPU isolation using boot parameters:

# Add to boot parameters
isolcpus=0,1 nohz_full=0,1 rcu_nocbs=0,1

Then pin the drivers container to those isolated CPUs:

services:
  drivers:
    cpuset-cpus: "0,1"  # Use isolated CPUs

For Small Regular "Staccato" Jitter (Trajectory Configuration Issues)​

If you see small, regular jitter where the arm appears to go to zero velocity at each trajectory step (creating a staccato effect), this is likely a trajectory configuration issue, not a real-time performance problem.

This can be caused by using both position and velocity command interfaces simultaneously. To fix this:

  1. Remove the velocity command_interface from your ros_controllers.yaml file
  2. Remove the velocity command_interface from your robot's ros2_control xacro file
  3. Ensure only the position command interface is used for joints
tip

If real-time optimization eliminates large jerky motions but small regular jitter persists, you've likely resolved the real-time issues and should focus on trajectory configuration instead.

Real-time kernel crashes with certain security policy settings​

Problem: The security application huntress-agent causes system crashes when it tries to sleep while holding a RT mutex.

sudo journalctl -b -k 1 will contain an error similar to: kernel: BUG: scheduling while atomic: kworker/dying/9/0x00000000

Solution: Remove huntress-agent or disable the RT kernel. The security package cannot be installed on a RT kernel, but if the application was installed before the RT kernel patch it will still run.