Real-time Performance and Trajectory Jitter Troubleshooting

Arm movements have noticeable hitches and jitter in real-time setup

Problem: When running the MoveIt Pro drivers container in real-time on x86/Intel or ARM/NVIDIA Jetson platforms, arm motions show noticeable hitches and jitter that can be seen and felt during movement, even with real-time kernel and priority settings configured.

Solution: This issue typically manifests in two different ways that require different solutions:

For Large Jerky Motions (Real-time Performance Issues)

If you see large, irregular jerky motions, this indicates real-time performance problems. Try these Docker container CPU optimization settings in your docker-compose.yaml:

services:
  drivers:
    cap_add:
      - IPC_LOCK
      - SYS_NICE
      - SYS_RESOURCE
    # ... existing configuration ...
    # Ensures the drivers container has RT priority
    ulimits:
      rtprio: 99
      memlock:
        soft: -1
        hard: -1
    # CPU optimization options
    cpuset: "0-3"  # Pin to specific CPUs for isolation
    cpu_shares: 2048    # Increase CPU priority (>1024) for CFS processes

See the relevant Docker documentation for details on each of these optimizations. The ros2_control Controller Manager documentation also has relevant information on ros2_control recommended configuration.

For advanced real-time optimization, consider CPU isolation using boot parameters:

# Add to boot parameters
isolcpus=0,1 nohz_full=0,1 rcu_nocbs=0,1

Then pin the drivers container to those isolated CPUs:

services:
  drivers:
    cpuset-cpus: "0,1"  # Use isolated CPUs

For Small Regular "Staccato" Jitter (Trajectory Configuration Issues)

If you see small, regular jitter where the arm appears to go to zero velocity at each trajectory step (creating a staccato effect), this is likely a trajectory configuration issue, not a real-time performance problem.

This can be caused by using both position and velocity command interfaces simultaneously. To fix this:

1. Remove the velocity command_interface from your ros_controllers.yaml file
2. Remove the velocity command_interface from your robot's ros2_control xacro file
3. Ensure only the position command interface is used for joints

tip

If real-time optimization eliminates large jerky motions but small regular jitter persists, you've likely resolved the real-time issues and should focus on trajectory configuration instead.

---

Real-time kernel crashes with certain security policy settings

Problem: The security application huntress-agent causes system crashes when it tries to sleep while holding a RT mutex.

sudo journalctl -b -k 1 will contain an error similar to: kernel: BUG: scheduling while atomic: kworker/dying/9/0x00000000

Solution: Remove huntress-agent or disable the RT kernel. The security package cannot be installed on a RT kernel, but if the application was installed before the RT kernel patch it will still run.

---