Skip to main content
Version: 8

Meta Quest for Teleop Setup Guide

The Meta Quest is a great tool for ML model data collection. It requires a little Unity app running on the Quest device. We will walk you through how to set this up in the following.

Meta Quest

Enable Developer Mode​

  1. Enable “Developer Mode” on your device so that it can run your own application.
    • Run the “Meta Horizon” app on your phone
    • Connect to the Meta device
    • In the device settings->Headset Settings->Developer Mode, turn ‘Developer Mode’ on.
    • Note: You may have to log in to your Facebook account and create an “organization” for you to be able to turn developer mode on.

Build Meta Quest APK from source​

If you need to build the Meta Quest app from source, follow these steps.

The app has been developed using Unity, and therefore, you will need to install Unity to build it.

  1. Clone the app repository somewhere in your local system.
  2. Install Unity Hub following these instructions. Start the hub, log in, and install Unity 6.1.
  3. Open the project from the github repository, or from a local folder if cloned already.
  4. The app needs the Android SDK to be built. In File -> Build Profiles enable “Android”. This may require downloading the Android SDK if you haven’t already.
  5. In Edit->Project Settings, go to the “XR Plugin Management” section and disable “OpenXR”, enable “Oculus”.
  6. Under XR Plugin Management->Oculus, make sure your device is checked in “Target Devices”. If you have a Quest 1, make sure to disable “Quest 3”, otherwise, support for the Quest 1 won’t be enabled 🤷
  7. In the main menu, Robotics->ROS Settings, set the IP address of the computer where you will be running all the ROS2 nodes, e.g., where MoveIt Pro runs.
  8. Connect the Meta Quest to your computer via a USB cable. You should see a message in your headset asking to confirm the debug connection to the computer and data sharing.
  9. Under File->Build Profiles->Android->Run Device, you should see your Meta Quest device listed. Select it.
  10. Click Build and Run, which should build the application, push it to the device, and run it.

Run the Meta Quest APK​

To run the Meta Quest app after it has already been installed:

  1. Put the headset on
  2. Open the App Library
  3. Choose “Unknown sources”. That should show the app.
  4. Click on the app.

When you run the app, you should see a blue sky, a grid, and the two remote controllers.

Connect to ROS​

note

This app collects the state of the Meta Quest controllers and publishes it to ROS via the Unity ROS TCP Connector.

To be able to see the topics and data published by the Meta Quest app, you will need a dedicated ROS2 node running on the host computer. This node will be the host endpoint for the connection to the Meta Quest.

  1. Clone Unity’s ROS-TCP-Endpoint repo (main-ros2 branch) in your workspace and build it with the rest of your ROS2 packages.

  2. Start the endpoint with the following command, using the IP of the host computer, where you are running the ROS ecosystem, e.g:

    ros2 run ros_tcp_endpoint default_server_endpoint --ros-args -p ROS_IP:=192.168.86.25

You should now be able to see the topics published by the headset, including TF.

Calibrate the Controller​

info

Understand how the controller pose is reported The controller pose is reported as an Odometry message in the /right_controller_odom topic. Its contents can be received in MoveIt Pro using the GetOdom Behavior.

The robot's initial controller pose is set near the VR world origin, adjusted for your height. In RViz, with TF enabled, you can visualize the target position relative to the world frame.

To manipulate this target position using the controller:

  1. Press the clutch button, which effectively "grabs" the moving target.
  2. Release the clutch to "drop" the target at its current location.
warning

Before initiating the "Data collection" Objective in MoveIt Pro, it's crucial to manually position the target pose close to the robot's end-effector. This is because the Objective will immediately activate a control loop, guiding the robot's end-effector towards the controller target pose, so a close initial proximity is desired.

Summary​

You should now have the Quest ready to talk to MoveIt Pro. Next, checkout our training data collection tutorial.