Link Search Menu Expand Document

Camera and IMU Coordinate Frames

Table of contents

  1. Setup
  2. Overview
    1. Primary and Secondary IMU Coordinate Frames
    2. Camera Coordinate Frame
    3. Camera Position and Orientation Relative to IMU
  3. Examples
  4. Calculating your Own IMU to Camera Relation
    1. Translational Offset
    2. Rotation

Setup

To set up a factory spec for any of modalai’s drone bodies, you can use voxl-configure-extrinsics (this is a part of VOXL Configure MPA, so if you ran that then you’re good to go!). This will place a file at /etc/modalai/extrinsics.conf that is used by various MPA services.

Overview

Primary and Secondary IMU Coordinate Frames

On VOXL, there is one ICM-20948 designated IMU1 (primary) and one MPU-9250 designated IMU0 (secondary). The parameters listed in this page are relative to the primary IMU. Their positions on the VOXL PCB are as follows:

voxl-core-imu-locations

Camera Coordinate Frame

VOXL uses the convention of X to the right, Y down, and Z out the front of the lens.

Camera Coordinate Frame

Camera Position and Orientation Relative to IMU

The position and orientation of the tracking camera relative to the IMU must be correct for Visual Inertial Odometry to function.

Examples

When running voxl-configure-extrinsics, you will be provided with choices between the presets for ModalAI’s various platforms. These have been created to accurately represent the real-world transformations between various points on the platforms. This is the example representing the M500 drone or the voxl flight deck:

{
  "name": "M500_flight_deck",
  "extrinsics": [{
      "parent": "imu1",
      "child":  "imu0",
      "T_child_wrt_parent": [-0.0484, 0.037, 0.002],
      "RPY_parent_to_child":  [0, 0, 0]
    }, {
      "parent": "imu0",
      "child":  "tracking",
      "T_child_wrt_parent": [0.065, -0.014, 0.013],
      "RPY_parent_to_child":  [0, 45, 90]
    }, {
      "parent": "imu1",
      "child":  "tracking",
      "T_child_wrt_parent": [0.017, 0.015, 0.013],
      "RPY_parent_to_child":  [0, 45, 90]
    }, {
      "parent": "body",
      "child":  "imu0",
      "T_child_wrt_parent": [0.02, 0.014, -0.008],
      "RPY_parent_to_child":  [0, 0, 0]
    }, {
      "parent": "body",
      "child":  "imu1",
      "T_child_wrt_parent": [0.068, -0.015, -0.008],
      "RPY_parent_to_child":  [0, 0, 0]
    }, {
      "parent": "body",
      "child":  "stereo_l",
      "T_child_wrt_parent": [0.1, -0.04, 0],
      "RPY_parent_to_child":  [0, 90, 90]
    }, {
      "parent": "body",
      "child":  "tof",
      "T_child_wrt_parent": [0.1, 0, 0],
      "RPY_parent_to_child":  [0, 90, 90]
    }, {
      "parent": "body",
      "child":  "ground",
      "T_child_wrt_parent": [0, 0, 0.1],
      "RPY_parent_to_child":  [0, 0, 0]
    }]
}

Note

The stereo cameras go through a precise calibration procedure to determine the orientation between them, so only the left stereo camera is provided in the extrinsics config file, the right camera has a specified relation to the left camera in /data/modalai/opencv_stereo_extrinsics.conf"

Calculating your Own IMU to Camera Relation

If you have various pieces in your own format or have modified a voxl product, you’ll need to modify the extrinsics file to ensure that all of the services work properly.

Translational Offset

Use a CAD model or a pair of calipers to measure the distance from the centroid of IMU1 to the camera module. Remember the values you enter into the config file are in units of meters.

Rotation

The rotation specs for these transforms are in the form of Roll, Pitch, Yaw (RPY), in units of degrees. A good example of this can be seen in the imu to tracking sections: showing the 45 degree downward facing camera also rotated to have z face forward:

{
  "parent": "imu1",
  "child":  "tracking",
  "T_child_wrt_parent": [0.017, 0.015, 0.013],
  "RPY_parent_to_child":  [0, 45, 90]
}