Camera and IMU Coordinate Frames

Table of contents

  1. Primary and Secondary IMU Coordinate Frames
  2. Camera Coordinate Frame
  3. Camera Position and Orientation Relative to IMU
  4. Examples
  5. Calculating your Own IMU to Camera Relation
    1. Translational Offset
    2. Rotation

Primary and Secondary IMU Coordinate Frames

On VOXL, there is one ICM-20948 designated IMU1 (primary) and one MPU-9250 designated IMU0 (secondary). The parameters listed in this page are relative to the primary IMU. Their positions on the VOXL PCB are as follows:

voxl-core-imu-locations

Camera Coordinate Frame

SNAV and the ModalAI Vision Lib use the convention of X to the right, Y down, and Z out the front of the lens.

Camera Coordinate Frame

Camera Position and Orientation Relative to IMU

The position and orientation of the tracking camera relative to the IMU must be correct for Visual Inertial Odometry to function. For SNAV and the ModalAI Vision Lib, this relation is defined in /etc/snav/mount.snav_dft_vio_app.xml

The TBC values are the XYZ offset FROM the origin of the IMU frame TO the origin of the camera frame represented in the IMU frame.

The OMBC values are the axis-angle representation of the rotation of the camera frame relative to the IMU1 frame.

Examples

When configuring SNAV or the ModalAI Vision lib as described here, you are given a list of pre-set camera to IMU relations to choose from which cover all options for ModalAI-provided development platforms. Three of those are provided here as examples.

IMU1 to tracking camera for VOXL in Red Tray:

  <SnavMountDownward>
    <!-- Vector from the origin of the IMU1 frame to the origin of the camera
         frame represented in the IMU frame -->
    <param name="mv_vislam_tbc_1" value="0.0152"/>
    <param name="mv_vislam_tbc_2" value="0.0021"/>
    <param name="mv_vislam_tbc_3" value="0.0095"/>

    <!-- Axis-angle representation of the rotation of the camera frame relative
         to the IMU1 frame  -->
    <param name="mv_vislam_ombc_1" value="-2.3561945f"/>
    <param name="mv_vislam_ombc_2" value="0.0f"/>
    <param name="mv_vislam_ombc_3" value="0.0f"/>

IMU1 to tracking camera for VOXL-Cam with right-side-up tracking Camera

  <SnavMountDownward>
    <!-- Vector from the origin of the IMU frame to the origin of the camera
         frame represented in the IMU frame -->
    <param name="mv_vislam_tbc_1" value="0.02715"/>
    <param name="mv_vislam_tbc_2" value="-0.0344"/>
    <param name="mv_vislam_tbc_3" value="0.0088"/>
    <!-- Axis-angle representation of the rotation of the camera frame relative
         to the IMU1 frame  -->
    <param name="mv_vislam_ombc_1" value="1.7599884"/>
    <param name="mv_vislam_ombc_2" value="1.7599884f"/>
    <param name="mv_vislam_ombc_3" value="0.7290111f"/>

IMU1 to tracking camera for VOXL-Cam with upside-down tracking

  <SnavMountDownward>
    <!-- Vector from the origin of the IMU frame to the origin of the camera
         frame represented in the IMU frame -->
    <param name="mv_vislam_tbc_1" value="0.02715"/>
    <param name="mv_vislam_tbc_2" value="-0.0344"/>
    <param name="mv_vislam_tbc_3" value="0.0088"/>

    <!-- Axis-angle representation of the rotation of the camera frame relative
         to the IMU frame  -->
    <param name="mv_vislam_ombc_1" value="-1.7599884"/>
    <param name="mv_vislam_ombc_2" value="1.7599884f"/>
    <param name="mv_vislam_ombc_3" value="-0.7290111f"/>

Calculating your Own IMU to Camera Relation

Translational Offset

Use a CAD model or a pair of calipers to measure the distance from the centroid of IMU1 to the camera module. Remember the tbc values you enter into the config file are in units of meters. The numbers don’t have to be perfect, within a few millimeters is fine. Refer again to the axis directions in the diagram above.

Rotation

Both the IMU and Camera coordinate frames are right-handed. We suggest putting your right hand into the following shape, lining it up with IMU1 on your VOXL, and assign XYZ to each finger however you feel most comfortable. Whichever finger you pick to correspond to XYZ, remain consistent through this process!!!

right-hand-coordinate

Now line up your right hand with the camera coordinate frame with Z pointing out the optical axis, X to the right, and Y down. Now write down the relation between this two coordinate frames however you feel most comfortable, eg rotation matrix, Euler/Tait-Bryan angles, axis-angle, etc.

The 3 rotation numbers that need to go into the config file are an axis-angle representation (reference) where the magnitude of the given vector is the rotation in radians.

If you are uncomfortable converting the rotation to axis-angle representation by hand, we suggest using this excellent online tool: https://www.andre-gaschler.com/rotationconverter. All due credit to the author of this tool.

Once again, these new values to into /etc/snav/mount.snav_dft_vio_app.xml