Calibrate Cameras

Calibrate camera intrinsics and stereo extrinsics on-board with voxl-calibrate-camera.

Table of contents

1. Overview
2. Before You Start
3. Recommended Chessboard
4. Command Line Options
5. Monocular Calibration Process
6. Stereo Calibration Process
7. After Calibration
8. Troubleshooting
9. Legacy ROS Calibration

Overview

VOXL Camera Calibration is a package in voxl-suite >= 0.5.0 for calibrating cameras on-board on voxl. If you are on an older voxl-suite image, we recommend that you upgrade and use this tool, but you can find a link to the deprecated ROS calibration process at the bottom of this page.

Use of this tool requires the ability to view the overlay image coming off of voxl. We recommend using the VOXL Web Portal to view the overlay images.

Before You Start

• Find the camera pipe name. The first argument to the calibrator is the name of the camera pipe published by voxl-camera-server, for example tracking, tracking_front, tracking_down, stereo_front, or stereo_rear. List the pipes on your VOXL with:

  voxl-list-pipes
  

• Use the right stream. Calibrate on the grayscale or raw pipe for a camera, not a color or encoded stream. The calibrator exits with Received invalid camera format if it is given the wrong one.
• Know your board. The -s size is the count of inner corners, not squares, and -l is the edge length of one square in meters. Measure the printed board rather than trusting the file it was printed from.
• Open the overlay before you start collecting. Run the command, open voxl-portal in a browser, and select the camera_calibrator_overlay stream. The overlay shows the thresholded image by default, which makes lighting problems obvious. Add -r to see the raw image instead.
• Tracking cameras use the fisheye model. Add -f for any tracking camera. If you leave it off for an ar0144 tracking camera, the tool warns and waits: press Ctrl-C and rerun with -f.

Recommended Chessboard

The size and spacing of the chessboard is configurable on the command line. Internally, ModalAI uses a 5x6 with 65.5mm spacing on a stiff, foam-backed board with matte finish to reduce glare. This is the default used by the calibrator tool unless a different size and spacing is set.

Note that the size is based on OpenCV’s definition of a chessboard and matches the ROS calibrator tool. That is to say, the size (4x7, 5x6, etc) is the count of the number of inner corners which is what is actually being detected by the algorithm (not number of squares). This is usually #squares-1 noting that partial outer squares should not be counted.

Here is an image for a 6x9: https://raw.githubusercontent.com/opencv/opencv/4.x/doc/pattern.png

Command Line Options

voxl-calibrate-camera <camera-pipe> [options]

The -t / --threshold option from earlier releases has been removed. The thresholded overlay is now the default.

Examples using a 6x8 checkerboard with 0.038 m squares:

voxl-calibrate-camera tracking -f -s 6x8 -l 0.038
voxl-calibrate-camera stereo_front -s 6x8 -l 0.038
voxl-calibrate-camera stereo_rear -s 6x8 -l 0.038

Monocular Calibration Process

To calibrate a monocular camera, such as the tracking camera on voxl, run the command below, open the web portal, and fill the rectangles like such:

voxl-calibrate-camera tracking --fisheye

Once you’ve filled all the rectangles, the calibration will run, which will take a few seconds, and then the calibration will be printed to the terminal window. If the calibration was successful, it will also write to /data/modalai/opencv_{camera-name}_intrinsics.yml

A rectangle accepts a sample only when every detected corner lies inside it and the chessboard covers roughly 40 to 50 percent of the rectangle’s area (the exact threshold depends on the lens type). Drawn corner markers alone do not count as a sample. The rectangle outline changes from red toward green as coverage increases. If collection stalls with corners visible, move the board closer to the camera until it fills most of the rectangle, keep the whole board inside the outline, and hold it still. Each rectangle needs 2 or 3 accepted samples before the next one appears.

These intrinsics files are also consumed by voxl-camera-server for Electronic Image Stabilization. Calibrating a camera used with EIS improves stabilization quality.

voxl2:/$ voxl-calibrate-camera tracking -f
Matrix
[277.7090664510777, 0, 290.6865358454301;
 0, 277.5687125242331, 240.1597738541476;
 0, 0, 1]
Distortion
[-0.006319990900085937;
 0.00155893534296691;
 0;
 0]
distortion_model: fisheye
Re-projection error reported by calibrateCamera: 0.172684
Calibration Succeded!

Writing data to: /data/modalai/opencv_tracking_intrinsics.yml
Saved!
Exiting Cleanly

When calibrating the standard OV7251 Tracking Camera, make sure to use the -f or --fisheye flag to make sure that the calibrator uses the fisheye lens distortion model.

Stereo Calibration Process

To calibrate a stereo camera, run the command below, open the web portal, and fill the rectangles like such:

voxl-calibrate-camera stereo_front

The stereo calibrator runs in two stages.

Stage 1, intrinsics. The calibrator calculates the intrinsics for each camera separately, so you will need to fill in all of the left rectangles, then all of the right rectangles. The same fill rule applies as for monocular calibration: the board must sit fully inside the rectangle and cover most of it before a sample is accepted.

Stage 2, extrinsics. By default the calibrator then guides you through a second set of targets that need the board visible in both cameras at once: two large close-range boxes (one per camera) followed by five small far-range boxes in the corners and center. Hold the board flat and still in each box until it is accepted. When the last box is done the extrinsics are calculated automatically.

If you prefer to choose the extrinsics frames yourself, run with -n / --no-targets. The calibrator then prints Sampling overlapping images now; press Enter in the terminal to capture a frame while the board is fully visible in both cameras, and type stop when you have enough. Take at least ten frames spread across the overlapping field of view at varied angles and distances. To redo only the extrinsics with existing intrinsics use -e, and to calibrate intrinsics only use -g.

At the end the tool prints whether it detected a horizontal or vertical pair and the distance between the cameras. Check that this baseline matches the physical spacing on your camera module.

If everything has been done properly, you will see output like this (this run used --no-targets; a target-guided run prints the same results without the sampling prompts):

voxl2:/$ voxl-calibrate-camera stereo
Sampling overlapping images now
Enter an empty line to take a sample or "stop" to finish sampling

Taking Frame

Taking Frame
stop
Stopping sampling
Calibrating Left Camera
Matrix
[493.7322756669245, 0, 306.7896643027532;
 0, 493.1380517350153, 217.4680594645219;
 0, 0, 1]
Distortion
[-0.1698366125016952;
 0.06988854948207439;
 -0.001138684086637882;
 -0.0007932118450081079;
 0]
distortion_model: plumb_bob
Re-projection error reported by calibrateCamera: 0.177678
Calibration Succeded!
Calibrating Right Camera
Matrix
[496.1483207581481, 0, 320.9542358074322;
 0, 495.5756728109903, 223.7654477880258;
 0, 0, 1]
Distortion
[-0.1725992238299667;
 0.06664783196280108;
 -0.00172072316840736;
 -0.0004606934467252117;
 0]
distortion_model: plumb_bob
Re-projection error reported by calibrateCamera: 0.179598
Calibration Succeded!
Calibrating Extrinsics
2 frames will be processed
R
[0.999777546053251, -0.0004080345076243199, 0.02108771954432934;
 0.002055883177889161, 0.9969371281114418, -0.07818015053240687;
 -0.02099123036171473, 0.07820611293723473, 0.9967161943839129]
T
[-0.08011754799309605;
 -0.001690038045698758;
 -0.0008435695774040179]
Re-projection error reported by stereoCalibrate: 0.259720
Extrinsics Calibration Succeded!

Saved intrinsics to: /data/modalai/opencv_stereo_intrinsics.yml
Saved extrinsics to: /data/modalai/opencv_stereo_extrinsics.yml
Exiting Cleanly

After Calibration

• The results are written to /data/modalai/opencv_<pipe>_intrinsics.yml and, for stereo pairs, /data/modalai/opencv_<pipe>_extrinsics.yml. Each file records the reprojection_error of the run.
• Restart the services that consume the calibration so they pick up the new files, for example voxl-open-vins-server or voxl-qvio-server for VIO and voxl-camera-server for EIS. See Debugging Services for the restart commands.

• Confirm every required file is present:

  voxl-check-calibration
  

• A calibration is only valid for the resolution it was captured at. If you change the camera resolution or swap the camera hardware, calibrate again. See Verifying Camera Calibration on the VIO page for how to read the files and the resolution check.

Troubleshooting

The allowed extrinsics reprojection error is 2.0 px at 640x480 and scales with the image area.

Legacy ROS Calibration

The deprecated ROS-based process is documented on the VOXL ROS Camera Calibration page for users on voxl-suite releases before 0.5.0.