VOXL Camera Server

Table of contents

1. Overview
   1. HAL3
   2. Exposure
      1. Manually Setting Exposure and Gain
   3. Output
   4. Inspecting
2. Configuration
3. Streaming Options
   1. Stream Over Web/HTTP using VOXL Portal
   2. Stream Over RTSP
   3. Stream to ROS
4. Source

Overview

VOXL Camera Server is built around the Modal Pipe Architecture (using libmodal_pipe) to serve up MIPI cameras to multiple clients.

HAL3

VOXL Camera Server uses Google’s HAL3 API to access raw camera data from qualcomm’s mm-qcamera-daemon pipeline.

Exposure

VOXL Camera Server primarily uses ModalAI’s libmodal_exposure for camera auto exposure/gain values, but this can also be changed to use the ISP’s exp/gain settings, or can be disabled altogether. Additionally, you can use voxl-send-command to manually send commands to update the exposure/gain of a specific camera. If a camera was using auto exposure/gain when such a command is sent, the AE will be disabled until the start_ae command is sent and the camera will sit at the requested exposure/gain.

Manually Setting Exposure and Gain

When an image sensor is configured to use a ModalAI exposure algorithm, auto_exposure_mode: modalai, one can manually set exposure and gain values from the command line. This can be useful for debugging image sensor performance.

voxl-send-command hires set_exp_gain <exposure> <gain>

For example:

voxl-send-command hires set_exp_gain 200 150

Output

The frames from camera server are published to /run/mpa via libmodal_pipe. The data is sent in a binary form with a metadata struct followed by the raw image data (in a format defined by the metadata). The metadata struct is currently source:

/**
 * The metadata for the camera image. One of these is sent before every frame
 */
typedef struct camera_image_metadata_t
{
    uint32_t magic_number;  ///< set to CAMERA_MAGIC_NUMBER
    int64_t  timestamp_ns;  ///< timestamp in apps-proc clock-monotonic of beginning of exposure
    int32_t  frame_id;      ///< iterator from 0++ starting from first frame when server starts on boot
    int16_t  width;         ///< image width in pixels
    int16_t  height;        ///< image height in bytes
    int32_t  size_bytes;    ///< size of the image, for stereo this is the size of both L&R together
    int32_t  stride;        ///< bytes per row
    int32_t  exposure_ns;   ///< exposure in microseconds
    int16_t  gain;          ///< ISO gain (100, 200, 400, etc..)
    int16_t  format;        ///< raw8, nv12, etc
    int32_t  reserved;
} __attribute__((packed)) camera_image_metadata_t;

The data will either be sent out in raw8 grayscale (for OV7251/other gray images) or YUV (for IMX214 or other color images). The complete list of image types that can be sent through a camera pipe can be found here.

Inspecting

The voxl-inspect-cam tool provides a simple way of inspecting camera frames in-terminal and the source serves as a good example for how to read from camera pipes on an embedded process.

Configuration

Whenever cameras are plugged/unplugged (which should only happen when the board is powered off), make sure to run voxl-configure-cameras. This script will generate a file at /etc/modalai/voxl-camera-server.conf containing the default settings for that camera configuration. If you want to change some aspects of the camera behavior such as disabling/framerate/resolution(for hires)/exposure setpoint you can do so by modifying that file and restarting camera server with systemctl restart voxl-camera-server.

Streaming Options

Stream Over Web/HTTP using VOXL Portal

VOXL Portal provides a simple, easy-to-use web interface for viewing images from camera server or any other service providing an image on VOXL.

Stream Over RTSP

The VOXL Streamer application can be used to stream video from voxl-camera-server over RTSP.

Stream to ROS

The VOXL MPA to ROS node can be used to provide data coming out of camera server to ROS.

Source

The source code for VOXL Camera Server can be found here