VOXL Camera Server
Table of contents
Overview
VOXL Camera Server is built around the Modal Pipe Architecture (using libmodal_pipe) to serve up MIPI cameras to multiple clients.
HAL3
VOXL Camera Server uses Google’s HAL3 API to access raw camera data from qualcomm’s mm-qcamera-daemon
pipeline. The code that makes the APi calls can be found here.
Exposure
VOXL Camera Server primarily uses ModalAI’s libmodal_exposure for camera auto exposure/gain values, but this can also be changed to use the ISP’s exp/gain settings, or can be disabled altogether. Additionally, you can use voxl-send-command
to manually send commands to update the exposure/gain of a specific camera. If a camera was using auto exposure/gain when such a command is sent, the AE will be disabled until the start_ae
command is sent and the camera will sit at the requested exposure/gain.
Output
The frames from camera server are published to /run/mpa
via libmodal_pipe. The data is sent in a binary form with a metadata struct followed by the raw image data (in a format defined by the metadata). The metadata struct is currently source:
/**
* The metadata for the camera image. One of these is sent before every frame
*/
typedef struct camera_image_metadata_t
{
uint32_t magic_number; ///< set to CAMERA_MAGIC_NUMBER
int64_t timestamp_ns; ///< timestamp in apps-proc clock-monotonic of beginning of exposure
int32_t frame_id; ///< iterator from 0++ starting from first frame when server starts on boot
int16_t width; ///< image width in pixels
int16_t height; ///< image height in bytes
int32_t size_bytes; ///< size of the image, for stereo this is the size of both L&R together
int32_t stride; ///< bytes per row
int32_t exposure_ns; ///< exposure in microseconds
int16_t gain; ///< ISO gain (100, 200, 400, etc..)
int16_t format; ///< raw8, nv12, etc
int32_t reserved;
} __attribute__((packed)) camera_image_metadata_t;
The data will either be sent out in raw8 grayscale (for OV7251/other gray images) or YUV (for IMX214 or other color images). The complete list of image types that can be sent through a camera pipe can be found here.
Inspecting
The voxl-inspect-cam tool provides a simple way of inspecting camera frames in-terminal and the source serves as a good example for how to read from camera pipes on an embedded process.
Configuration
Whenever cameras are plugged/unplugged (which should always happen when the board is powered off), make sure to run voxl-configure-cameras. This script will generate a file at /etc/modalai/voxl-camera-server.conf
containing the default settings for that camera configuration. If you want to change some aspects of the camera behavior such as disabling/framerate/resolution(for hires)/exposure setpoint you can do so by modifying that file and restarting camera server with systemctl restart voxl-camera-server
.
Streaming Options
Stream Over Web/HTTP using VOXL Portal
VOXL Portal provides a simple, easy-to-use web interface for viewing images from camera server or any other service providing an image on VOXL.
Stream Over RTSP
The VOXL Streamer application can be used to stream video from voxl-camera-server over RTSP.
Stream to ROS
The VOXL MPA to ROS node can be used to provide data coming out of camera server to ROS.
Source
The source code for VOXL Camera Server can be found here