Use MPA Camera Streams in GStreamer

Table of contents

1. Overview
2. Check the Target
3. Read a Camera Pipe
   1. Supported Formats
   2. Source Properties
4. Serve a Custom Pipeline over RTSP
   1. Launcher Options
5. Troubleshooting

Overview

The mpasrc GStreamer source element lets you consume MPA camera streams in your own pipelines. It manages the Modal Pipe connection, reads camera metadata, and outputs video buffers with GStreamer caps (format, dimensions, and framerate) and timestamps. Applications can use it through GStreamer without implementing their own Modal Pipe client and appsrc feeding loop.

All three components ship in the voxl-streamer package:

Use voxl-streamer for the configured camera service and its QGroundControl integration through voxl-mavcam-manager. Use gst-launch-1.0 or gst-rtsp-launch to experiment with custom pipelines. Neither utility reads voxl-streamer.conf; specify the source and downstream processing in the pipeline itself.

Check the Target

Run the examples on VOXL, where the camera pipes exist, after installing the voxl-streamer package. The package installs the plugin at /usr/lib/aarch64-linux-gnu/gstreamer-1.0/libgstmpa.so and the launcher at /usr/bin/gst-rtsp-launch.

gst-inspect-1.0 mpasrc
gst-rtsp-launch --help
voxl-list-pipes

gst-inspect-1.0 should report the Modal Pipe camera source, its source pad, and its properties. Start the appropriate publisher, such as voxl-camera-server or voxl-uvc-server, and choose a camera pipe from voxl-list-pipes. The examples use tracking and hires_small_encoded; substitute pipes available on your system. The voxl-streamer service does not need to be running to use mpasrc.

The target also needs each downstream GStreamer element used in the chosen example. Check individual elements with gst-inspect-1.0, for example gst-inspect-1.0 h265parse.

Read a Camera Pipe

For a headless check, read 60 buffers from a raw tracking pipe, print the negotiated caps, and discard the video:

gst-launch-1.0 -v mpasrc pipe-name=tracking num-buffers=60 ! fakesink sync=false

fakesink does not display or record the frames. On a target with a working graphical display and the required elements, display the raw stream with:

gst-launch-1.0 mpasrc pipe-name=tracking ! videoconvert ! autovideosink

mpasrc exposes the source format; it does not encode or decode video. For example, an H.264 pipe needs a parser and decoder before display:

gst-launch-1.0 mpasrc pipe-name=hires_small_encoded ! h264parse ! avdec_h264 ! videoconvert ! autovideosink

This display example requires the avdec_h264 decoder on the target. For H.265 input, use h265parse and avdec_h265. The _encoded pipe suffix does not identify the codec. Check the publisher’s configuration, or replace pipe-name=tracking with pipe-name=hires_small_encoded in the headless command to inspect the encoded caps without decoding.

Supported Formats

These are the plugin’s supported source formats. The downstream elements must also accept them. Other MPA formats, including the separate stereo and floating-point image formats, are not supported by this element.

Source Properties

Set these properties when constructing the pipeline, before playback:

For example, consume every third raw frame with at most two frames waiting in the source queue:

gst-launch-1.0 -v mpasrc pipe-name=tracking decimator=3 queue-size=2 ! fakesink sync=false

mpasrc reads format, dimensions, and framerate from the pipe’s info file when available, falling back to frame metadata. It timestamps output relative to the first camera timestamp and opens and closes its MPA client with the source lifecycle. Temporary publisher disconnects use Modal Pipe’s automatic reconnection. If a publisher changes format or resolution, downstream elements must accept the new caps; a fixed pipeline may need to be restarted or reconfigured.

The source favors low latency: it drops old queued frames and flushes a backed-up pipe. This also applies to H.264/H.265 input even though deliberate decimation is disabled, so it is not a lossless recording guarantee. Reduce source resolution, framerate, or bitrate if downstream processing cannot keep up. Queue size controls this source’s queue, not buffering elsewhere in the pipeline or player.

The read-only properties frames-received, frames-output, and frames-dropped expose counters on a running element. Applications can query them through GObject; gst-inspect-1.0 describes the properties but does not report counters from another running process. Counters reset when the source starts, and frames-dropped counts decimation and source queue drops, not all losses elsewhere in the stream.

Serve a Custom Pipeline over RTSP

gst-rtsp-launch accepts one quoted pipeline argument. Each stream must have an RTP payloader named pay0, pay1, and so on. The launcher shares the media pipeline between RTSP clients.

For an H.265 camera pipe:

gst-rtsp-launch \
    '( mpasrc pipe-name=hires_small_encoded ! h265parse ! rtph265pay name=pay0 pt=96 )'

For an H.264 camera pipe, use this pipeline instead:

gst-rtsp-launch \
    '( mpasrc pipe-name=hires_small_encoded ! h264parse ! rtph264pay name=pay0 pt=96 )'

Both examples pass encoded video through without re-encoding. Choose the parser and payloader to match the actual pipe codec. Raw camera input requires an encoder before the RTP payloader; voxl-streamer configures the platform encoder for you when that is all you need.

From a computer that can reach VOXL, open rtsp://<VOXL_IP>:8554/test in an RTSP player that supports the source codec, such as VLC or a suitably configured QGroundControl. The launcher prints a loopback URL (127.0.0.1) when listening on the default address; replace it with VOXL’s reachable IP when connecting remotely. Stop the foreground launcher with Ctrl+C.

Launcher Options

For example, serve an H.265 stream on a different port and mount:

gst-rtsp-launch --port 8555 --mount /camera \
    '( mpasrc pipe-name=hires_small_encoded ! h265parse ! rtph265pay name=pay0 pt=96 )'

This stream is available at rtsp://<VOXL_IP>:8555/camera. Use a free port for each server process. The normal voxl-streamer endpoint remains rtsp://<VOXL_IP>:8900/live; the launcher’s defaults can coexist with that service. gst-rtsp-launch does not need a --standalone flag and does not manage the service process. To advertise a custom endpoint through MAVLink, configure voxl-mavcam-manager for that URI.

Troubleshooting

For source diagnostics on a headless target:

GST_DEBUG=mpasrc:5 gst-launch-1.0 -v mpasrc pipe-name=tracking ! fakesink sync=false

For transport and player latency tuning, see Low Latency Video Streaming.