VOXL Streamer: Stream RTSP video from connected cameras

Table of contents

1. Overview
2. Prerequisites
3. Configuration
   1. Parameters
   2. Streaming in 720p and 4K
      1. Optimized Encoding Data Path
   3. Viewing with QGroundControl
   4. Viewing with VLC
4. Viewing output from a connected camera
5. Streaming UVC cameras from voxl-uvc-server
6. Viewing output from multiple connected cameras
   1. Output parameters

Overview

voxl-streamer is a background systemd service that enables streaming live video from MPA pipes as an RTSP video stream over IP.

There are a variety of ways to stream video from a camera connected to the VOXL and the method to use can vary depending on the application. The general case is to take video from a camera connected via MIPI, USB / UVC, or HDMI and stream it using the Real Time Streaming Protocol (RTSP). The streamed video is typically viewed using QGroundControl or VLC. The voxl-streamer application is easy to use and supports these standard use cases. It is based on the GStreamer open source multimedia framework and incorporates OpenMAX (OMX) for hardware acceleration of video encoding to h.264 (AVC). The source code is located here.

Prerequisites

voxl-streamer requires an active IP network connection. The IP connection can be over WiFi, Ethernet, or a wide variety of other wireless network connectivity options enabled by Voxl peripherals. The computer hosting the viewer application must be able to connect to the Voxl at an IP address. Record the IP address of the Voxl as it will be needed to view the streams.

Configuration

The voxl-streamer service is configured with the file ‘/etc/modalai/voxl-streamer.conf’ Which looks like this in its default configuration:

{
	"input-pipe":	"hires",
	"bitrate":	1000000,
	"decimator":	2,
	"port":	8900,
	"rotation":	0
}

Parameters

1. input-pipe : The pipe that publishes frames or encoded data into voxl-streamer
   
2. bitrate: The bitrate of the stream to publish in bits per second
   
3. decimator: The decimator enables sub-sampling of frames for bandwidth reduction. Every X (X = decimator) frame is processed by the voxl-streamer, a value of 1 publishes every frame
   
4. port: The port the stream will be posted to at the url of the rtsp stream
   
5. rotation: The rotation of the frame
   

This file can be edited manually, although we recommend using the voxl-configure-streamer utility which takes in optional arguments to set each field and leaves the others default. The available arguments are as follows:

voxl2:/$ voxl-configure-streamer -h

Command line arguments are as follows:

-b --bitrate    <#>     | Specify bitrate     (Default 1000000)
-d --decimator  <#>     | Specify decimator   (Default 2)
-h --help               | Print this help message
-i --input-pipe <name>  | Specify input pipe  (Default hires)
-p --port       <#>     | Specify Port number (Default 8900)
-r --rotation   <#>     | Specify rotation    (Default 0)

For example, to configure and enable streamer for the hires color camera found on the Sentinal, M500, and Starling reference drones, run the following:

voxl2:/$ voxl-configure-streamer -i hires
Using config:
	Name:      hires
	Bitrate:   1000000
	Decimator: 2
	Port:      8900
	Rotation:  0
Writing new configuration to /etc/modalai/voxl-streamer.conf
voxl2:/$ systemctl enable voxl-streamer
Created symlink /etc/systemd/system/multi-user.target.wants/voxl-streamer.service → /etc/systemd/system/voxl-streamer.service.
voxl2:/$ systemctl start voxl-streamer

Note the output resolution of the RTSP stream will be the same as the resolution of the MPA pipe image stream. voxl-streamer does not do any resizing. If you wish to stream a different resolution, you must configure voxl-camera-server to open the MIPI camera with a different preview resolution.

Validate that the MPA pipe image stream is functioning and check its resolution as follows:

voxl2:/$ voxl-inspect-cam hires

| Pipe Name |size(bytes)| height | width  |exposure(ms)| gain | frame id |latency(ms)|Framerate(hz)| format
|     hires |   460800  |    480 |    640 |      11.50 |  321 |  497778  |      43.5 |     30.0    | NV12

Here we can see that voxl-camera-server is outputing images from the hires camera at VGA resolution and 30hz which is the default.

Streaming in 720p and 4K

Note: This documentation is reflecting sdk 0.9

Hardware setup (images):

Top view of VOXL2:

Bottom view of VOXL2:

Software setup:

In order to ensure voxl-streamer is configured to stream at 720p, please ensure that voxl-camera-server is configured to take frames at 720p. This can be done by editing the voxl-camera-server configuration file (/etc/modalai/voxl-camera-server.conf) to reflect the preview_width and preview_height of 1280 and 720 pixels, respectfully. Once this is done, the user can then restart voxl-camera-server (sudo systemctl restart voxl-camera-server) and focus purely on voxl-streamer.

In order to get voxl-streamer to run high quality video at 720p, the bitrate needs to be increased from the configuration file used for voxl-streamer. So the same steps as above (in the “Configuration” section) need to be followed for initial setup plus a few added steps:

1. Run voxl-configure-streamer -i hires
   
2. Using whichever text editor you use (vim or nano for most), proceed to edit the bitrate value from 1Mbps to 3Mbps within the voxl-streamer config file (/etc/modalai/voxl-streamer.conf)
   
3. Start and enable voxl-streamer by running: sudo systemctl start voxl-streamer and sudo systemctl enable voxl-streamer
   

You can now open the stream up in VLC with the steps found below in “Viewing with VLC”.

The steps for 4K streaming are very similar to the above information - the main difference being that you need to now change the preview_width and preview_height to reflect 4K dimensions (aka: 3840x2160 for most cameras) in the /etc/modalai/voxl-camera-server.conf file. Once again, restart the voxl-camera-server, then run through step 2 through 3 listed earlier in this section (unless you havent already ran voxl-configure-streamer, then steps 1 - 3) and change the bitrate to something around the likes 10Mbps and restart the service. Proceed to open and watch the stream in VLC or QGC!

Watch the tutorial for step-by-step instructions

Optimized Encoding Data Path

voxl-camera-server v1.4.5+ enables an optimized, hardware encoded datapath for video streaming referred to as ‘Stream output’. See voxl-camera-server for how to stream the encoded output (h264) from voxl-camera-server. The pipe name for this optimized path is ‘hires_stream’ and can be inspected using voxl-inspect-cam

Run voxl-configure-streamer -i hires_stream to utilize with voxl-camera-server v1.4.5+. Below is a video of how to setup 720P encoded streaming via voxl-camera-server and voxl-streamer:

Viewing with QGroundControl

• Press the purple QGC logo in the top left corner in order to Access the Application Settings menu.
• Under the General tab, scroll down until you find the Video section.
• Under the Video Source dropdown, choose RTSP Video Stream
• In the RTSP URL field. enter the RTSP URL, default: rtsp://<voxl ip address>:8900/live
• The aspect ratio can be chosen to match the aspect ratio of the source video. For example, If the source video is VGA (640 x 480), the aspect ration will be 640 / 480 or 1.3333
• Close and re-open QGC
• You will now be able to view the video stream under QGroundControl’s Fly view.

Viewing with VLC

• On the host system, run the VLC software
• Go to Media > Open Network Stream
• Enter RTSP URL, default: rtsp://<voxl ip address>:8900/live
• Click Play to view the video stream

Viewing output from a connected camera

The test pattern is useful to validate that the viewer is configured properly and that all of the prerequisites have been satisfied. The next step is to stream the video from a connected camera. The application relies on a configuration file containing one or more named configuration records and a configuration name to specify which of the configuration records to use. The name of the configuration record to use can be specified in the configuration file or on the command line with the -c option. The default configuration file is located at /etc/modalai/voxl-streamer.conf. A different configuration file can be specified using the -f option.

Streaming UVC cameras from voxl-uvc-server

***Important note: Direct UVC support in voxl-streamer is being deprecated in favor of voxl-uvc-server. Use the uvc- configuration records with voxl-uvc-server***

There are a few configuration records in the default configuration file to deal with MPA input from UVC cameras using voxl-uvc-server. The uvc-generic configuration record will suffice for many webcam style UVC cameras.

• voxl-streamer -c uvc-generic

However, certain UVC cameras have different frame characteristics and can benefit from tailored configurations. For example:

• Flir Boson 640: voxl-streamer -c uvc-flir-boson
• PureThermal Mini Pro with Flir Lepton: voxl-streamer -c uvc-flir-lepton

Viewing output from multiple connected cameras

voxl-streamer can be running as multiple processes in parallel to support streaming of multiple cameras. The OpenMAX hardware acceleration for h264 encoding can handle multiple simultaneous streams. The default port number for the RTSP URI is 8900 but that needs to be changed for other instances since only one process can use a port at a time. So, for example, this will start 2 instances:

# voxl-streamer
# voxl-streamer -c tracking-mpa -p 8901

The first command starts the default camera on the default port. The second command starts the tracking camera on port 8901.

Note: The mavlink-camera-manager application can be used to advertise RTSP streams and camera names to QGroundControl so that streams can be individually chosen by camera name. The mavlink-camera-manager application is documented here

Output parameters

For output there is a single “stream” subsection. This contains the desired resolution of the output stream specified as “width”, and “height” in pixels. Notice that this can be different than the input resolution. The voxl-streamer application will rescale the image appropriately.

The “bitrate” parameter specifies the bit rate of the stream in bits per second (bps). The video encoder will be configured to meet this streaming bitrate.

To control the frame rate there are 2 options based on the interface type. For the test frames and the UVC frames the “rate” parameter specifies the desired output frame rate in frames per second (fps). For MPA there is an optional “decimator” parameter. This determines how many of the input frames will be used to generate the output stream. For example, a value of 1, the default, will choose every input frame. A value of 2 will choose one out of every two frames, a value of 3 will choose one out of every three frames, etc.

Finally, there is an optional “rotation” tag that can choose a rotation of 0, 90, 180, or 270 degrees for the output image relative to the input image. Take care with the specification of the output resolution when there is a rotation. For example, an output resolution of 640 x 480 (VGA) must be specified as 480 x 640 if a rotation of 90 or 270 is used.