VOXL Streamer: Stream RTSP video from connected cameras
Table of contents
- Overview
- Prerequisites
- Installation
- Running voxl-streamer
- Viewing a Video Test Pattern
- Viewing output from a connected camera
- Viewing output from multiple connected cameras
Overview
voxl-streamer enables streaming live video from VOXL using the following interfaces:
- MIPI
- USB / UVC
- USB / libgphoto2
- HDMI
There are a variety of ways to stream video from a camera connected to the VOXL and the method to use can vary depending on the application. The general case is to take video from a camera connected via MIPI, USB / UVC, or HDMI and stream it using the Real Time Streaming Protocol (RTSP). The streamed video is typically viewed using QGroundControl or VLC. The voxl-streamer application is easy to use and supports these standard use cases. It is based on the GStreamer open source multimedia framework and incorporates OpenMAX (OMX) for hardware acceleration of video encoding to h.264 (AVC). The source code is located here.
Prerequisites
voxl-streamer requires system image 3.0.0 or greater. It also requires an active IP network connection. The IP connection can be over WiFi, Ethernet, or a wide variety of other wireless network connectivity options enabled by Voxl peripherals. The computer hosting the viewer application must be able to connect to the Voxl at an IP address. Record the IP address of the Voxl as it will be needed to view the streams.
Installation
voxl-streamer is installed via ipk package.
yocto:/home/root/ipk# opkg update
yocto:/home/root/ipk# opkg install voxl-streamer
Running voxl-streamer
voxl-streamer is a command line application and can be started on the command line:
yocto:/home/root# voxl-streamer
Using configuration file /etc/modalai/voxl-streamer.conf
Stream available at rtsp://127.0.0.1:8900/live
Use the -h option to view online help:
yocto:/home/root# voxl-streamer -h
Usage: voxl-streamer <options>
Options:
-d Show extra debug messages.
-v Show extra frame level debug messages.
-u <uvc device> UVC device to use (to override what is in the configuration file).
-c <name> Configuration name (to override what is in the configuration file).
-f <filename> Configuration file name (default is /etc/modalai/voxl-streamer.conf).
-h Show help.
Viewing a Video Test Pattern
The simplest way to see voxl-streamer in action is to use the video test source. This will cause a video test pattern to be available on the stream. It does not require any cameras to be connected to the Voxl:
yocto:/home/root# voxl-streamer -c video-test
Using configuration video-test
Using configuration file /etc/modalai/voxl-streamer.conf
Stream available at rtsp://127.0.0.1:8900/live
When a connection is made to the stream the following message will appear:
A new client has connected to the RTSP server
Viewing with QGroundControl
- Press the purple QGC logo in the top left corner in order to Access the
Application Settingsmenu. - Under the
Generaltab, scroll down until you find theVideosection. - Under the
Video Sourcedropdown, chooseRTSP Video Stream - In the
RTSP URLfield. enter the RTSP URL, default:rtsp://<voxl ip address>:8900/live - The aspect ratio can be chosen to match the aspect ratio of the source video. For example, If the source video is VGA (640 x 480), the aspect ration will be 640 / 480 or 1.3333
- You will now be able to view the video stream under QGroundControl’s
Flyview.
Viewing with VLC
- On the host system, run the VLC software
- Go to
Media>Open Network Stream - Enter RTSP URL, default:
rtsp://<voxl ip address>:8900/live - Click Play to view the video stream
Viewing output from a connected camera
The test pattern is useful to validate that the viewer is configured properly and that all of the prerequisites have been satisfied. The next step is to stream the video from a connected camera. The application relies on a configuration file containing one or more named configuration records and a configuration name to specify which of the configuration records to use. The name of the configuration record to use can be specified in the configuration file or on the command line with the -c option. The default configuration file is located at /etc/modalai/voxl-streamer.conf. A different configuration file can be specified using the -f option. Controlling how to stream a connected camera is done via the configuration file. The following sections explain the configuration file in more detail.
Configuration file format
The configuration file utilizes JSON to format the records. Here is an example from the default configuration file that is installed with the package:
{
"configuration": "hires",
"tracking": {
"input": {
"interface": "mpa",
"mpa-camera": "tracking" },
"output": {
"stream": {
"width": 640,
"height": 480,
"decimator": 2,
"bitrate": 1000000 } }
},
The configuration file can be edited with standard text editors.
Video source
voxl-streamer receives video frames via either Linux named pipes as part of Modal Pipe Architecture (MPA), UVC, or internally generated test frames. There are a variety of Voxl applications with built-in support for MPA.
Streaming MIPI cameras
voxl-camera-server is an MPA enabled application that can be used to provide the video streams from cameras connected via MIPI to voxl-streamer. In this case voxl-camera-server must be running and configured properly to provide the video frames. Assuming that voxl-camera-server is up and running this section explains how to setup the configuration file to access the video streams. The default configuration for voxl-camera-server is to have a high resolution (hires) camera on connector 0, a pair of stereo cameras on connector 1, and a tracking camera on connector 2. voxl-streamer can stream the images from any of these sources.
High resolution (hires) camera example
The default configuration file has a configuration record named “hires-mpa”. Using this configuration record will allow voxl-streamer to stream the video frames from the hires camera. This record is chosen using the “configuration” tag in the configuration file or can be specified on the command line with the -c option.
"hires": {
"input": {
"interface": "mpa",
"mpa-camera": "hires/preview" },
"output": {
"stream": {
"rotation": 90,
"width": 480,
"height": 640,
"decimator": 2,
"bitrate": 1000000 } }
},
Record name
The very first tag is the name of the configuration record. Each record must have a unique name. Each record has a subsection for input parameters and another for output parameters.
Input parameters
"hires-mpa": {
"input": {
"interface": "mpa",
"mpa-camera": "hires/preview" },
For input, the interface is chosen as “mpa” which means that the video frames will come via the Modal Pipe Architecture (MPA). Other choices include “uvc” for a UVC camera, or “test” for a test pattern.
The “mpa-camera” tag is only valid for interface type “mpa”. This allows the voxl-streamer application to access the correct MPA interface to receive the desired frames. The valid choices are “hires/preview”, “stereo”, and “tracking”. Other input parameters such as resolution and format are passed to voxl-streamer from the source in the form of MPA meta-data.
"uvc-video": {
"input": {
"interface": "uvc",
"device": "/dev/video0" },
For UVC cameras, the interface is “uvc”. The “device” parameter is the device name for the camera. All other input parameters are negotiated by gstreamer as part of the UVC protocol. The device name can be provided on the command line with option “-u”.
Note: It is sometimes difficult to know which /dev/videoX corresponds to the desired UVC camera. The Qualcomm camera drivers will occupy some of these device names but they cannot be accessed as UVC cameras. For this reason a convenience script called start-uvc-stream.sh is provided to search for the correct device name and then launch voxl-streamer with that name.
"video-test": {
"input": {
"interface": "test",
"frame": {
"width": 1280,
"height": 720,
"format": "uyvy" } },
For the test interface, frames are generated internal to the application using the Gstreamer videotestsrc element. This requires a full specification of the desired frame parameters for both resolution and format. Supported formats include “uyvy”, “nv12”, “nv21”, and “gray8”.
Output parameters
For output there is a single “stream” subsection. This contains the desired resolution of the output stream specified as “width”, and “height” in pixels. Notice that this can be different than the input resolution. The voxl-streamer application will rescale the image appropriately.
The “bitrate” parameter specifies the bit rate of the stream in bits per second (bps). The video encoder will be configured to meet this streaming bitrate.
To control the frame rate there are 2 options based on the interface type. For the test frames and the UVC frames the “rate” parameter specifies the desired output frame rate in frames per second (fps). For MPA there is an optional “decimator” parameter. This determines how many of the input frames will be used to generate the output stream. For example, a value of 1, the default, will choose every input frame. A value of 2 will choose one out of every two frames, a value of 3 will choose one out of every three frames, etc.
Finally, there is an optional “rotation” tag that can choose a rotation of 0, 90, 180, or 270 degrees for the output image relative to the input image. Take care with the specification of the output resolution when there is a rotation. For example, an output resolution of 640 x 480 (VGA) must be specified as 480 x 640 if a rotation of 90 or 270 is used.
Image overlay
It is possible to overlay an image onto the video stream (e.g a company logo). This is an optional feature and can be configured with an additional record in the configuration. There is an example in the default configuration file called “hires-mpa-logo”. The overlay record specifies, at a minimum, the location of the desired image overlay file. By default it will be placed in the top left corner of the video. The optional “offset_x” and “offset_y” parameters can move the image around on the video. Positive values will move the image relative to the top left corner. Negative values will move the image relative to the bottom right corner.
Default configuration file
The default configuration file is located at /etc/modalai/voxl-streamer.conf. It is shown below:
{
"configuration": "tracking",
"hires": {
"input": {
"interface": "mpa",
"mpa-camera": "hires_preview" },
"output": {
"stream": {
"rotation": 0,
"width": 640,
"height": 480,
"decimator": 2,
"bitrate": 1000000 } }
},
"hires-logo": {
"input": {
"interface": "mpa",
"mpa-camera": "hires_preview" },
"output": {
"stream": {
"rotation": 0,
"width": 640,
"height": 480,
"decimator": 2,
"bitrate": 1000000 } },
"overlay": {
"location": "/etc/modalai/modalai.png",
"offset_x": -1,
"offset_y": -1 }
},
"stereo": {
"input": {
"interface": "mpa",
"mpa-camera": "stereo" },
"output": {
"stream": {
"width": 640,
"height": 960,
"decimator": 2,
"bitrate": 1000000 } }
},
"tracking": {
"input": {
"interface": "mpa",
"mpa-camera": "tracking" },
"output": {
"stream": {
"width": 640,
"height": 480,
"decimator": 2,
"bitrate": 1000000 } }
},
"dfs-disparity": {
"input": {
"interface": "mpa",
"mpa-camera": "dfs_disparity" },
"output": {
"stream": {
"width": 640,
"height": 480,
"decimator": 2,
"bitrate": 1000000 } }
},
"qvio-overlay": {
"input": {
"interface": "mpa",
"mpa-camera": "qvio_overlay" },
"output": {
"stream": {
"width": 640,
"height": 544,
"decimator": 2,
"bitrate": 1000000 } }
},
"tflite-overlay": {
"input": {
"interface": "mpa",
"mpa-camera": "tflite" },
"output": {
"stream": {
"width": 640,
"height": 480,
"decimator": 2,
"bitrate": 1000000 } }
},
"hdmi-mpa": {
"input": {
"interface": "mpa",
"mpa-camera": "v4l2" },
"output": {
"stream": {
"width": 640,
"height": 360,
"decimator": 2,
"bitrate": 1000000 } }
},
"gphoto2-mpa": {
"input": {
"interface": "mpa",
"mpa-camera": "gphoto2" },
"output": {
"stream": {
"width": 640,
"height": 360,
"decimator": 2,
"bitrate": 1000000 } }
},
"video-test": {
"input": {
"interface": "test",
"frame": {
"width": 1280,
"height": 720,
"format": "uyvy" } },
"output": {
"stream": {
"rotation": 0,
"width": 640,
"height": 360,
"rate": 15,
"bitrate": 1000000 } }
},
"uvc-video": {
"input": {
"interface": "uvc",
"device": "/dev/video0" },
"output": {
"stream": {
"rotation": 0,
"width": 640,
"height": 360,
"rate": 15,
"bitrate": 1000000 } }
}
}
Viewing output from multiple connected cameras
voxl-streamer can be running as multiple processes in parallel to support streaming of multiple cameras. The OpenMAX hardware acceleration for h264 encoding can handle multiple simultaneous streams. The default port number for the RTSP URI is 8900 but that needs to be changed for other instances since only one process can use a port at a time. So, for example, this will start 2 instances:
# voxl-streamer
# voxl-streamer -c tracking-mpa -p 8901
The first command starts the default camera on the default port. The second command starts the tracking camera on port 8901.
Note: The mavlink-camera-manager application can be used to advertise RTSP streams and camera names to QGroundControl so that streams can be individually chosen by camera name. The mavlink-camera-manager application is documented here