AI Detection Setup Guide

Table of contents

1. Overview
   1. How It Works
   2. Prerequisites
2. VTX Configuration
   1. Configuration Parameters
   2. Parameter Notes
3. VRX Configuration
   1. Configuration Parameters
4. voxl-tflite-server Setup
5. Verifying Operation
   1. On the VTX (Drone)
   2. On the VRX (Ground Unit)
6. Troubleshooting
   1. No bounding boxes visible on VRX
   2. Poor detection performance
   3. Bandwidth concerns

Overview

The VTX/VRX system supports real-time AI object detection overlay on the video stream. This feature uses the voxl-tflite-server and voxl-vtx services running on the drone/VTX to detect objects in the camera feed and transmit bounding box data to the voxl-vrx service running on the VRX for rendering on the display or goggles.

How It Works

The AI detection pipeline works as follows:

1. Drone/VTX: The voxl-tflite-server application processes camera frames and generates object detection results (bounding boxes, labels, confidence scores) which are then published to MPA
2. Transmission: The voxl-vtx process then subscribes to the tflite_data pipe created by voxl-tflite-server and transmits the object detection data to the VRX
3. VRX (Ground Unit): The voxl-vrx process receives the object detection data and renders bounding boxes with optional labels on the video display

Prerequisites

• Drone/VTX/VRX on SDK 1.6.0+
• Compatible TensorFlow Lite model deployed on the drone/VTX
• AI detection enabled in voxl-vtx and voxl-vrx configuration files

VTX Configuration

On the drone/VTX, edit /etc/modalai/voxl-vtx.conf to enable AI detection transmission.

The configuration is done per video profile, edit the following parameters in the profile you want the detection to be on:

{
  ...
  "profiles": [
    {
      "name": "Object Detection",
      "source": "hires_default_misp_encoded",  // Should be the encoded version of the camera pipe used for object detection in voxl-tflite-server
      ...
      "ai_detection_enable": true,      // Enable/Disable AI detection for this profile
      "ai_detection_qty": 5,            // The maximum number of object detections to transmit to the VRX
      "ai_detection_threshold": 0.75    // The confidence threshold for an object to be considered for transmission
    }
  ]
  ...
}

Configuration Parameters

Parameter Notes

• ai_detection_qty: Higher values transmit more detected objects but increase bandwidth usage. Typically 2-5 objects is sufficient.
• ai_detection_threshold: Only detections with confidence greater than this value are transmitted. Range: 0.0 (all detections) to 1.0 (only very confident detections). A value of 0.75 means 75% confidence or higher.

VRX Configuration

On the VRX, edit /etc/modalai/voxl-vrx.conf to enable AI detection rendering:

{
  ...
  "ai_detection_enable": true,        // Enable/Disable AI detection
  "ai_detection_show_labels": false   // Enable/Disable object class labels
  ...
}

Configuration Parameters

voxl-tflite-server Setup

The voxl-tflite-server process must be properly configured on the drone/VTX:

1. Ensure voxl-tflite-server is installed on the drone/VTX.
2. Configure the service using voxl-configure-tflite-server-single.
   • Set the input pipe to match your camera source for the video feed (e.g., hires_default_misp_color if hires_default_misp_encoded is used as video source)
3. It is recommended to skip a few frames (set skip_n_frames in /etc/modalai/voxl-tflite-server.conf to 2, 3, or 4) to reduce the CPU usage, the voxl-tflite-server process can be quite CPU intensive.
4. Set the allow_multiple field to false. This is necessary so that the MPA pipe enumerates as tflite_data, which is what voxl-vtx expects.
5. Restart and verify the service is running: systemctl restart voxl-tflite-server then systemctl status voxl-tflite-server.
6. Check that detection data is being published: voxl-inspect-detections tflite_data.

Refer to the voxl-tflite-server documentation for complete setup instructions.

Verifying Operation

On the VTX (Drone)

Check that tflite data is being generated:

voxl-inspect-detections tflite_data

On the VRX (Ground Unit)

You should see bounding boxes rendered on the video display. If ai_detection_show_labels is enabled, you’ll also see class names.

Troubleshooting

No bounding boxes visible on VRX

1. Verify ai_detection_enable: true in both VTX and VRX config files
2. Check voxl-tflite-server is running on the drone/VTX: systemctl status voxl-tflite-server
3. Verify objects are being detected: voxl-inspect-detections tflite_data on the drone/VTX
4. Ensure VTX/VRX link is working (video streaming properly)
5. Check ai_detection_threshold isn’t too high (try lowering the configured threshold)

Poor detection performance

1. Lower ai_detection_threshold to see more detections (may increase false positives)
2. Verify TFLite model is appropriate for your use case

Bandwidth concerns

1. Reduce ai_detection_qty to transmit fewer objects
2. Increase ai_detection_threshold to only send high-confidence detections