AI Detection Setup Guide

Table of contents

1. Overview
   1. How It Works
   2. Prerequisites
2. VTX Configuration
   1. Configuration Parameters
   2. Parameter Notes
3. VRX Configuration
   1. Configuration Parameters
4. Object Detection with voxl-tflite-server
5. Object Tracking with voxl-nano-tracker
6. Verifying Operation
   1. On the VTX (Drone)
   2. On the VRX (Ground Unit)
7. Troubleshooting
   1. No bounding boxes visible on VRX
   2. Poor detection performance
   3. Bandwidth concerns

Overview

The VTX/VRX system supports a real-time AI overlay on the video stream: an onboard service detects or tracks objects in the camera feed, voxl-vtx transmits the bounding box data alongside the video, and voxl-vrx renders the boxes (with optional labels) on the display or goggles.

How It Works

1. Drone/VTX: an onboard AI service publishes detection results (bounding boxes, labels, confidence scores) to an MPA pipe. Two sources are supported:
   • voxl-tflite-server — object detection: box every object a deep-learning model recognizes. Display-only. Publishes on tflite_data.
   • voxl-nano-tracker — object tracking: follow one target selected via RC or voxl-portal. Can also command vehicle yaw via VFC. Publishes on object_tracker_meta (the shipped config default on current SKUs).
2. Transmission: voxl-vtx subscribes to the pipe named by ai_detection_source in the active video profile and transmits the detection data to the VRX
3. VRX (Ground Unit): voxl-vrx receives the detection data and renders bounding boxes with optional labels on the video display

Prerequisites

• Drone/VTX/VRX on SDK 1.6.0+ (SDK 1.6.3+ for voxl-nano-tracker; the object_tracker_meta default requires SDK 1.6.6+)
• AI detection enabled in voxl-vtx and voxl-vrx configuration files

VTX Configuration

On the drone/VTX, edit /etc/modalai/voxl-vtx.conf to enable AI detection transmission.

The configuration is done per video profile, edit the following parameters in the profile you want the detection to be on:

{
  ...
  "profiles": [
    {
      "name": "Object Detection",
      "source": "hires_default_misp_encoded",  // Should be the encoded version of the camera pipe used for object detection in voxl-tflite-server
      ...
      "ai_detection_enable": true,               // Enable/Disable AI detection for this profile
      "ai_detection_source": "object_tracker_meta",  // Pipe to read detections from: object_tracker_meta (nano-tracker) or tflite_data (tflite-server)
      "ai_detection_qty": 3,                     // The maximum number of object detections to transmit to the VRX
      "ai_detection_threshold": 0.5              // The confidence threshold for an object to be considered for transmission
    }
  ]
  ...
}

Configuration Parameters

Parameter Notes

• ai_detection_qty: Higher values transmit more detected objects but increase bandwidth usage. Typically 2-5 objects is sufficient.
• ai_detection_threshold: Only detections with confidence greater than this value are transmitted. Range: 0.0 (all detections) to 1.0 (only very confident detections). A value of 0.75 means 75% confidence or higher.

VRX Configuration

On the VRX, edit /etc/modalai/voxl-vrx.conf to enable AI detection rendering:

{
  ...
  "ai_detection_enable": true,        // Enable/Disable AI detection
  "ai_detection_show_labels": false   // Enable/Disable object class labels
  ...
}

Configuration Parameters

Object Detection with voxl-tflite-server

voxl-tflite-server boxes every object its model recognizes. It is display-only — it never influences flight. Point ai_detection_source at tflite_data in the active VTX profile, then configure voxl-tflite-server on the drone/VTX:

1. Ensure voxl-tflite-server is installed on the drone/VTX.
2. Configure the service using voxl-configure-tflite.
   • Set the input pipe to match your camera source for the video feed (e.g., hires_default_misp_color if hires_default_misp_encoded is used as video source)
3. It is recommended to skip a few frames (set skip_n_frames in /etc/modalai/voxl-tflite-server.conf to 2, 3, or 4) to reduce the CPU usage, the voxl-tflite-server process can be quite CPU intensive.
4. Set the allow_multiple field to false. This is necessary so that the MPA pipe enumerates as tflite_data, which is what voxl-vtx expects.
5. Restart and verify the service is running: systemctl restart voxl-tflite-server then systemctl status voxl-tflite-server.
6. Check that detection data is being published: voxl-inspect-detections tflite_data.

Refer to the voxl-tflite-server documentation for complete setup instructions.

Object Tracking with voxl-nano-tracker

object_tracker_meta is the shipped config default on current SKUs. voxl-nano-tracker tracks a single target selected from an RC channel (default: channel 12) or voxl-portal; its configuration lives at /etc/modalai/voxl-nano-tracker.conf. Enable it with:

voxl-configure-nano-tracker factory_enable

voxl-nano-tracker is more than a display overlay. Paired with VFC position mode, AOI yaw tracking commands the drone’s yaw to keep the tracked target centered. Understand the AOI Yaw Tracking guide and its warnings before enabling tracking in flight, and be ready to take manual control.

Verifying Operation

On the VTX (Drone)

Check that detection data is being generated on your configured source pipe:

voxl-inspect-detections tflite_data            # tflite-server
voxl-inspect-detections object_tracker_meta    # nano-tracker

On the VRX (Ground Unit)

You should see bounding boxes rendered on the video display. If ai_detection_show_labels is enabled, you’ll also see class names.

Troubleshooting

No bounding boxes visible on VRX

1. Verify ai_detection_enable: true in both VTX and VRX config files
2. Check the source service is running on the drone/VTX: systemctl status voxl-tflite-server (or voxl-nano-tracker)
3. Verify detections are being published: voxl-inspect-detections tflite_data (or object_tracker_meta) on the drone/VTX
4. Confirm the VTX profile’s ai_detection_source matches the pipe your source service publishes
5. Ensure VTX/VRX link is working (video streaming properly)
6. Check ai_detection_threshold isn’t too high (try lowering the configured threshold)

Poor detection performance

1. Lower ai_detection_threshold to see more detections (may increase false positives)
2. Verify TFLite model is appropriate for your use case

Bandwidth concerns

1. Reduce ai_detection_qty to transmit fewer objects
2. Increase ai_detection_threshold to only send high-confidence detections