VOXL Vision Hub

Table of contents

1. Overview
2. How It Works
3. Pipes
4. Configuration
   1. /etc/modalai/voxl-vision-hub.conf
   2. /etc/modalai/vfc.conf
   3. voxl-configure-vision-hub
5. Offboard Demo Modes
   1. Other Requirements to Run
   2. Running
6. Offboard Flight Modes
   1. Flight Modes
   2. Backtrack
   3. Waypoint Missions (wps)
   4. PX4 Configuration
7. Debugging

Overview

voxl-vision-hub is the main hub managing communication between VOXL MPA services and autopilots such as PX4 and ArduPilot. It forwards VIO to the autopilot as MAVLink odometry, fuses obstacle sensors for Vision Obstacle Avoidance (VOA), and provides onboard autonomous flight behaviors in PX4 offboard mode — from simple demos (figure 8, trajectory following) up to VFC, the VOXL Flight Controller used on FPV platforms.

voxl-vision-hub also underpins other tools such as voxl-calibrate-px4-horizon, MAVROS, and MAVSDK, so you will often find it must be running in the background for those to work.

If you want to develop applications that involve flight control without using ROS, voxl-vision-hub is the recommended starting point and will save you thousands of lines of code.

How It Works

In the voxl-vision-hub GitLab, you will find all the source code for the project. An overview of what each module does:

• autopilot_monitor.c: subscribes to the mavlink_onboard pipe published by voxl-mavlink-server and tracks autopilot state (attitude, flight mode, armed state)
• config_file.c: reads/writes the config file /etc/modalai/voxl-vision-hub.conf
• fixed_pose_input.c: accepts a secondary (non-VIO) pose input on the vvhub_fixed_pose_input pipe to correct VIO drift; this should generally be a slow-but-accurate source (see examples/fixed_frame_pipe_example.c in the repo)
• geometry.c: helper functions for transforming coordinate frames, used for VIO, VOA, and relocalization
• horizon_cal.c / horizon_cal_file.c: in-flight IMU horizon calibration (find the orientation at which the drone hovers perfectly still) and its result file. See horizon calibration — voxl-vision-hub must be running and voxl-calibrate-px4-horizon drives the process.
• imu_manager.c: subscribes to the imu_apps pipe and keeps a short ring buffer of IMU samples for the other modules
• main.c: starts every module, then starts the configured offboard mode
• mavlink_for_ros.c: opens the localhost UDP port for MAVROS and MAVSDK to reach voxl-mavlink-server. Only active when en_localhost_mavlink_udp is enabled in the config (disabled by default).
• mavlink_io.c: reads/writes MAVLink to the autopilot and GCS through voxl-mavlink-server’s pipes
• obs_pc_filter.c: obstacle point-cloud filter; ring-buffers ~2 seconds of point cloud data, temporally filters it, and outputs a clean cloud at a constant rate
• offboard_figure_eight.c: demo mode — flies a figure 8 around the point where offboard mode was engaged
• offboard_follow_tag.c: demo mode — follows an AprilTag (ModalAI R&D only, see warning below)
• offboard_trajectory.c: follows polynomial trajectories received on the plan_msgs pipe (e.g. from voxl-mapper); expects the input path to start at the drone’s current position
• offboard_vfc.c: VFC — the VOXL Flight Controller, a full onboard attitude/altitude/position controller with RC-selectable submodes (see VFC below)
• offboard_backtrack.c: records the last N seconds of flown path and, on RC-link loss or a switch trigger, flies it backwards to return toward where the link was healthy
• offboard_wps.c: local-frame waypoint mission mode, reading waypoints from a CSV (supports photo capture, land, and return actions)
• offboard_mode.c: dispatches whichever offboard mode the config file selects
• pose_filter.c / tag_manager.c: low-pass filter over AprilTag localizations from voxl-tag-detector’s tag_detections pipe, used by the relocalization feature
• state_manager.c: aggregates non-VIO state sources — the state pipe from voxl-state-estimator (optic-flow state used by VFC flow submodes) and, in HITL mode, the simulator’s hitl_vio pipe
• trajectory_monitor.cpp: for trajectory mode; predicts collisions from current pose, VOA data, and the trajectory
• transform_ringbuf.c: ring buffer of timestamped transforms used throughout
• vio_manager.c: transforms VIO output from body frame to a gravity-aligned local frame on the floor, health-checks VIO, and manages the fallback to the secondary VIO pipe
• voa_manager.c: subscribes to the configured obstacle-sensor pipes (voa_inputs), converts them into point clouds, and fuses them for collision prevention
• voxl_vfc_config.c: loader for the separate VFC config file /etc/modalai/vfc.conf

Pipes

Pipes published by voxl-vision-hub:

It also creates the vvhub_fixed_pose_input sink pipe, where an external localizer can write a pose_4dof_t stream to correct VIO drift.

Pipes subscribed to include the primary and secondary VIO pipes, imu_apps, state (from voxl-state-estimator), tag_detections, plan_msgs, object_tracker, the MAVLink pipes from voxl-mavlink-server (mavlink_onboard, mavlink_to_gcs, mavlink_from_gcs), and every enabled voa_inputs entry.

Configuration

/etc/modalai/voxl-vision-hub.conf

The main config file is created with defaults on first start (or with voxl-vision-hub -c). A legacy /etc/modalai/voxl-vision-px4.conf from old SDKs is migrated automatically. Obsolete keys (all the QGC IP/port settings, ADSB, and PX4 UART settings) are removed from the file on load — GCS connectivity is now entirely voxl-mavlink-server’s job.

Ensure every parameter is reasonable before attempting flight.

MAVROS / MAVSDK:

VIO:

Note that your platform’s SKU configuration (voxl-configure-mpa) decides which VIO service actually runs — voxl-open-vins-server on current SKUs, voxl-qvio-server on older platforms.

Offboard behavior:

Waypoint (wps) mode:

Object-tracking trigger (used by VFC’s yaw-follow, see AOI yaw tracking):

Fixed-frame (AprilTag relocalization):

VOA (collision prevention — see VOA for the full guide):

voa_inputs is an array of up to 6 sensor sources, each with enabled, type (point_cloud, tof, or rangefinder), input_pipe, frame, max_depth, min_depth, cell_size, threshold, x_fov_deg, y_fov_deg, and conf_cutoff. If the array is empty, defaults are created for dfs_point_cloud, stereo_front_pc, stereo_rear_pc, tof, and rangefinders — disable or delete the entries that don’t exist on your vehicle.

/etc/modalai/vfc.conf

Since voxl-vision-hub 1.9.2 the VFC controller has its own config file, /etc/modalai/vfc.conf, also auto-created with defaults. It holds:

• RC mapping — vfc_thrust_ch 3, vfc_roll_ch 1, vfc_pitch_ch 2, vfc_yaw_ch 4, vfc_submode_ch 6, plus stick deadbands
• Submode PWM windows — vfc_<submode>_mode_rc_min/_max pairs mapping positions of the submode switch to VFC submodes; a 0/0 window disables that submode (see Flight Modes below)
• Backtrack — backtrack_seconds 60, backtrack_rc_chan 10, backtrack_rc_thresh 1500
• Limits — vfc_max_thrust 0.8, vfc_tilt_max 0.436 rad, vfc_yaw_rate_max 3.0, vfc_vxy_max 3.0, vfc_vz_max 1.0, vfc_thrust_hover 0.25
• Control gains — separate PID gain sets for flow submodes (vfc_kp_z, vfc_kp_xy, …) and VIO submodes (vfc_kp_z_vio, vfc_kp_xy_vio, …), plus filters and transition times
• VIO health gates — vfc_q_min 15 (minimum VIO quality), vfc_points_min
• Misc — vfc_rate 100 Hz, vfc_en_submode_announcement (send submode changes to the GCS as STATUSTEXT), vfc_disable_fallback, vfc_vio_pipe (defaults to ov), vfc_traj_csv path

VFC tuning values placed in /etc/modalai/voxl-vision-hub.conf are ignored. Since voxl-vision-hub 1.9.2 the VFC configuration lives in /etc/modalai/vfc.conf.

voxl-configure-vision-hub

The service’s configuration helper. voxl-configure-vision-hub wizard runs interactively; the notable non-interactive presets are:

voxl-configure-vision-hub enable            # enable + start the service
voxl-configure-vision-hub disable           # stop + disable
voxl-configure-vision-hub factory_enable    # reset config to factory default
voxl-configure-vision-hub factory_enable_trajectory
voxl-configure-vision-hub factory_enable_backtrack
voxl-configure-vision-hub factory_enable_vfc_fpv       # VFC preset for FPV platforms
voxl-configure-vision-hub factory_enable_vfc_starling  # VFC preset for Starling
voxl-configure-vision-hub configure_hitl    # hardware-in-the-loop setup
voxl-configure-vision-hub update_vfc_settings   # pull measured hover thrust into vfc.conf

Per-SKU presets (e.g. d0013) also exist and are what voxl-configure-mpa applies at SKU configuration time. reset_config_file_to_default deletes both voxl-vision-hub.conf and vfc.conf and regenerates defaults.

Offboard Demo Modes

voxl-vision-hub includes demonstration offboard modes that run out of the box: figure_eight and trajectory (plus the R&D-only follow_tag).

follow_tag mode is for ModalAI R&D only and is not recommended — the vehicle will chase a tag with no obstacle awareness.

Other Requirements to Run

voxl-vision-hub subscribes to pipes published by other services, which must be running:

• voxl-mavlink-server
• voxl-imu-server
• a VIO service — voxl-open-vins-server (current SKUs) or voxl-qvio-server (legacy platforms)
• voxl-camera-server
• voxl-tag-detector (only for follow_tag)

For trajectory mode, another service such as voxl-mapper must be publishing trajectories to the plan_msgs pipe.

Running

Power on your VOXL-equipped drone, ensuring that all the required services are running. In manual mode, using a radio transmitter, bring your drone to a hover in a safe and open space. When ready, switch the drone into offboard mode with your radio transmitter, and the configured autonomous behavior will begin.

The legacy offboard demo modes (figure_eight, follow_tag, trajectory) do not support autonomous takeoff and landing. The VFC wps mission mode does support takeoff, RTL, and land (voxl-vision-hub 1.8.23 / 1.9.1 and newer).

Offboard Flight Modes

voxl-vision-hub can fly the UAS in OFFBOARD mode using VFC (the VOXL Flight Controller) — a complete onboard attitude/altitude/position controller selected with offboard_mode: "vfc". VFC is the flight stack used on ModalAI FPV platforms such as Stinger.

Flight Modes

The following vision-enabled flight modes are supported through VFC in OFFBOARD mode:

• Manual (thrust + attitude)
• Altitude Hold
• Flow (optic-flow position hold, works with thermal flow)
• Hybrid Flow
• Position (Visual Inertial Odometry)
• Trajectory (replays the CSV at vfc_traj_csv)
• Backtrack

VFC automatically falls back to a safer submode when VIO, flow, or altitude health checks fail (disable with vfc_disable_fallback), announces submode changes to the GCS as STATUSTEXT messages, and auto-tunes hover thrust in flight. With object tracking running, VFC can also yaw-follow a tracked target, gated by the tracking_trigger_ch RC channel (see AOI yaw tracking).

Backtrack

Backtrack continuously records the last backtrack_seconds (default 60) of position and yaw. When triggered — by RC-link loss, or by the switch on backtrack_rc_chan crossing backtrack_rc_thresh — it flies the recorded path in reverse to return the vehicle toward where the link was healthy, then hands control back. It is available both as a VFC submode and as a standalone offboard_mode: "backtrack".

Waypoint Missions (wps)

offboard_mode: "wps" flies a local-frame waypoint mission from a CSV file (written by tools like voxl-mapper to /tmp/voxl-mapper-traj_waypoints.csv). Each waypoint carries x, y, altitude, and an optional action — supported actions include taking a hires snapshot and land/return. The wps_* config fields above control interpolation, pauses, and whether the VFC controller flies the mission.

PX4 Configuration

In PX4 the flight mode is mapped to channel 6 with the majority of positions as “offboard mode” which is integer value 7 in the PX4 config. When the channel 6 mode switch is away from the user it has a high PWM value, so PX4 flight modes 5 and 6 that correspond to that position are set to “altitude mode” which is a value of 1:

1 1 COM_FLTMODE1 7 6
1 1 COM_FLTMODE2 7 6
1 1 COM_FLTMODE3 7 6
1 1 COM_FLTMODE4 7 6
1 1 COM_FLTMODE5 1 6
1 1 COM_FLTMODE6 1 6

Once that channel 6 mode switch is in either the middle position or all the way towards the user, PX4 goes into offboard mode and simply takes roll/pitch/yaw/thrust commands from VFC.

VFC monitors the same channel 6 switch position to decide which offboard submode to be in. By default the middle position (roughly 1500 µs) is VFC position mode, which uses VIO. When the switch is all the way towards the user (roughly 2000 µs), VFC swaps into the flow submode. This is configurable via /etc/modalai/vfc.conf (voxl-vision-hub 1.9.2 and newer); the relevant section looks like:

"vfc_submode_ch": 6,
"vfc_alt_mode_rc_min": 0,
"vfc_alt_mode_rc_max": 0,
"vfc_flow_mode_rc_min": 1700,
"vfc_flow_mode_rc_max": 2100,
"vfc_hybrid_flow_mode_rc_min": 0,
"vfc_hybrid_flow_mode_rc_max": 0,
"vfc_position_mode_rc_min": 1300,
"vfc_position_mode_rc_max": 1700,
"vfc_traj_mode_rc_min": 0,
"vfc_traj_mode_rc_max": 0,

Debugging

voxl-inspect-vfc prints live VFC state from the vfc pipe — per-axis setpoint/current/error, thrust, hover thrust, and desired vs actual submode. Use -t for tuning data and -v for a live plot.

Run voxl-vision-hub --debug_offboard on the bench over ADB, with propellers removed and a radio connected, to validate which submodes are being triggered in VFC. Running the binary manually automatically stops the background service. Other useful flags:

voxl-vision-hub -c    # regenerate/validate the config files and exit
voxl-vision-hub -o    # print odometry being sent to the autopilot
voxl-vision-hub -b    # print MAVLink messages being sent
voxl-vision-hub -d    # print MAVLink messages being received
voxl-vision-hub -r    # print VOA filter debug info

For general service debugging (logs, restarts), see Debugging Services.