Flying PX4 with VIO for Indoor Navigation
Table of contents
Flying with VIO
Confirm PX4 is Getting VIO Data
As part of the package installation a voxl-vision-px4 systemd service is created which starts the interface automatically on boot. This takes up to about 30 seconds to start. By the time you have plugged in a battery, walked back to your laptop, and opened QGroundControl voxl-vision-px4 should be running. Confirm before flight by opening the mavlink inspector and checking that X&Y values are non-zero in the ODOMETRY mavlink telemetry packets.
Alternatively, if your computer running QGroundControl has its volume turned up you should hear it announce the swap from “manual” to “position” mode when you flip the flight mode switch from up to middle. If VIO data is not being transmitted, the system will fall back to altitude mode and you will hear “position mode rejected, altitude flight mode”. In this case, go back to manual flight mode (switch up) and wait a few more seconds for voxl-vision-px4 to finish initializating.
If this doesn’t work, go back to Installation Step 1 and reconfirm VIO is working. You can also check the voxl-vision-px4 systemd service for indications of what might have gone wrong. If you have changed the Flight Core’s PX4 EKF2 settings, you may need to set the parameters back into VIO mode. To make this easier, we have a parameters file helper located here which are the parameters that ship with the VOXL m500 drone. Specifically the EKF2_AID_MASK is often overlooked as an important parameter for VIO flight.
adb shell
bash
systemctl status voxl-vision-px4
Confirm Data is in the Right Coordinate Frame
While looking at the ODOMETRY data in QGroundControl’s Mavlink Inspector, observe the xyz data as you move the quadcopter around. Make sure it aligns with NED coordinate frame. If there is a coordinate system mismatch PX4 will initially take off then very quickly run away out of control.
TODO: Video/pictures
Take off in Position Mode
It is safer to take off in Position Flight Mode than it is to flip to this mode mid-flight from Manual Flight Mode.
Take off and land with the same instructions as listed above but this time with the Flight Mode switch in the middle position. The quadcopter should take off straight up and be much easier to control than in manual mode.
Be ready to flip back to manual mode should anything go wrong. It is safer to flip to manual mode and land than to kill the motors mid-flight. Killing the motors mid-flight may result in the propellers loosening from the reverse-torque and flying straight up off the quadcopter. Only kill the motors after landing and spooling down the motors OR in a serious emergency.
Flip to Offboard Mode
While flying in Position Mode, you may flip the Flight Mode Switch all the way down to enable offboard mode. The quadcopter should immediately fly back to a position 2 meters above wherever VIO initialized. If you plugged in the battery and let the VOXL and PX4 power up at the takeoff location (suggested normal practice) then it should hold a position 2 meters above the takeoff location.
In Offboard Mode the 2 joysticks on the RC controller are ignored and the PX4 follows any commands given to it by VOXL over the UART link. In this case VOXL just sends a static location.
Troubleshooting VIO
Visual Inertial Odometry (VIO) is a vision-based system that fuses computer vision features with IMU to generate dead reckoning odometry. The two critical facets of the technology are:
- Visible features for the system to track from frame to frame
- Low-noise IMU measurements
Tracking performance degrades, sometimes significantly, if either of those two conditions are not met. For instance, the aircraft is flying in low-light or over low-feature environments like plain cement. Or, the aircraft design has a lot of vibration or does not sufficiently dampen the IMU. These issues are present in all types of vision-based SLAM systems. The ModalAI m500, RB5 Flight, Seeker, Starling and Flight Deck are all designed with these considerations in mind.
It is strongly recommended to be comfortable switching between manual and position control using a backup R/C while getting comfortable with when and where VIO works well. VIO is definitely not guaranteed to work in all scenarios.
Debugging
- The VIO Overlay in voxl-portal is a very good debug tool. The VIO Overlay enables viewing exactly what is going on with the VIO system and the system can be tuned quickly using that tool.
From the command line, voxl-inspect-qvio is a very helpful tool.
- It is critical that the camera to IMU relationship is configured correctly.
- Besides lack of visual features, vibration can significantly degrade VIO’s performance. The log analysis tools at https://logs.px4.io are great for analyzing IMU noise of a flight. voxl-inspect-imu and voxl-inspect-vibrationcan also be used to evaluate noise performance in real-time.
- Self-illumination is an option to add to the airframe. Add diffuse light, make sure not to create hot spots or glare.
- If purely an indoor application, using a different image sensor in the assembly may be better that the tracking sensor. VIO could be reconfigured to use one of the stereo sensors instead of the tracking sensor. The 45deg tracking sensor is there to support outdoor flights as well. If purely indoors, a forward facing sensor may be better. See Configure Extrinsics
- Leverage the voxl-tag-detector for AprilTag relocalization. AprilTags are pretty typical in robotic warehouse environments. A well-surveyed AprilTag could help remove drift.
Troubleshooting Tips
Logging VIO Data
Use the following command to log VIO data, this is helpful for offline analysis
voxl-logger --preset_odometry
“VOXL rebooting PX4”
If voxl-vision-px4 reports “VOXL rebooting PX4”, this is likely due to poor camera calibration. Follow the instructions here to calibrate the tracking sensor.
Position Mode Rejected
If LOCAL_POSITION_NED and MAVLink Odometry message are reading 0 for X and Y after movement of the robot, it is likely PX4’s EKF2 is set to require a magnetomter to initialize.
PX4 1.12 and later have the following parameters which enable using VIO without a magnetometer:
## disable mag
SYS_HAS_MAG 0
EKF2_MAG_TYPE 5
PX4 1.11 and earlier do not arm without a magnetometer as it requires a magnetometer to initialize the PX4 EKF. ModalAI’s version of Flight Core Firmware has a patch to remove this requirement. This should enable arming without a GPS+Mag unit connected to Flight Core or VOXL Flight.
ArduPilot VIO
Support for ArduPilot is progressing! Here is the ArduPilot pull request for VOXL-compatible Visual Inertial Odometry support Github