Link Search Menu Expand Document

Flying PX4 with VIO for Indoor Navigation

Table of contents

  1. Flying with VIO
    1. Confirm PX4 is Getting VIO Data
    2. Confirm Data is in the Right Coordinate Frame
    3. Take off in Position Mode
    4. Flip to Offboard Mode
  2. Next
  3. Troubleshooting VIO
    1. Debugging
      1. Troubleshooting Tips

Flying with VIO

Confirm PX4 is Getting VIO Data

As part of the IPK package installation a voxl-vision-px4 systemd service is created which starts the interface and ModalAI Vision Lib automatically on boot. This takes up to about 30 seconds to start. By the time you have plugged in a battery, walked back to your laptop, and opened QGroundControl voxl-vision-px4 should be running. Confirm before flight by opening the mavlink inspector and checking that X&Y values are non-zero in the ODOMETRY mavlink telemetry packets.

voxl-vio-mavlink-inspector.png

Alternatively, if your computer running QGroundControl has its volume turned up you should hear it announce the swap from “manual” to “position” mode when you flip the flight mode switch from up to middle. If VIO data is not being transmitted, the system will fall back to altitude mode and you will hear “position mode rejected, altitude flight mode”. In this case, go back to manual flight mode (switch up) and wait a few more seconds for voxl-vision-px4 to finish initializating.

If this doesn’t work, go back to Installation Step 1 and reconfirm VIO is working. You can also check the voxl-vision-px4 systemd service for indications of what might have gone wrong. If you have changed the Flight Core’s PX4 EKF2 settings, you may need to set the parameters back into VIO mode. To make this easier, we have a parameters file helper located here which are the parameters that ship with the VOXL m500 drone. Specifically the EKF2_AID_MASK is often overlooked as an important parameter for VIO flight.

adb shell
bash
systemctl status voxl-vision-px4

Confirm Data is in the Right Coordinate Frame

While looking at the ODOMETRY data in QGroundControl’s Mavlink Inspector, observe the xyz data as you move the quadcopter around. Make sure it aligns with NED coordinate frame. If there is a coordinate system mismatch PX4 will initially take off then very quickly run away out of control.

TODO: Video/pictures

Take off in Position Mode

It is safer to take off in Position Flight Mode than it is to flip to this mode mid-flight from Manual Flight Mode.

Take off and land with the same instructions as listed above but this time with the Flight Mode switch in the middle position. The quadcopter should take off straight up and be much easier to control than in manual mode.

Be ready to flip back to manual mode should anything go wrong. It is safer to flip to manual mode and land than to kill the motors mid-flight. Killing the motors mid-flight may result in the propellers loosening from the reverse-torque and flying straight up off the quadcopter. Only kill the motors after landing and spooling down the motors OR in a serious emergency.

Flip to Offboard Mode

While flying in Position Mode, you may flip the Flight Mode Switch all the way down to enable offboard mode. The quadcopter should immediately fly back to a position 2 meters above wherever VIO initialized. If you plugged in the battery and let the VOXL and PX4 power up at the takeoff location (suggested normal practice) then it should hold a position 2 meters above the takeoff location.

In Offboard Mode the 2 joysticks on the RC controller are ignored and the PX4 follows any commands given to it by VOXL over the UART link. In this case VOXL just sends a static location.

Next

Next page: Apriltag Relocalization

Troubleshooting VIO

Visual Inertial Odometry (VIO) is a vision-based system that fuses computer vision features with IMU to generate dead reckoning odometry. The two critical facets of the technology are:

  1. Visible features for the system to track from frame to frame
  2. Low-noise IMU measurements

Tracking performance degrades, sometimes significantly, if either of those two conditions are not met. For instance, the aircraft is flying in low-light or over low-feature environments like plain cement. Or, the aircraft design has a lot of vibration or does not sufficiently dampen the IMU. These issues are present in all types of vision-based SLAM systems. The ModalAI m500, RB5 Flight, Seeker, Starling and Flight Deck are all designed with these considerations in mind.

Debugging

  • The VIO Overlay in voxl-portal is a very good debug tool. The VIO Overlay enables viewing exactly what is going on with the VIO system and the system can be tuned quickly using that tool.
  • From the command line, voxl-inspect-qvio is a very helpful tool.

  • It is critical that the camera to IMU relationship is configured correctly.
  • Besides lack of visual features, vibration can significantly degrade VIO’s performance. The log analysis tools at https://logs.px4.io are great for analyzing IMU noise of a flight. voxl-inspect-imu and voxl-inspect-vibrationcan also be used to evaluate noise performance in real-time.
  • Self-illumination is an option to add to the airframe. Add diffuse light, make sure not to create hot spots or glare.
  • If purely an indoor application, using a different image sensor in the assembly may be better that the tracking sensor.VIO could be reconfigured to use one of the stereo sensors instead of the tracking sensor. The 45deg tracking sensor is there to support outdoor flights as well. If purely indoors, a forward facing sensor may be better.
  • Leverage the voxl-tag-detector for AprilTag relocalization. AprilTags are pretty typical in robotic warehouse environments. A well-surveyed AprilTag could help remove drift.

Troubleshooting Tips

If voxl-vision-px4 reports “VOXL rebooting PX4”, this is likely due to poor camera calibration. Follow the instructions here to calibrate the tracking sensor.