Link Search Menu Expand Document
ModalAI Technical Docs

VOXL Vision PX4 Collision Prevention

The voxl-vision-px4 service supports consuming a point cloud from the Depth from Stereo service and processing it into an obstacle_distance mavlink message for use by PX4. voxl-vision-px4 will use its history of VIO data to correct for the motion of the stereo cameras over time during the depth processing. Without VIO enabled it will just use PX4’s reported attitude to correct for orientation.

Table of contents

  1. VOXL Configuration
  2. PX4 Configuration
  3. Debugging Collision Prevention
    1. voxl-vision-px4
      1. Logging VOA Data
    2. voxl-portal to View Depth Map
    3. Calibrating Stereo Image Sensors
  4. Frames of Reference Used
  5. Next

VOXL Configuration

  • VOXL needs to be equipped with calibrated stereo cameras to use this feature. If you purchased a Flight Deck or M500 Developer Drone then the cameras should have been calibrated from the factory.

  • Next, make sure the Depth from Stereo server is running. This can be done by looking at voxl-inspect-services. It is disabled by default by voxl-configure-mpa so you will likely need to enable it with systemctl enable voxl-dfs-server.

  • Make sure the "en_voa" field in /etc/modalai/voxl-vision-px4.conf file is set to true which is is by default. This option is there mostly to enable you to disable voxl-vision-px4 from consuming DFS data if you wish to use DFS for something other than PX4’s Collision Prevention.

  • Check that point cloud data is being published from voxl-dfs-server using the voxl-inspect-points[/voxl-inspect-points/) tool.

  • Make sure you have a correct extrinsic relation from the body frame to the stereo frame in /etc/modalai/extrinsics.conf. For ModalAI reference drones this is set up automatically with voxl-configure-extrinsics and by extension the higher level voxl-configure-mpa tools. This can be checked with the voxl-inspect-extrinsics tool.

PX4 Configuration

The above only enables voxl-vision-px4 to send obstacle data to PX4. PX4 must then be configured to use this data. For full information see the PX4 docs here: https://docs.px4.io/v1.10/en/computer_vision/collision_prevention.html

We suggest the following parameters for testing this indoors:

CP_DIST: 1.5
CP_DELAY: 0.0
CP_GUIDE_ANG: 0.0
CP_GO_NO_DATA: 1

The delay can safely be set to 0 since we do accurate timestamping and delay compensation inside voxl-vision-px4. Increasing the delay over 0 will only cause twitchiness and unpredictable performance.

Debugging Collision Prevention

voxl-vision-px4

voxl-vision-px4 can be started with the following argument to print obstacle data to the screen before it is sent to PX4.

-s, --debug_stereo          print detected stereo obstacles as linescan points

The parameter en_send_voa_to_qgc can be set to true in /etc/modalai/voxl-vision-px4.conf to send the same mavlink OBSTACLE_DISTANCE messages to qgroundControl that are are sent to PX4. They can then be plotted with qGroundControl’s mavlink_inspector tool.

Logging VOA Data

Use the following command to log VOA data, this is helpful for offline analysis

voxl-logger --cam stereo --cam dfs_disparity

voxl-portal to View Depth Map

The voxl-portal Cameras page can be used to view image sensor data, including processed stereo data. The following data is useful:

  • Stereo is the data that goes into voxl-dfs-server - code
  • DFS Disparity Scaled is the data that goes into the collision prevention algorithm.
  • voa_pc is the output of the VOA algorithm and what is sent on the obstacle_distance message code

Calibrating Stereo Image Sensors

The stereo algorithm is highly sensitive to extrinsic calibration. If the sensors have been moved at all, it is likely they need to be re-calibrated. Details of how to calibrate stereo image sensors can be found here using voxl-portal.

Frames of Reference Used

The geometry module inside voxl-vision-px4 keeps track of the following frames of reference.

px4_chart-geometry_overview

voxl-vision-px4 maintains a short history of visual odometry data to compensate for the time delay when computing the depth-from-stereo data as well as the motion and tilt of the drone body.

px4_chart-stereo_frames

Next

Next page: Nuttx Shell