voxl-vision-px4 service supports consuming a point cloud from the Depth from Stereo service and processing it into an obstacle_distance mavlink message for use by PX4. voxl-vision-px4 will use its history of VIO data to correct for the motion of the stereo cameras over time during the depth processing. Without VIO enabled it will just use PX4’s reported attitude to correct for orientation.
- VOXL Configuration
- PX4 Configuration
- Debugging Collision Prevention
- Frames of Reference Used
VOXL needs to be equipped with calibrated stereo cameras or a PMD TOF to use this feature. If you purchased a Flight Deck or M500 Developer Drone then the cameras should have been calibrated from the factory.
Next, make sure the Depth from Stereo server is running if you are using DFS for VOA. This can be done by looking at voxl-inspect-services. It is disabled by default by voxl-configure-mpa so you will likely need to enable it with
systemctl enable voxl-dfs-server.
Make sure the
/etc/modalai/voxl-vision-px4.conffile is set to
truewhich is is by default. This option is there mostly to enable you to disable voxl-vision-px4 from consuming DFS data if you wish to use DFS for something other than PX4’s Collision Prevention.
Check that point cloud data is being published from voxl-dfs-server using the voxl-inspect-points[/voxl-inspect-points/) tool.
Make sure you have a correct extrinsic relation from the
bodyframe to the
/etc/modalai/extrinsics.conf. For ModalAI reference drones this is set up automatically with voxl-configure-extrinsics and by extension the higher level voxl-configure-mpa tools. This can be checked with the voxl-inspect-extrinsics tool.
The above only enables voxl-vision-px4 to send obstacle data to PX4. PX4 must then be configured to use this data. For full information see the PX4 docs here: https://docs.px4.io/v1.10/en/computer_vision/collision_prevention.html
We suggest the following parameters for testing this indoors:
The delay can safely be set to 0 since we do accurate timestamping and delay compensation inside voxl-vision-px4. Increasing the delay over 0 will only cause twitchiness and unpredictable performance.
voxl-vision-px4 can be started with the following argument to print obstacle data to the screen before it is sent to PX4.
-s, --debug_stereo print detected stereo obstacles as linescan points
en_send_voa_to_qgc can be set to
/etc/modalai/voxl-vision-px4.conf to send the same mavlink OBSTACLE_DISTANCE messages to qgroundControl that are are sent to PX4. They can then be plotted with qGroundControl’s mavlink_inspector tool.
Use the following command to log VOA data, this is helpful for offline analysis
voxl-logger --cam stereo --cam dfs_disparity
The voxl-portal Cameras page can be used to view image sensor data, including processed stereo data. The following data is useful:
- Stereo is the data that goes into voxl-dfs-server - code
- DFS Disparity Scaled is the data that goes into the collision prevention algorithm.
- voa_pc is the output of the VOA algorithm and what is sent on the obstacle_distance message code
The stereo algorithm is highly sensitive to extrinsic calibration. If the sensors have been moved at all, it is likely they need to be re-calibrated. Details of how to calibrate stereo image sensors can be found here using voxl-portal.
The geometry module inside voxl-vision-px4 keeps track of the following frames of reference.
voxl-vision-px4 maintains a short history of visual odometry data to compensate for the time delay when computing the depth-from-stereo data as well as the motion and tilt of the drone body.