libmodal-cv
Introduction
libmodal-cv is ModalAI’s internal computer vision library designed to be run on VOXL. The library implements a number of discrete computer vision tasks as well as a few pipelines. Both the algorithms and pipelines make use of hardware optimizations to use less CPU and power than their complements in other libraries like OpenCV. Importantly, the code for libmodal-cv is closed source and so its usage is only available through headerfiles. These headerfiles can be found in /usr/include/modalcv on VOXL and VOXL 2.
The following sections enumerate some of the common paradigms in the library as well as a brief listing on each algorithm and pipeline which has been implemented.
Common Paradigms
Types in the library are typically prefixed with mcv_cvp_ and suffixed with _t. (e.g. mcv_cvp_undistort_config_t). Nearly all modules start with some sort of configuration object which takes basic type or enum parameters. For example in the descriptor calculation and matching module we have a configuration object:
/**
* @brief Represents the configuration for descriptor calculation and matching
*/
typedef struct mcv_cvp_dcm_config_t
{
int w; // width of the input image
int h; // height of the input image
} mcv_cvp_dcm_config_t;
which simply needs the width and height for the image that the descriptor calculation and matching will be performed on. These configuration objects are usually passed to some sort of init method which returns some sort of void* handle. For example:
typedef void* mcv_cvp_undistort_handle;
/**
* @brief Initializes the undistort process using the specified configuration
*
* @param config A populated undistort configuration object
* @return The generated undistort handle if successful, NULL otherwise
*/
mcv_cvp_undistort_handle mcv_cvp_undistort_init(mcv_cvp_undistort_config_t config);
Inside of this mcv_cvp_undistort_handle is all of the internal information needed to operate the module. This oftentimes contains proprietary information related to the memory management, hence why this is returned as a void *. After calling an init function and receiving a non-NULL handle, one can then call the corresponding “process” function using the handle. The structure of the process functions vary from module to module as different modules need different numbers of images and different types of output structures. Some modules also contain multiple process functions when there are more than one way to use the same functionality. An example process function might look like:
/**
* @brief Executes the undistort process for a provided input image.
*
* @param handle The undistort handle created via mcv_cvp_undistort_init
* @param inImg The input image for undistorting
* @param outImg The output image object
* @return 0 on success or -1 on failure
*/
int mcv_cvp_undistort_process(
mcv_cvp_undistort_handle handle,
uint8_t* inImg,
uint8_t* outImg
);
We can see that for image undistortion, we need to provide the undistortion handle we obtained from the init call as well as a pointer to the input image for undistortion and a pointer to write the output (undistorted) image to. Again, all of the internal information needed to perform the undistortion is stored within the handle. Typically as much preprocessing as possible is performed in the init call so that these process function calls can be repeatedly called in a loop or callback context. Once you’re done with a specific module, you need only to call that module’s deinit function which typically has a signature like:
/**
* @brief De-initializes the undistort process
*
* @param handle The handle generated from mcv_undistort_init
* @return 0 on success or -1 on failure
*/
int mcv_cvp_undistort_deinit(mcv_cvp_undistort_handle handle);
needing only the handle to tear down all of the necessary memory structures. Some modules may need more, though, so always adhere to the method signature.