How to run Ubuntu on VOXL using Docker
Table of contents
- Overview
- Requirements
- Configuration
- Download & Push a Docker Image to VOXL
- Running a Docker Image
- Running a Docker Image with the voxl-docker tool
- optional: give the image your own tag
- Making Changes and Archive the image
- Configuring a program to run automatically inside a docker on boot
- Building Your Own Docker Image for VOXL
- Troubleshooting
- Access to MIPI cameras
- Access to the DSPs
- Access to the GPU on VOXL 1
- Move Docker image from /data to /mnt/sdcard
Overview
This document provides an overview of how to run docker containers on your VOXL developer board!
Requirements
The latest platform image installed
Configuration
VOXL 2
VOXL 2 has a Debian operating system and does not need any special treatment to use Docker beyond a normal apt style install.
For VOXL 2, please ignore the rest of this document and do not run any of the commands listed here.
Instead, follow the official Docker installation instructions.
VOXL and VOXL Flight
The latest platform image has Docker support built in.
After installing the latest platform image, run the configuration script voxl-configure-docker-support. Instructions below:
/ # voxl-configure-docker-support
detected APQ8096
Starting Wizard
Do you want to wipe the /etc/modalai/docker-autorun-script.sh file
back to factory default?
1) yes
2) no
#? 1
Copying default docker-autorun-script.sh to /etc/modalai/
Do you want to enable the modalai docker-daemon systemctl service?
This service is the same as the original docker systemctl service
but with customizations for VOXL. The original service will be
disabled and must remain disabled
1) yes
2) no
#? 1
disabling original docker service
Removed symlink /etc/systemd/system/multi-user.target.wants/docker.service.
enabling modalai docker-daemon service
Created symlink from /etc/systemd/system/default.target.wants/docker-daemon.service to /etc/systemd/system/docker-daemon.service.
Do you want to enable the modalai docker-autorun systemctl service?
this will let you configure a docker image to load on boot using
the user-configurable /etc/modalai/docker-autorun.sh file
1) yes
2) no
#? 1
disabling original docker service
enabling modalai docker-daemon service
loading hello-world docker image
successfully loaded hello-world
before
REPOSITORY TAG IMAGE ID CREATED VIRTUAL SIZE
hello-world latest 47bdf2e73073 15 months ago 13.26 kB
<none> <none> efc161607398 6 years ago 2.088 kB
tagging image as hello-image:latest
after
REPOSITORY TAG IMAGE ID CREATED VIRTUAL SIZE
hello-world latest 47bdf2e73073 15 months ago 13.26 kB
hello-image latest efc161607398 6 years ago 2.088 kB
hello-world docker image successfully loaded
enabling docker-autorun
Created symlink from /etc/systemd/system/default.target.wants/docker-autorun.service to /etc/systemd/system/docker-autorun.service.
done configuring voxl-docker-support
Now you can list the available images.
yocto:/# docker images
REPOSITORY TAG IMAGE ID CREATED VIRTUAL SIZE
hello-world latest efc161607398 Less than a second ago 2.088 kB
Install the roskinetic Docker to test
Download & Push a Docker Image to VOXL
Download the pre-built docker image roskinetic-xenial_v1_0.tgz from ModalAI Downloads. To build this image yourself from docker files, follow the instructions here. For the remainder of these instructions, it’s easier to use the pre-built image for now.
VOXL’s flash memory is broken up into partitions, the largest of which is /data/ which is 16GB. voxl-docker-support already configures docker to use this partition for storage. When pushing a large docker image to VOXL you should also push to this directory.
The following steps push an image and load it on VOXL. This will be slow so we suggest using a USB3.0 cable if you have one.
me@mylaptop:~$ adb push roskinetic-xenial_v1_0.tgz /data/
me@mylaptop:~$ adb shell
/ # bash
yocto:/# cd /data/
yocto:/data# docker load -i roskinetic-xenial_v1_0.tgz
Now look to make sure the image is loaded
yocto:/data# docker images
REPOSITORY TAG IMAGE ID CREATED VIRTUAL SIZE
hello-world latest efc161607398 3 years ago 2.088 kB
roskinetic-xenial v1.0 ef8f2f502423 About a minute ago 1.633 GB
Running a Docker Image
When running an image, we suggest telling docker to mount the home directory in Yocto to a directory inside the docker image to facilitate transferring data in and out. Here, we mount /home/root/ in Yocto to /root/yoctohome/ in docker. We also suggest using the –net=host option so processes such as ROS running inside Docker can transparently communicate with processes outside of the Docker. With this method ROS nodes can run and communicate both inside and outside of a Docker image seamlessly.
docker run -it --rm --privileged --net=host --name roskinetic -v /home/root:/root/yoctohome/:rw -w /root/ roskinetic-xenial:v1.0 /bin/bash
For a complete list of arguments, see the Official Docker Documentation. However, for the purposes of running docker images on VOXL, here are the arguments you will likely wish to use:
| Argument | Description |
|---|---|
| -it | Interactive mode, usually used when you wish to open a bash shell inside the docker. |
| –rm | Automatically remove the container when it exits to prevent containers stacking up. |
| –privileged | Gives the docker container access to /dev/ |
| –net=host | Lets network interfaces appear the same inside and outside of the docker container. |
| –name {name} | Give a name to the running instance so you can identify and attach to it by name. |
| -v {outside}:{inside}:rw | Mounts a directory outside the container to a directory inside the container. |
| -w {working_dir} | Set the working directory inside the container. |
Running a Docker Image with the voxl-docker tool
If you are using voxl-docker-support >= v1.1 as included with voxl-software-bundle 3.0 you can launch the docker image with voxl-docker which is faster to type and will mount your Yocto home directory inside the docker image at ~/yoctohome/
yocto:/home/root# voxl-docker -i roskinetic-xenial:v1.0
roskinetic:~$ ls
ros_catkin_ws yoctohome
roskinetic:~$
optional: give the image your own tag
To save time, you can tag the docker image you just loaded with a shorter name.
yocto:/home/root# docker tag roskinetic-xenial:v1.0 kinetic
yocto:/home/root# voxl-docker -i kinetic
roskinetic:~$ ls
ros_catkin_ws yoctohome
roskinetic:~$
Making Changes and Archive the image
Most of the time, you should be working in the shared directory /root/yoctohome/ inside the docker image so that changes you make in this directory are saved when you close the docker instance. However. if you wish you make changes to the docker image itself such as installing a new package, you need to commit these changes.
In one terminal, start the docker image and make the changes you want.
yocto:/home/root# voxl-docker -i roskinetic-xenial:v1.0
roskinetic:~$ sudo apt install htop
Now, leave this session running and in a second terminal use docker ps to see the running container instance.
yocto:/# docker ps
CONTAINER ID IMAGE COMMAND CREATED
616362e8cdb1 roskinetic-xenial:v1.0 "/bin/bash" About a minute ago
Then copy the container id of the instance you want to commit and use docker commit to save the changes. You can see a new image has been made right away.
yocto:/# docker commit 616362e8cdb1 roskinetic-xenial:htop
40869821b17f1b49196b354a3bbdf9fab70bdb4310b0b20d8470a361fa19ac59
yocto:/# docker images
REPOSITORY TAG IMAGE ID CREATED VIRTUAL SIZE
roskinetic-xenial htop 40869821b17f 29 seconds ago 1.635 GB
roskinetic-xenial v1.0 0e12749bdc00 2 hours ago 1.633 GB
To back up your newly modified image and transfer to other devices you can save it in an archive. The output of the following will match the prebuilt image we distribute at downloads.modalai.com
yocto:/# docker save roskinetic-xenial:v1.0 | gzip > /data/roskinetic-xenial_v1_0.tgz
Configuring a program to run automatically inside a docker on boot
The voxl-docker-support package includes a systemd service called docker-autorun which, once enabled, will execute the /etc/modalai/docker-autorun.sh script. Put whatever you like in that autorun script, but it’s intended for things running inside docker containers.
The docker_autorun.sh script contains two examples:
hello world
This example just starts the hello-world docker image which does nothing except print to the screen and exit. Note that we run in non-interactive mode with the -n argument and set the entrypoint as an empty string with the -e argument as we don’t want it executing anything
voxl-docker -n -i hello-world -e ""
mavros
Assuming you have following the mavros instructions, this example runs mavros automatically on boot. Note we use the -w argument to set the working directory to where the run_mavros.sh script is located.
voxl-docker -n -i roskinetic-xenial:v1.0 -w /root/yoctohome/mavros_test/ -e "/bin/bash run_mavros.sh"
Building Your Own Docker Image for VOXL
To build this image from docker files, follow the instructions here.
Troubleshooting
If you have issues, check the two systemd services provided by voxl-docker-support. Most likely you just need to reboot.
yocto:/# systemctl status docker-start
● docker-start.service
Loaded: loaded (/etc/systemd/system/docker-start.service; enabled; vendor preset: enabled)
Active: active (running) since Thu 1970-01-01 00:00:06 UTC; 11min ago
yocto:/# systemctl status docker-prepare
● docker-prepare.service
Loaded: loaded (/etc/systemd/system/docker-prepare.service; enabled; vendor preset: enabled)
Active: inactive (dead) since Thu 1970-01-01 00:00:36 UTC; 11min ago
Process: 3691 ExecStart=/usr/bin/docker-prepare.sh (code=exited, status=0/SUCCESS)
Main PID: 3691 (code=exited, status=0/SUCCESS)
Access to MIPI cameras
The MIPI cameras are accessed using Qualcomm proprietary 32-bit libraries and services that are not available in the Docker environment. There are a few workarounds to this.
voxl-camera-server
voxl-camera-server will publish camera frames using libmodal_pipe. These frames are available to a Docker if the /run/mpa directories are shared into the Docker
ROS
Use a ROS node outside of the Docker to publish camera frames and put ROS node(s) inside of the Docker that subscribe to the camera frames.
Access to the DSPs
The sensors DSP (sDSP) and applications DSP (aDSP) communicate to the CPUs via an RPC mechanism that has not yet been ported to the Docker environment.
Access to the GPU on VOXL 1
OpenCL is used to access the GPU. For example, OpenCV can be built with OpenCL support to accelerate certain operations using the GPU hardware. To get OpenCL working in Docker use the example project located here to get started.
Move Docker image from /data to /mnt/sdcard
In order to change where the root of the docker runtime points to in the docker service go to /etc/systemd/system/docker-daemon.service and change the following line so that it uses the sd card(ext4 format) in the voxl ExecStart=/usr/bin/docker daemon -g /mnt/sdcard
restart the service for the changes to take effect: systemctl restart docker-daemon.service
Delete the directories that docker uses in the data partition:cd /data; rm -rf containers linkgraph.db overlay tmp volumes graph network repositories-overlay trust
[NOTE] The previous commands will delete any images that were previously available.
You can now run voxl-configure-docker-support.sh and a hello world image will be created and the directories that were deleted from the /data partition will be in the /mnt/sdcard directory.
You can verify that you have freed up the /data partition via df -h
[Special Note] Not all SD cards can be used for Docker. In our lab tests a SanDisk Ultra 16GB card worked fine but a SanDisk Extreme 32GB card failed.