Search by Tags

Pre-provisioning Docker Containers onto a TorizonCore image

 

Article updated at 17 Feb 2021
Compare with Revision




Select the version of your OS from the tabs below. If you don't know the version you are using, run the command cat /etc/os-release or cat /etc/issue on the board.



Remember that you can always refer to the Torizon Documentation, there you can find a lot of relevant articles that might help you in the application development.

Torizon 5.0.0

Introduction

In this article, we cover how to use TorizonCore Builder Tool to pre-provisioning Docker Containers onto a TorizonCore image to be deployed on several units in a production line using Toradex Easy Installer.

Development, Production Programming and Torizon Platform Services

The workflow described in this article adds container images to a Toradex Easy Installer image that can be flashed to the hardware for production programming. It is different from how container updates are applied using Torizon Platform Services or the Visual Studio Code IDE Extension for Torizon. The use of each method is recommended in different situations.

In summary, here's when you should use each:

  • Use Visual Studio Code IDE Extension for Torizon: to be used when you are developing and evaluating application containers, the Visual Studio Code IDE Extension for Torizon allows you to quickly generate, develop, deploy and debug Docker application containers.
  • Pre-provision container images onto TorizonCore: to be used when you are preparing your own custom TorizonCore image for production programming, allowing it to boot with your containers from the first time, without the need to pull those from the internet. You should not use this approach to generate images that will be used in updates. If you want to update simultaneously OS and application containers, you should use the synchronous update feature of our Torizon Platform.
  • Upload container images to Torizon Platform: to be used when you want to update the application on a running fleet of TorizonCore devices, without the need to re-flash the devices or update OS images into Torizon Platform.

Read the article Deploying Container Images to TorizonCore for a more comprehensive overview of all the methods for deploying containers to a TorizonCore image on various steps of the development cycle.

This article complies to the Typographic Conventions for Torizon Documentation.

Prerequisites

To complete these instructions you need:

Installing TorizonCore Builder

To install TorizonCore Builder, read our statements on OS and shell compatibility, then follow the instructions below, in order.

Create and enter a working directory where your customization will be stored:

$ mkdir ~/tcbworkdir
$ cd ~/tcbworkdir

Use the setup script named tcb-env-setup.sh, available on the Toradex Github, to setup TorizonCore Builder:

$ wget https://raw.githubusercontent.com/toradex/tcb-env-setup/master/tcb-env-setup.sh
$ source tcb-env-setup.sh

For advanced usage, run source tcb-env-setup.sh -h, or see the project README.

If using Windows, you must pass extra parameters to the script when using the following commands:

Tip: The setup script installs a bash completion script for TorizonCore Builder, making it possible to autocomplete commands and parameters by just pressing the TAB key.

Verify that the command torizoncore-builder is available:

$ torizoncore-builder --help

Warning: Make sure to: (1) source the script, (2) every time you open a new terminal, (3) inside the working directory you have previously created. Otherwise, it will not work as intended and, most importantly, the torizoncore-builder alias will not be created properly.

Pre-provisioning Docker Containers onto a TorizonCore image

The process to preinstall or pre-provision Docker Containers onto that installer image comprises two stages:

  1. Bundling the Docker Containers: This step creates a tarball containing all the containers as well as a Docker Compose file referencing those containers using a specific hash.

  2. Applying the Docker Containers Bundle to a TorizonCore image: In this step, the tool takes a Toradex Easy Installer image of TorizonCore (with the Docker engine) and combines it with the previously bundled Docker Containers.

Bundling the Docker Container Images

The downloading (pulling) of Docker container images from the Docker registries is done with TorizonCore Builder bundle command. This command takes a configuration Docker Compose YAML file as input and generates a directory, with name and location defined in the --bundle-directory argument, with the bundled (downloaded from the Docker registries) Docker container images. You can learn more about this command at our command manual.

A sample Docker Compose YAML file can be downloaded, based on your device, by clicking one of the links below:

With a Docker Compose YAML file present in your working directory, execute the command below:

$ torizoncore-builder bundle docker-compose.yml --bundle-directory bundle

The container bundle tarball along with a modified version of the Docker Compose file will be located at the new directory bundle in your working directory.

In case you are referencing multi-platform container images, the above command will likely fail because no platform was specified. To solve this, you should pass the --platform switch selecting the appropriate platform which depends on the target device.

For devices based on i.MX6/i.MX7 based SoMs:

$ torizoncore-builder bundle --platform=linux/arm/v7 docker-compose.yml --bundle-directory bundle

And for those based on i.MX8 based SoMs:

$ torizoncore-builder bundle --platform=linux/arm64 docker-compose.yml --bundle-directory bundle

Note: Pass the --platform switch to torizoncore-builder bundle when working with multi-platform Docker images.

Applying Docker Container Images to a Custom Image

There are two possible approaches to apply the customization and generate a custom Toradex Easy Installer image, described in the next two sections Approach 1 and Approach 2. These approaches in some cases are interchangeable and in some not as described in the next sections.

To learn about TorizonCore Builder workflow and the different approaches to use the tool, with explanatory diagrams, please refer to the TorizonCore Builder - Workflow article.

Attention: Both approaches generate a custom Toradex Easy Installer image as output, so the approaches should be followed alternatively and not in sequence.

Approach 1: Applying Docker Container Images to a Custom Image Using the Build Command

TorizonCore Builder build command generates a custom TorizonCore image with the Docker container images ready to be installed with Toradex Easy Installer, named torizon-core-docker-colibri-imx6-Tezi_5.3.0.CUSTOM in the example below. This is achieved using a configuration YAML file, tcbuild.yaml as default. It requires a Toradex Easy Installer image of TorizonCore, torizon-core-docker-colibri-imx6-Tezi_5.3.0+build.7.tar in this case, as input.

This is the recommended approach on production programming and on CI/CD (continuous integration / continuous development) pipelines.

To learn about TorizonCore Builder workflow and the different approaches to use the tool, with explanatory diagrams, please refer to the TorizonCore Builder - Workflow article.

Here there are two possible configuration YAML file approaches. The first is passing the directory with the pre-bundled Docker container images as a parameter and the second is passing the Docker Compose file as a parameter.

Pre-Bundled Container Images Directory As Parameter

The directory with the Docker container images is passed in output: bundle: dir.

tcbuild.yaml
# Sample configuration file
input:
  easy-installer:
    local: images/torizon-core-docker-colibri-imx6-Tezi_5.3.0+build.7.tar
#Sample customization: insert pre-provisioned Docker container images with pre-bundled container images
output:
  easy-installer:
    local: torizon-core-docker-colibri-imx6-Tezi_5.3.0.CUSTOM
    bundle:
      dir: bundle/

Docker Compose File As Parameter

Alternatively to the configuration file above, one can pass the Docker Compose file YAML file instead of the directory with the Docker container images. This method does not require the use of the bundle command prior to it, as it implicitly executes the bundle and the combine steps together in just one step. However, it downloads the Docker container images from the Docker registries, an operation that generally takes some time, every time the build command is executed. The directory with the Docker Compose file is passed in output: bundle: compose-file.

tcbuild.yaml
#Sample customization: insert pre-provisioned Docker container images
output:
  easy-installer:
    local: torizon-core-docker-colibri-imx6-Tezi_5.3.0.CUSTOM
    bundle:
      compose-file: custom/docker-compose.yml

Build The Custom Toradex Easy Installer Image

To generate the TorizonCore image, run the command below, in the same directory where the tcbuild.yaml file is:

$ torizoncore-builder build

...
1091 metadata, 12741 content objects imported; 412.2 MB content written                                                                                                                                   
Pulling done.
Deploying OSTree with checksum 58629613a342197c31c5911d0874aac1b0fcb46b68a63f59760c03bacc4df08a
Deploying done.
Copy files not under OSTree control from original deployment.
Packing rootfs...
Packing rootfs done.

=>> Build command successfully executed!

In case of using a configuration file with a different name than tcbuild.yaml, run the command specifying the configuration file name:

$ torizoncore-builder build --file <configuration_file_name>

Approach 2: Applying Docker Container Images to a Custom Image Using Standalone Commands

In this second approach, instead of using a configuration YAML file and a one-step command, the generation of the custom TorizonCore image with the Docker container images is done using standalone commands, each performing one step towards this generation.

This approach is especially useful when making incremental changes, generating multiple images with different container images. As you will see, once you have a custom Toradex Easy Installer image without containers, generating images with different container images requires just the combine command with a different --bundle-directory argument.

To learn about TorizonCore Builder workflow and the different approaches to use the tool, with explanatory diagrams, please refer to the TorizonCore Builder - Workflow article.

To generate a custom Toradex Easy Installer image without the containers, that will be combined with the built Docker container images, follow the sequence of steps below.

Generate Custom Toradex Easy Installer Image Without Containers

You just need to execute this once. If the custom Toradex Easy Installer Image of TorizonCore without the containers has not changed this sequence of commands unpack, union and deploy do not need to be executed again. Therefore, it is possible to jump to the combine command.

If you have not unpacked an image yet, download a base TorizonCore image (preferably without containers) inside the TorizonCore Builder working directory, then run the command below to unpack it. In the example below the torizon-core-docker-colibri-imx6-Tezi_5.3.0+build.7.tar image is used as a reference:

$ torizoncore-builder images unpack torizon-core-docker-colibri-imx6-Tezi_5.3.0+build.7.tar

If you want to change the TorizonCore base image, download the new image and run the images unpack command again, passing the new image as the argument.

For more details about the images unpack command, please check the images unpack command in the commands manual.

Instead of using the images unpack you can use the images download command. This command checks which is the connected Toradex SoM, downloads the compatible latest quarterly release of a TorizonCore image without containers, and unpacks this image.

$ torizoncore-builder images download --remote-host 192.168.1.117 --remote-username torizon --remote-password torizon

Change the arguments --remote-host,--remote-username and --remote-password to your board IP Address, username and password, respectively.

For more details on how the images download command works, please check the images download command in the commands manual.

Merge customizations to base Toradex Easy Installer image of Torizon - use whatever branch name you want. In this case, there are no customizations, like a new kernel module or a splash screen, but it is still appropriate to run this command as the branch name will be needed in the next command.

As an example, to commit changes into a branch named custom-branch use the command below, accordingly with the TorizonCore Builder version:

$ torizoncore-builder union custom-branch

Applying changes from STORAGE/dt.
Commit 58629613a342197c31c5911d0874aac1b0fcb46b68a63f59760c03bacc4df08a has been generated for changes and is ready to be deployed.
$ torizoncore-builder union --union-branch=custom-branch

Warning: We recommend that you switch to the latest version of TorizonCore Builder to enjoy its simpler and more consistent user interface besides other improvements and bug fixes.

For more details about the union command, please check the union command in the commands manual.

Use the previously created custom-branch branch to generate a TorizonCore image, ready to be installed with the Toradex Easy Installer, that will be in the directory torizon-core-docker-colibri-imx6-Tezi_5.3.0.CUSTOM in this case. This image, however, still does not have the containers:

$ torizoncore-builder deploy custom-branch --output-directory torizon-core-docker-colibri-imx6-Tezi_5.3.0.CUSTOM

Combine the Docker Containers Bundle with a TorizonCore image

At this point, you should have the following directory/file structure relative to your working directory:

├── bundle
├── docker-compose.yml
└── torizon-core-docker-colibri-imx6-Tezi_5.3.0.CUSTOM
    ├── image.json
    ├── ...
    └── wrapup.sh

Now we can create a new image by combining the custom Toradex Easy Installer image without containers, generated in the anterior step, with the Docker container images, that were built with the bundle command. To combine them, run the command below:

$ torizoncore-builder combine --image-directory torizon-core-docker-colibri-imx6-Tezi_5.3.0.CUSTOM --bundle-directory bundle --output-directory torizon-core-docker-colibri-imx6-Tezi_5.3.0.CUSTOM-WITH-CONTAINERS

The --image-directory argument, torizon-core-docker-colibri-imx6-Tezi_5.3.0.CUSTOM in this case, is the source image without containers. The --bundle-directory argument, named bundle in this case, is the directory with the built Docker container images. The --output-directory argument, torizon-core-docker-colibri-imx6-Tezi_5.3.0.CUSTOM-WITH-CONTAINERS in this case, is the directory containing the custom Toradex Easy Installer image with the Docker containers that will be generated by the combine command.

Deploy The Custom Toradex Easy Installer Image

The custom Easy Installer image with the Docker container images can then be deployed (flashed) to the device using Toradex Easy Installer. This is the recommended approach for production programming. You can learn more about this by reading our article about production programming and provisioning at scale.

During development and once the device is on the field the most recommended way of deploying a custom image of TorizonCore (with customizations such as a splash screen, device tree overlays, etc) and Docker container images is doing it separately.

To more about the deployment of Docker container images and how to proceed in each situation, please refer to the Deploying Container Images to TorizonCore article. Also, to learn more about TorizonCore Builder workflow and how to proceed with a custom TorizonCore image on each situation, please refer to the TorizonCore Builder - Workflow article.

Remarks About Using Private Registries

If you are using a private Docker registry, you will need to pass other command-line options. For a quick reference, run torizoncore-builder bundle --help or read the TorizonCore Builder Tool - The Bundle Command.

For Torizon Platform, you will also have to add the private registry credentials to the TorizonCore image. Read the article Using Private Registries With Torizon Platform for more details.

Torizon 4.0.0

Introduction

This article shows how to preinstall or pre-provision a Docker Containers onto a TorizonCore image with a Docker engine. This process is especially useful when preparing an image to be flashed on several units in production.

In this article, we show how to preinstall a Docker Container onto a TorizonCore image using either a Linux Host PC or a Windows machine.

The process comprises two stages:

  • Bundle the Docker Containers: This step creates a tarball containing all the containers as well as a Docker Compose file, which references those containers using a specific hash.

  • Combine the Docker Containers Bundle with a TorizonCore image: In this step, the script takes a Toradex Easy Installer image of TorizonCore with the Docker engine and combines it with the previously bundled Docker Containers.

The process makes use of Docker in Docker, which requires the script to have access to the host Docker engine.

This article complies to the Typographic Conventions for Torizon Documentation.

Pre-requisites

The pre-requisite to complete these instructions are:

Using TorizonCore builder on Linux

Preparation

Make sure to use an empty working directory and navigate to it. This directory is shared (bind-mounted) with the container this tool runs in, and it stores temporary files and the final image.

$ mkdir ~/myworkdir/
$ cd ~/myworkdir/

Inside the myworkdir directory, create the image and image_output folders.

$ mkdir image image_output

For convenience, create an alias called torizoncore-builder for the container execution:

$ alias torizoncore-builder='docker run --rm -it -v $(pwd):/workdir --net=host \
-v /var/run/docker.sock:/var/run/docker.sock torizon/torizoncore-builder'

Note: The bind mounts of the host Docker need to match the location inside the torizoncore-builder script (/builder). For that reason, you should use this alias (torizoncore-builder) only in this directory which it was created.

Gather the image file and the docker-compose file

Copy your project's docker-compose.yml file to the working directory.

As an example, we will use a simple docker-compose file:

docker-compose.yml
version: '2.4'
 
services:
  hello-world: 
    image: hello-world

Add the .tar file containing the TorizonCore Image with no-container pre-provisoned for offline instalation to the image directory.

After copying these two files to the working directory, we check its content:

$ tree .
.
├── docker-compose.yml
├── image
│   └── teziimage.tar
└── image_output

Create a container bundle

With the files on the directory, call the bundle command to create a container bundle:

$ torizoncore-builder bundle --host-workdir=$(pwd) --platform=linux/arm/v7 --file docker-compose.yml

Attention: Replace --platform=linux/arm/v7 by --platform=linux/arm/v8 when using a 64-bit SoC (iMX8).

This step downloads the Docker in Docker container image and uses the docker-compose.yml template to download all images referenced in this Docker Compose file. The tool then compresses the complete set of Docker containers on a tar file.

The container bundle tarball will be located at the new directory ./bundle/.

Combine the container images bundle with TorizonCore image

Before running this step, make sure to have a TorizonCore image tarball available in your work directory, as explained earlier.

To combine the container images bundle with TorizonCore image:

$ torizoncore-builder combine --image-directory ./image --output-directory ./image_output

After running this process, a combined TorizonCore image is available in the output directory specified by the --output-directory flag. This image is ready to be installed into the SoM using Toradex Easy Installer

Using TorizonCore builder on Windows 10

Warning: This procedure requires Windows 10 with WSL2 installed

To execute the following instructions, the Windows Subsystem for Linux version 2(WSL2) is necessary. WSL2 is available when updating Windows 10 to the version 2004 (Build 19041) or higher.

Preparation

Open the Windows command prompt (cmd) and make sure to use an empty working directory and navigate to it. This directory is shared (bind-mounted) with the container this tool runs in, and it stores temporary files and the final image.

C:\Users\My User>mkdir myworkdir/
C:\Users\My User>cd myworkdir/
C:\Users\My User\myworkdir>

Inside the myworkdir directory, create the image and image_output folders.

C:\Users\My User\myworkdir>mkdir image
C:\Users\My User\myworkdir>mkdir image_output

Gather the image file and the docker-compose file

Copy your project's docker-compose.yml file to the working directory.

As an example, we will use a simple docker-compose file:

docker-compose.yml
version: '2.4'
 
services:
  hello-world: 
    image: hello-world

Add the .tar file containing the TorizonCore Image with no-container pre-provisoned for offline instalation to the image directory.

After copying these two files to the working directory, we check the myworkdir directory content:

├── docker-compose.yml
├── image
│   └── teziimage.tar
└── image_output

Create a container bundle

With the files on the directory, call the bundle command to create a container bundle:

C:\Users\My User\myworkdir>docker run --rm -it --net=host ^
-v "C:\Users\My User\myworkdir":/workdir -v /var/run/docker.sock:/var/run/docker.sock ^
torizon/torizoncore-builder bundle --host-workdir="/c/Users/My User/myworkdir" ^
--platform=linux/arm/v7 --file docker-compose.yml

Attention: Replace --platform=linux/arm/v7 by --platform=linux/arm/v8 when using a 64-bit SoC (iMX8).

This step downloads the Docker in Docker container image and uses the docker-compose.yml template to download all images referenced in this Docker Compose file. The tool then compresses the complete set of Docker containers on a tar file.

The container bundle tarball will be located at the new directory ./bundle/.

Combine the container images with TorizonCore

Before running this step, make sure to have a TorizonCore image tarball available in your work directory, as explained earlier.

To combine the container images bundle with TorizonCore image:

C:\Users\My User\myworkdir>docker run --rm -it --net=host ^
-v "C:\Users\My User\myworkdir":/workdir -v /var/run/docker.sock:/var/run/docker.sock ^
torizon/torizoncore-builder combine --image-directory image ^
--output-directory image_output

After running this process, a combined TorizonCore image is available in the output directory specified by the --output-directory flag. This image is ready to be installed into the SoM using Toradex Easy Installer