Torizon 5.0.0

Introduction

TorizonCore Builder Tool is a tool that allows you to easily generate a custom image of TorizonCore with device configuration changes, a custom device tree and overlays, pre-provisioned containers, external kernel modules, and a splash screen. This final image can then be deployed to the board using SSH, Toradex Easy Installer, or Torizon Platform Services updates features. The tool is designed to be used in production programming and on development as well.

In this article, the workflow of the tool will be explained in detail. It takes a Toradex image of TorizonCore and source codes of the device tree overlays, the device tree and the kernel modules, a splash screen image, and a docker-compose file of the application containers as input and generates as output an image of TorizonCore with all these customizations.

• 

  TorizonCore Builder Workflow

*OS Customization Files are device tree and overlays source codes, external kernel modules source files, a splash screen image, and directories with the configuration changes captured from the device. They can also be called version-controlled artifacts, as they should be version-controlled.

Approaches to Use TorizonCore Builder

There are two approaches to use TorizonCore Builder, which are in many cases interchangeable and have their own advantages and disadvantages, depending on the way TorizonCore Builder is used:

• Using The Build Command
• Using Standalone Commands

The first approach, with the build command, allows you to implement all customizations in one step, through a configuration YAML file, making it generally the easier approach. Integrating several customizations, such as including a kernel module, a custom device tree, and an overlay in one image, is just a matter of adding all of them to the customization YAML file.

Also, the build command is stateless, meaning that given the same inputs, it will always produce the same output. This makes this approach the recommended way to use on production programming and on CI/CD pipelines. When using the standalone commands, commands depend on a previous state, and they may produce different outputs depending on the order that they are executed. For example, executing dto apply and them dto remove will produce a different output compared to when executing dto remove and them dto apply.

However, every time the build command is executed, it runs a full pipeline (cycle) from scratch: it unpacks an input image, applies all the customizations, and creates an output custom image. If you plan to make incremental changes, it may become too slow and possibly annoying to run the full pipeline every time.

With the second approach, using standalone commands, you don't have to run a full pipeline every time you want to apply another customization, like a different splash screen or a new overlay, to your custom TorizonCore image. This can be useful if you are making incremental customizations on testing and development. As it does not execute a full pipeline every time, making different customization can be easier and faster.

For example, given a previous image of TorizonCore with all the wanted customizations, you can change the splash screen by just applying the new one with the splash command and executing the merge and deploy steps.

As explained in the next section, even with the build command, in some cases, standalone auxiliary commands are used in order to have all the necessary input artifacts.

Approach 1: Using The Build Command

• 

  TorizonCore Builder Workflow - Build Command

Workflow

• Create a configuration file with the desired settings (the build command can even help you with that);
• Run the auxiliary commands where needed, so all local artifacts possibly referenced by the configuration file are available. Also, depending on the input method, download an Easy Installer image of Torizon from Toradex;
• Customize the configuration file in order to select the desired input image, include the desired customization, and get the desired outputs.
• Run the build command;
• Deploy the output image to the board or to the Torizon Platform Services server;

Creating a configuration file

The configuration file follows the YAML format and syntax. The tool expects the configuration file to be made up of three sections, represented by the top-level properties (keys): input, customization, and output, as outlined below. The default name of this file is tcbuild.yaml.

 tcbuild.yaml
# Configuration file outline:
input:
  # Input section items (required).
customization:
  # Customization section items.
output:
  # Output section items (required).

The contents of the file are organized in a tree structure where the leaves are configurable properties. In order to set a certain property, you need to make sure all its parent properties are also uncommented (by removing the first # character).

Auxiliary Commands

Before running the build command, it is necessary to have all the input artifacts, accordingly with the desired customizations. This includes:

• A splash screen image.
• The source code of the device tree and the device tree overlays.
• The source code of the external kernel modules.
• The file with the configuration changes captured from the device.
• The docker-compose (YAML) configuration file of the application container or the directory with the container images.
• The Easy Installer image tarball of Torizon from Toradex (depending on the input mode) that can be obtained in the Toradex Download Links (Torizon, Linux BSP, WinCE and Partner Demos) article.

Some of these artifacts can be easily obtained using auxiliary TorizonCore Builder commands. Such as:

• The source code of the device tree and the device tree overlays from Toradex. They can be obtained with the dt checkout command.

    $ torizoncore-builder dt checkout

• The file with the configuration changes captured from the device. This can be easily obtained with the isolate command.

    $ torizoncore-builder isolate --remote-host 192.168.1.117 --remote-username torizon --remote-password torizon --changes-directory changes

• The directory with the container images. This can be generated from the docker-compose configuration file with the bundle command.

    $ torizoncore-builder bundle docker-compose.yml --bundle-directory bundle

Customizing the Configuration File

As explained, the configuration file contains three sections: input, customization, and output.

• The input section defines the source Toradex Easy Installer image of Torizon from Toradex, without containers, that will be customized.
• The customization section determines the modifications to be applied to the input to produce the desired output.
• The output section provides the details about the output artifact(s) to be produced by the torizoncore-builder build command.

To learn more about the structure of the file and see all the possible values for each section, check the "build" command - Detailed Manual.

Below is an example file that:

• As input, takes a locally downloaded tarball of a Toradex Easy Installer image, a URL with the Toradex Easy Installer image location or image parameters such as version, machine, distro, and build number;
• As customizations, applies a splash screen, a specific device tree, a display overlay, and a directory containing configuration changes captured from the device;
• As output, takes a directory containing the Docker container images of the applications and the directory to where the generated Toradex Easy Installer image with the containers and customizations will be deployed;

 tcbuild.yaml
input:
  easy-installer:
    local: torizon-core-docker-colibri-imx6-Tezi_5.3.0+build.7.tar

customization:
  splash-screen: splash.png
  filesystem:
     - changes/
  device-tree:
    include-dirs:
      - device-trees/include/
    custom: device-trees/dts-arm32/imx6dl-colibri-eval-v3.dts
    overlays:
      add:
        - device-trees/overlays/colibri-imx6_lcd-edt7_overlay.dts

output:
  easy-installer:
    local: torizon-core-docker-colibri-imx6-Tezi_5.3.0.CUSTOM
  bundle:
    dir: bundle/

Building The Image

To build this custom Toradex Easy Installer image of Torizon from this configuration file, simply execute the build command.

$ torizoncore-builder build --file tcbuild.yaml

As tcbuild.yaml is the default name, the --file argument is not required in this case and is just included here for demonstration purposes.

Deploying The Image

The output image can be deployed in three different ways to the board:

• Using Toradex Easy Installer.
• Directly on the board through SSH.
• Through Torizon Platform Services server.

As the output of the command is a Custom Easy Installer image of Torizon, to deploy it to the Torizon Platform Services server or through SSH, this custom image first needs to be unpacked, with the images unpack command.

$ torizoncore-builder images unpack torizon-core-docker-colibri-imx6-Tezi_5.3.0.CUSTOM

After this, the custom image can be deployed to Torizon Platform Services server with the platform push command:

$ torizoncore-builder platform push --credentials credentials.zip base

Or directly to the board through SSH with the deploy command:

$ torizoncore-builder deploy --remote-host 192.168.1.117 --remote-username torizon --remote-password torizon --reboot

The SSH and Platform Services ways do not support the deployment of images with pre-provisioned containers. To learn more about the deployment of pre-provisioned containers, please refer to Deploying Container Images to TorizonCore article.

Approach 2: Using The Standalone Commands

• 

  TorizonCore Builder Workflow - Standalone Commands

Workflow

• Download an Easy Installer image of Torizon from Toradex (the images download command recognizes which is the device and automatically downloads and unpacks the corresponding image of Torizon).
• Unpack the downloaded image;
• Run the auxiliary commands where needed, so all local artifacts possibly referenced by the configuration file are available;
• Apply the desired customizations in the unpacked image;
• Merge these customizations;
• (Just when including application containers) Deploy the output image without containers locally and combine it with the container images;
• Deploy the output image to the board or to the Torizon Platform Services server;

Downloading And Unpacking The Torizon Image From Toradex

There are two different ways to obtain an unpacked image of Torizon from Toradex.

For the first way, it is necessary an Easy Installer image tarball of Torizon from Toradex (depending on the input mode), that can be obtained in the Toradex Download Links (Torizon, Linux BSP, WinCE and Partner Demos) article.

After downloading this image, unpack it with the images unpack command. As TorizonCore Builder is a container, it will be unpacked in a Docker volume that will be later used by the customization and deployment commands without the need of passing it.

$ torizoncore-builder images unpack torizon-core-docker-colibri-imx6-Tezi_5.3.0+build.7.tar

Another way to obtain this unpacked image is with the images download command. This command connects to the board and downloads the appropriate Easy Installer image of Torizon from Toradex, unpacking the image in the same way as the images unpack command after the download. In this way, instead of passing the image tarball, the arguments passed are the device's IP address, username, and password.

$ torizoncore-builder images download --remote-host 192.168.1.117 --remote-username torizon --remote-password torizon

Differently from the build command, unless you want to change the version of Torizon or the directory with the unpacked image after you unpack the image once, it is not necessary to run these steps again. Making a different custom image from the same base image can be done without executing this step again.

Auxiliary Commands

Before applying the customizations to the custom TorizonCore image, it is necessary to have all the input artifacts accordingly to the desired customizations. This includes:

• A splash screen image.
• The source code of the device tree and the device tree overlays.
• The source code of the external kernel modules.
• The file with the configuration changes captured from the device.
• The docker-compose (YAML) configuration file of the application container or the directory with the container images.
• The Easy Installer image tarball of Torizon from Toradex (depending on the input mode) that can be obtained in the Toradex Download Links (Torizon, Linux BSP, WinCE and Partner Demos) article.

Some of these artifacts can be easily obtained using auxiliary TorizonCore Builder commands. Such as:

• The source code of the device tree and the device tree overlays from Toradex. They can be obtained with the dt checkout command.

    $ torizoncore-builder dt checkout

• The file with the configuration changes captured from the device. This can be easily obtained with the isolate command.

    $ torizoncore-builder isolate --remote-host 192.168.1.117 --remote-username torizon --remote-password torizon --changes-directory changes

• The directory with the container images. This can be generated from the docker-compose configuration file with the bundle command.

    $ torizoncore-builder bundle docker-compose.yml --bundle-directory bundle

Applying The Customizations

TorizonCore Builder has standalone commands for each desired customization.

To generate the image exemplified in the approach using the build command with the same customizations, one should run the following commands:

• Run the splash command to apply the splash-screen.

    $ torizoncore-builder splash splash.png

• Run the dt apply command to compile the device tree and store the generated device tree blob.

    $ torizoncore-builder dt apply device-trees/dts-arm32/imx6dl-colibri-eval-v3.dts

• Run the dto apply command to compile the device tree overlay (of the display in this example) and store the generated device tree overlay blob.

    $ torizoncore-builder dto apply device-trees/overlays/display-edt7_overlay.dts

Merging The Customizations Into The Image

The union command them merges all these customizations generated by the isolate, splash, kernel, dt, and dto commands. It generates a custom image of Torizon with all these customizations included.

$ torizoncore-builder union custom-branch --changes-directory changes

Combining with Containers

First, the custom image generated with the union command needs to be deployed into an output directory with the deploy command. This output directory will contain a Custom Easy Installer image of Torizon without containers.

$ torizoncore-builder deploy custom-branch --output-directory torizon-core-docker-colibri-imx6-Tezi_5.3.0.CUSTOM

After this, combine this custom image with the container images generated with the bundle command as explained above. This is done using the combine command.

$ torizoncore-builder combine --image-directory torizon-core-docker-colibri-imx6-Tezi_5.3.0.CUSTOM-WITHOUT-CONTAINERS --bundle-directory bundle --output-directory torizon-core-docker-colibri-imx6-Tezi_5.3.0.CUSTOM-WTIH-CONTAINERS

Deploying The Image

The output image can be deployed in three different ways to the board:

• Using Toradex Easy Installer. First, the image generated by the union command needs to be deployed locally, with the deploy command passing a local directory as an argument.

    $ torizoncore-builder deploy custom-branch --output-directory torizon-core-docker-colibri-imx6-Tezi_5.3.0.CUSTOM

• Directly on the board through SSH, with the deploy command, passing the device IP address, username, and password as arguments.

    $ torizoncore-builder deploy --remote-host 192.168.1.117 --remote-username torizon --remote-password torizon --reboot

• Through Torizon Platform Services updates server, with platform push command.

    $ torizoncore-builder platform push --credentials credentials.zip custom-branch

Attention: The SSH and Platform Services ways do not support the deployment of images with pre-provisioned containers. To learn more about the deployment of pre-provisioned containers, please refer to Deploying Container Images to TorizonCore article.

Deployment Methods Comparison: Development, Production Programming and In Field

As described in the deploy sections of each approach, there are three methods to deploy the custom image of TorizonCore to the device. Here is the recommended use of each:

• SSH: to be used when developing and evaluating single customizations and combinations of them. It allows for quick and direct deployment of the generated custom image of Torizon, without the need of using Toradex Easy Installer or the Torizon Platform Services.
• Toradex Easy Installer: to be used when you are creating your own custom TorizonCore image for production programming. It is the only method that allows the deployment of images with application containers pre-installed, allowing the board to boot with your containers from the first boot.
• Torizon Platform Services: to be used when you want to update the OS or the application on a running fleet of TorizonCore devices (also called In Field). It allows you to update your TorizonCore remotely, securely, on multiple devices at once, and without needing to re-flash the board.

Despite the recommendations, in some cases, it is possible to interchange between the methods. If you want, instead of using the SSH method, you can for example use the Torizon Platform Services updates features and the Toradex Easy Installer methods during development.

For example, we have reports of customers with remote teams and contractors who used the Torizon Platform Services for collaborative development.