Overview

To simplify the process of generating custom images for deployment and provisioning, the TorizonCore Builder tool provides the “build” command, whose inputs and outputs are depicted below:

• 

  Inputs and outputs of the torizoncore-builder build command

The build process is fully controlled by a configuration file (having a default name of tcbuild.yaml) which informs the tool what installer image to take as input, what modifications are to be applied (along with the associated data) and what output to produce. Usually the input would be a Toradex Easy Installer image, while the output would be a new, customized image of the same kind.

The configuration file and the artifacts referenced by it are common filesystem objects such as files or directories which can be version controlled allowing for reproducible builds. The artifacts used to customize the output include a splash-screen image, source files for device-tree and device-tree overlays, source code for kernel modules, etc.

Configuration File Contents

The configuration file follows the YAML format and syntax. The tool expects it to be made up of three sections, represented by the top-level properties (keys): input, customization and output as outlined below:

# Configuration file outline:
input:
  # Input section items (required).
customization:
  # Customization section items.
output:
  # Output section items (required).

The expected types of the elements in the file are indicated within parenthesis: objects are elements having child properties that in YAML parlance make up a mapping; string, boolean, number, integer types are leaf elements; properties of type array of a certain type represent sequences in the YAML naming. We recommend the usage of double quote " characters around string values (even though they are not required in YAML) just to avoid them to be interpreted as numbers in some cases.

Note: Use double quote characters around strings.

Many of the properties in the configuration file accept paths to files. Relative paths (those not starting with a / character) are actually relative to the working directory of torizoncore-builder (which is usually set when the alias to invoke the tool is set) and must be underneath it. Absolute paths (which start with a / character) are also possible but they must be specified from the perspective of the container where torizoncore-builder will run. Due to the complications this might cause, we discourage the use of absolute paths in the configuration file.

Warning: Do not use absolute paths in the configuration file.

Input Section

The input section defines the source artifact for customization. The definition can be provided in two different ways (mutually exclusive):

• easy-installer (object): With this property, the source for customization is a Toradex Easy Installer image, which, in turn, can be specified in three different ways (also mutually exclusive):
  • local (string): Specifies a local directory or tarball containing the image.
  • remote (string): Specifies an HTTP or HTTPS URL used to download the image. For integrity checks, the expected SHA-256 checksum of the file can be appended to the path part of the URL using a semicolon as separator, e.g.: http://artifactory.toradex.com/.../tezi-image.tar;sha256sum=ad523…da02?key=val1&key2=val2. Notice that the sha256sum parameter comes before the query parameters (if any). The name of the downloaded file will be determined from the last part of the path (the so called slug), if present. The parameter filename can be employed to force a specific name for the file, e.g.: http://artifactory.toradex.com/.../tezi-image.tar;filename=teziimage_new.zip?key=val1&key2=val2.
  • toradex-feed (object): Defines individual image attributes that will be used by the tool to download a standard image from the official feed server:
    • version (string): Software version in the form major.minor.patch (e.g. “5.2.0”).
    • release (string: nigthly/monthly/quarterly) Release type (to choose the feed used).
    • machine (string): One of the supported module types (ex: colibri-imx7, apalis-im6, etc.).
    • distro (string: torizon/torizon-rt/torizon-upstream/torizon-upstream-rt): Distribution type (at the moment we support regular kernel or PREEMPT_RT).
    • variant (string: torizon-core-docker/torizon-core-podman, optional): Variant of the torizon image.
    • build-number (string or number): Build number as a string or number, e.g. “252“ or 252.
    • build-date (string or number): Build date as a string or number, e.g. "20210408" or 20210408 – this property is not required when using a quarterly release.
• ostree (object): not available yet.

Customization Section

The customization section determines the modifications to be applied to the input to produce the desired output. It has the following structure:

• splash-screen (string, optional): Local file path of a PNG file used to generate the splash screen image.
• filesystem (array of strings, optional): List of directories to be merged with the input – the corresponding directory trees will be applied on top of the ones present in the input image, in the listed order. These directory trees can be created manually or as a result of customizing a live device and running the torizoncore-builder isolate command.
• device-tree (object, optional): This entry allows the configuration of a base device tree (describing the whole hardware setup of the target device) and of additional overlays (describing specific peripherals or configurations).
  • include-dirs (array of strings, optional): List of directories where to look for include files while compiling device tree source files (used both for full device trees and for overlays).
  • custom (string, optional): Path of a device tree source (dts) file that will be built and integrated with the image, replacing existing ones. Beware that when a custom device tree is set, all overlays possibly present in the base image are cleared.
  • overlays (object, optional):
    • clear (boolean, optional): Boolean indicating whether or not to ignore all the binary overlays (dtbo files) that may already be present in the base image. This property is useful only if no custom device tree was set. Default: false.
    • remove (array of strings, optional): List of specific binary overlays (only the base names of the dtbo files) to be ignored or not taken from the base image. This property is useful only if no custom device tree was set.
    • add (array of strings, optional): List of paths for device tree source files that will be compiled and added to the image. Those overlays will provide specific changes to the base device tree and will be applied in the order listed in the configuration file. The tool will test the overlay against the current device tree (DT) only if a DT has been set (via the custom property or predefined in the base image).
• kernel (object, optional): This property provides access to the facilities provided by the torizoncore-builder kernel command.
  • arguments (array of strings, optional): List of custom arguments to be passed to the kernel of the customized image.
  • modules (array of objects): List of kernel modules to be built from source and deployed with the image.
    • source-dir (string): Path to directory containing the kernel module source code. For further details, please refer to Building External Kernel Modules With Torizon.
    • autoload (boolean, optional): Boolean indicating whether the module should be automatically loaded upon system boot.

Output Section

The output section provides the details about the output artifact(s) to be produced by the torizoncore-builder build command. The section has this structure:

• ostree (object, optional): The configuration items that are specific to OSTree are kept under this property; they directly relate to the torizoncore-builder union command.
  • branch (string, optional): Name of the OSTree repository branch that will contain the commit with the changes applied by the tool.
  • commit-subject (string, optional): OSTree commit subject. Default: "TorizonCore Builder <timestamp>"
  • commit-body (string, optional): OSTree commit body message.
• easy-installer (object, optional): The configuration items specific to the Toradex Easy Installer image output are under this property.
  • local (string): Name of a local directory for the generated image.
  • name (string, optional): Name of the image as shown by the installer.
  • description (string, optional): Short description of the image (displayed in the image list screen of the installer).
  • licence (string, optional): Path of an HTML file with the image licence (displayed before installation when running interactively).
  • release-notes (string, optional): Path of an HTML file with the image release notes (accessible from the installer UI).
  • accept-licence (boolean, optional): Accept/Deny the licence. Lincence acceptance is required when autoinstall is enabled.
  • autoinstall (boolean, optional): Set the value of the autoinstall property on the image.json file.
  • autoreboot (boolean, optional): Add/Remove auto-reboot in the wrapup.sh file.
  • bundle (object, optional) – first form: Specifies a Docker compose file to be processed. The container images referenced by the compose file will be fetched during build time and a modified version of the compose file referencing the fetched image IDs will be stored in the output image.
    • compose-file (string): Path of a docker-compose file that will be included in the final image along with the required container images (i.e. with the images referenced by it). This will be considered only if the image variant is torizon-core-docker.
    • platform (string, optional): Platform for fetching multi-platform container images. If not specified, the tool will select between “linux/arm/v7” or “linux/arm64“ depending on the architecture of the input image.
    • username (string, optional): Docker login username to be used if accessing a private registry is required.
    • password (string, optional): Docker login password to be used if accessing a private registry is required.
    • registry (string, optional): Registry host name (e.g. hub.docker.com); this will be used only if username is also set.
  • bundle (object, optional) – second form:
    • dir (string): Path to directory containing the container images bundle to be combined with the installer image. Such bundle can be obtained by executing the torizoncore-builder bundle command beforehand.
  • provision (object, optional): Data for provisioning a device for secure updates are under this property.
    • mode (string): This string field defines the provisioning mode and can be set to one of the following three values:
      • disabled: Disable provisioning completely;
      • offline: With this mode, a device where the resulting image is installed will be ready to accept offline-updates produced by the Torizon Platform account from where the "shared data" came from;
      • online: With this mode, a device where the resulting image is installed will contain information allowing it to automatically register itself to the Torizon Platform (using credentials contained in the "online data" coming from a certain Torizon Platform account); besides it will also be able to accept offline-updates.
    • shared-data (string): Path to archive containing shared data; such an archive can be obtained from the Torizon Platform via the torizoncore-builder platform provisioning-data command. This property must be set both in offline and online modes.
    • online-data (string, optional): String containing sensitive data required for online provisioning (base64-encoded); such data can also be obtained from the Torizon Platform via the torizoncore-builder platform provisioning-data command or even directly from the UI. The property must be set only in online mode and it is recommended to set it by employing variable substitution whenever possible - this way, its value can be kept outside of version control (for example, one can keep the sensitive data as an environment variable in their CI system which would be passed to torizoncore-builder upon build time only).

Variable Substitution

In order to simplify the usage of the configuration file in cases where some values need to be set differently on each invocation of the build command, we provide a feature commonly offered by command-line shells called variable substitution. If such feature is enabled, all occurrences of $VAR in string values are replaced by the corresponding value of variable named VAR, which should be provided as part of the torizoncore-builder build command-line invocation; the notation $$ can be used for inserting a single literal $ sign within strings in the configuration file. Additionally, the following notations are also allowed:

• ${VAR}: Same as $VAR; when the variable is not set in command-line, torizoncore-builder will issue a warning and assume the variable value is the empty string.
• ${VAR:-default}: If VAR is not set or empty, use default as its value.
• ${VAR-default}: If VAR is not set, use default as its value.
• ${VAR:?error}: If VAR is not set or empty, issue the error message error and exit the tool.
• ${VAR?error}: If VAR is not set, issue the error message error and exit the tool.

In a configuration file, the variable substitution notation would be used like this:

# ...
      username: "${USERNAME?Please provide a username}"
      password: "${PASSWORD?Please provide a password}"
# ...
      commit-subject: "${SUBJECT-default commit subject}"

Command-line Interface

The build command has two different modes of operation. The first one allows the creation of a template configuration file that can then be modified to suit one's needs. In this case, usage is:

torizoncore-builder build [--file NAME] --create-template

$ torizoncore-builder build --file - --create-template

torizoncore-builder build [--file NAME] [--set VAR=VALUE] [--no-subst] [--force]

Version Controlling

In order to have reproducible builds, one should version control the inputs to the build command – in general, this entails keeping the configuration file and each of the files referenced by it under version control. Moreover, if the configuration file employs variable substitution, the same values should be passed as arguments to build to ensure the same results. The input image does not necessarily need to be version controlled by the user because Toradex already takes care of that – so if you reference a certain quarterly or monthly release, you know that the exact referenced version will be available to you over the Internet (at least for a period of time).

Device-tree (DT) and device-tree overlay (DTO) source files are a kind of input to the build command that may require especial treatment. If you create your own DT or DTO source files then you would keep them under version control as usual. However, if you want to use DT and/or DTO source files supplied by Toradex, then the recommended approach is to set up a Git submodule referencing our device-trees repo – this way, one could more easily keep in sync with upstream changes which is commonly desired.