Search by Tags

Build TorizonCore from Source With Yocto Project/OpenEmbedded

 

Article updated at 02 Dec 2021
Compare with Revision




Select the version of your OS from the tabs below. If you don't know the version you are using, run the command cat /etc/os-release or cat /etc/issue on the board.

Torizon 5.0.0

Introduction

TorizonCore is the base operating system in which Torizon is built. It is a minimal Yocto-based embedded Linux image and, even though customers may usually stick to application development on top of containers, sometimes they may need to customize the base image for their needs.

Note: If you just want to start using TorizonCore as-is with no customization in the original image, see the Quickstart Guide for information on how to install it on your Toradex Computer-on-Module.

Before trying to build TorizonCore using Yocto Project/Open Embedded, briefly read what TorizonCore Builder can do. It's a Toradex-provided tool that can help in customizing your TorizonCore image, without having to deal with Yocto.

The Toradex manifest repository contains a Repo manifest and setup scripts for the TorizonCore build system. If you want to modify, extend or port TorizonCore to a new hardware platform, this is the manifest repository to use.

The build system uses various components from the Yocto Project, most importantly the OpenEmbedded build system, the bitbake task executor and various application and BSP layers.

You have two options to build TorizonCore (and possibly any Yocto-based image):

  • Use a native distribution like Ubuntu to build TorizonCore.
    • Pros: has the advantage that you can easily understand what is going on.
    • Cons: you may be stuck on a certain Distribution Version which is supported by the Yocto Release where TorizonCore is built on; you need to install all dependencies by yourself in every new development machine.
  • Use a Docker container and build everything inside that container.
    • Pros: it has the advantage that it will work independently of the version of the host distribution; it works under Windows; the commands used inside the container are the exact same as if you would build on the native distribution.
    • Cons: it may seem to be more complicated at the first look.

You can build TorizonCore natively under your favorite Linux flavor. However, not all distribution version/Yocto version combos are allowed. If your distribution is not supported you will get a warning when running your first bitbake command. In such cases, it may happen that your current distribution can't build TorizonCore. You may consider trying to use a Docker container then.

This article complies to the Typographic Conventions for Torizon Documentation.

Common Prerequisites

The following information is something that you should determine before proceeding with either build option.

Manifest Branch

The manifest branch refers to the git branch of the manifest repository that will be used for setup. The available branches related to TorizonCore images are:

TorizonCore version Branch
5.x.y dunfell-5.x.y
0.0.0 master

These branches come from the official Toradex Manifest Repository.

Note: master is maintained to monitor changes in the external layers. Builds can be broken for a longer period of time. This branch is not recommended for productization.

Manifest File

The manifest file refers to the specific file in the manifest repository that will be used for setup. The available files related to Torizon images are:

  • torizoncore/default.xml
  • torizoncore/integration.xml
  • torizoncore/next.xml

It is recommended to use torizoncore/default.xml. The other two manifest files are used by Toradex to monitor future changes and may not be stable.

TorizonCore Distro

Within the greater Torizon platform, Toradex provides different variations of TorizonCore. In the OpenEmbedded community these are known as "Distros". For TorizonCore, Toradex provides the following Distros:

Distro Kernel Base Kernel Config
torizon Downstream kernel from SoC vendor (NXP) (default for iMX8* based modules) Default (without the PREEMPT_RT patch)
torizon-upstream Mainline kernel (default for iMX6/iMX7 based modules) Default (without the PREEMPT_RT patch)
torizon-rt Downstream kernel from SoC vendor (NXP) Fully preemptive (real-time Linux PREEMPT_RT patch)
torizon-upstream-rt Mainline kernel Fully preemptive (real-time Linux PREEMPT_RT patch)

Machine

Supported MACHINE targets (officially tested by Toradex):

Machine Name Corresponding Toradex Module
apalis-imx6 Apalis iMX6
apalis-imx8 Apalis iMX8
colibri-imx7-emmc Colibri iMX7D 1GB (equipped with eMMC flash)
colibri-imx6 Colibri iMX6
colibri-imx6ull-emmc Colibri iMX6ULL 1GB (equipped with eMMC flash) [1]
colibri-imx8x Colibri iMX8X V1.0C or newer
verdin-imx8mm Verdin iMX8M Mini
verdin-imx8mp Verdin iMX8M Plus
  • [1] Added on the 5.4.0-devel-202109 (September 2021 monthly pre-release)

TorizonCore Image

Supported image targets for bitbaking:

Image Description
torizon-core-docker Default image supported by Toradex

- Network Manager: NetworkManager
- Init system: systemd
- Container Engine: Docker
- OTA Framework: OSTree & Uptane
torizon-core-podman Experimental image using Podman instead of Docker

Same Features as above except Container Engine is Podman

Build Process

Choose from the tabs below to either do a native or a containerized build:

Prerequisites

You must have a host PC configured with the following prerequisites:

Download Metadata

To configure the scripts and download the build metadata, do:

$ mkdir ~/bin
$ PATH=~/bin:$PATH

$ curl http://commondatastorage.googleapis.com/git-repo-downloads/repo > ~/bin/repo
$ chmod a+x ~/bin/repo

In addition, make sure you have Git user name and e-mail configured:

$ git config --global user.email "you@example.com"
$ git config --global user.name "Your Name"

Run repo init to bring down the latest stable version of Repo. You must specify a URL for the manifest, which specifies the various repositories that will be placed within your working directory. Furthermore, a branch must be specified. Available manifest branches are listed in the above table.

To check out the latest TorizonCore release, including monthly pre-releases:

$ repo init -u https://git.toradex.com/toradex-manifest.git -b dunfell-5.x.y -m torizoncore/default.xml

Alternatively, if you want to work on a specific version, check out using tags. Find out available tags on the Git repository. See the example below for 5.6.0:

repo init -u https://git.toradex.com/toradex-manifest.git -b refs/tags/5.6.0 -m torizoncore/default.xml

Tip: For reproducible builds, you should at least lock in on a specific version. If reproducible builds are critical for your use case, consider fetching all source code and storing it on a local machine after the setup.

A successful initialization will end with a message stating that Repo is initialized in your working directory. Your client directory should now contain a .repo directory where files such as the manifest will be kept.

To pull down the metadata sources to your working directory from the repositories as specified in the repo manifest, run:

$ repo sync --no-clone-bundle

When downloading from behind a proxy (which is common in some corporate environments), it might be necessary to explicitly specify the proxy that is then used by repo:

$ export HTTP_PROXY=http://<proxy_user_id>:<proxy_password>@<proxy_server>:<proxy_port>
$ export HTTPS_PROXY=http://<proxy_user_id>:<proxy_password>@<proxy_server>:<proxy_port>

More rarely, Linux clients experience connectivity issues, getting stuck in the middle of downloads (typically during "Receiving objects"). It has been reported that tweaking the settings of the TCP/IP stack and using non-parallel commands can improve the situation. You need root access to modify the TCP setting:

$ sudo sysctl -w net.ipv4.tcp_window_scaling=0
$ repo sync -j1

Setup Environment

The default distribution (DISTRO) variable is automatically set based on the chosen MACHINE (either torizon or torizon-upstream). To choose another DISTRO, specify so in the below command, similar to MACHINE. For more information on available DISTROs please see the previous section TorizonCore Distro.

Setup the work environment by using the setup-environment script:

$ [MACHINE=<MACHINE>] source setup-environment [BUILDDIR]

If MACHINE is not provided, the script will list all possible machines and force one to be selected. For more information on available machines please consult the Machine list.

Start Building

To build the TorizonCore image:

The above setup script should properly prepare the environment with correct defaults. Additionally you'll want to modify the conf/local.conf file by adding ACCEPT_FSL_EULA="1".

$ bitbake torizon-core-docker

The TorizonCore Build Environment is a project that extends CROPS to build a TorizonCore image. CROPS is a Docker image for building Yocto based distros which is maintained by the Yocto Project.

Prerequisites

The following prerequisites are mandatory:

  • Linux / Windows host with Docker installed.

Additionally, the following is required on a Windows Machine only:

  • WSL2
  • A suitable WSL2 Linux Distribution (Ubuntu 18.04 was tested)

For more information on setting up WSL2 please see the section Install the Windows Subsystem for Linux (WSL2).

Basic Build Configuration

Linux Setup

Create a working directory where the Yocto build will be stored. For example:

$ cd ~
$ mkdir ~/yocto-workdir

Then run the following minimal docker run command to bring up the build container. Don't forget to substitute <machine> and <image> based on the module and TorizonCore image you want to build:

$ docker run --rm -it --name=crops -v ~/yocto-workdir:/workdir --workdir=/workdir -e MACHINE=<machine> -e IMAGE=<image> torizon/crops:dunfell-5.x.y startup-tdx.sh

Windows Setup

For Windows due to the heavy Linux filesystem dependencies of OpenEmbedded it will be easier to setup a working directory via WSL2. This working directory will be used to store the resulting build files on your host machine.

First, open a Windows command prompt and execute wsl. This should put you in a prompt "inside" your installed WSL2 Linux distribution. From here you can setup a folder where you want to store the build files. For example, if you want to set this up inside the home directory then do as follows:

$ cd
$ explorer.exe .

This will open up your WSL2 home directory with Windows File Explorer. From here you can copy the exact file path for later use (i.e. \wsl$\Ubuntu\home\coj)

Once you've setup your desired working directory you can now run the following minimal docker run command via Windows command prompt (don't forget to substitute <machine> and <image> based on the module and TorizonCore image you want to build):

$ docker run --rm -it --name=crops -v \wsl$\Ubuntu\home\coj:/workdir --workdir=/workdir -e MACHINE=<machine> -e IMAGE=<image> torizon/crops:dunfell-5.x.y startup-tdx.sh

Build Information

For either Windows or Linux, once you've ran the appropriate command, this will then start to setup and build a Torizon image of IMAGE for module MACHINE. For more information on valid options for these variables please consult the Common Prerequisites at the beginning of this article.

Note: Before the build begins you will be prompted to accept the NXP/Freescale EULA which is required for some packages. Make sure to accept this prompt before leaving to get coffee!

Once the build is complete you can view the build output from either in the container or your host machine's working directory that was setup up prior.

Container Arguments

The above docker run commands were just minimal commands required to get the container to setup and start a build. However, there are additional arguments that can be passed to either further configure the build or change its behavior. These arguments are passed into the container as environment variables via the -e flag of docker run:

  • -e MACHINE=<machine>: This is the target machine that the image will be built for. This is required and will not default to any value if it is not set. For a list of all compatible machines see the Machine list.
  • -e DISTRO=<distro>: This is the Torizon DISTRO that will be built. This will default to either torizon or torizon-upstream depending on the chosen MACHINE if this variable is not explicitly set. For other valid options please see the TorizonCore Distro list.
  • -e BRANCH=<branch>: This is the manifest repository branch. If not passed it defaults to dunfell-5.x.y. For a list of other options see the Manifest Branch.
  • -e MANIFEST=<manifest>: This is the specific manifest file that is to be used. If not passed it defaults to torizoncore/default.xml. For other options see the Manifest File.
  • -e IMAGE=<IMAGE>: This is the Yocto build recipe that is to be built. If not passed the container will simply setup the build environment and then run a shell inside the without starting any build, for if you want to do further configurations before building. For a list of valid image recipes see the TorizonCore Images list.

Customization

The Yocto Project can be quite defying and hard to use. In the following documentation, we cover some basic functionality that is likely you will have to go through during your development:

It does not replace the official Yocto Project documentation, though. At the end of the article Build a Reference Image with Yocto Project, we provide links to it, and you will most likely need to study it to some degree.

Custom Layers

On TorizonCore all your custom layers must be version controlled by Git, due to how we include layer revision information with OSTree. More details are provided in the aforementioned article Hello World integration into OpenEmbedded.

Deploy to Torizon OTA

This is a feature available on prime-tiers of the Torizon Platform Services. If you want to learn more about the Torizon Platform Services pricing and plans, please contact us.

Request access to evaluate the Torizon Platform Services premium features

To be able to deploy your custom image to Torizon OTA, you need to meet the following criteria:

  • Maintain the OSTree and Aktualizr references unchanged at your Yocto Build setup. They are responsible for the system update and OTA process.

Then, all you need from Yocto is the resulting image in the Toradex Easy Installer format, which you will use with the TorizonCore Builder Tool to unpack the image and then push it to Torizon OTA.

Read the article Signing and Pushing TorizonCore Images to Torizon OTA for more details about this process.

Keep in mind, though, that once you build your own TorizonCore image you are fully responsible for regularly building and deploying updates as well.

Torizon 4.0.0

Introduction

TorizonCore is the base operating system in which Torizon is built. It is a minimal Yocto-based embedded Linux image and, even though customers may usually stick to application development on top of containers, sometimes they may need to customize the base image for their needs.

Note: If you just want to start using TorizonCore as-is with no customization in the original image, see the Quickstart Guide for information on how to install it on your Toradex Computer-on-Module.

The toradex-torizon-manifest project on Github repository contains a Repo manifest and setup scripts for the TorizonCore build system. If you want to modify, extend or port TorizonCore to a new hardware platform, this is the manifest repository to use.

The build system uses various components from the Yocto Project, most importantly the OpenEmbedded build system, the bitbake task executor and various application and BSP layers.

Toradex TorizonCore is based on Linux microPlatform from foundries.io and therefore requires the meta-lmp layer.

You have two options to build TorizonCore (and possibly any Yocto-based image):

  • Use a native distribution like Ubuntu to build TorizonCore. This is the way documented in this article.
    • Pros: has the advantage that you can easily understand what is going on.
    • Cons: you may be stuck on a certain Distribution Version which is supported by the Yocto Release where TorizonCore is built on; you need to install all dependencies by yourself in every new development machine.
  • Use a Docker container and build everything inside that container.
    • Pros: it has the advantage that it will work independently of the version of the host distribution; it works under Windows; the commands used inside the container are the exact same as if you would build on the native distribution.
    • Cons: it may seem to be more complicated at the first look.

This article complies to the Typographic Conventions for Torizon Documentation.

You can build TorizonCore natively under your favorite Linux flavor. However, not all distribution version/Yocto version combos are allowed. If your distribution is not supported you will get a warning when running your first bitbake command. In such cases, it may happen that your current distribution can't build TorizonCore. We recommend following the "Containerized TorizonCore Build" article if this happens.

Prerequisites

You must have a host PC configured with the following prerequisites:

Download Metadata

To configure the scripts and download the build metadata, do:

$ mkdir ~/bin
$ PATH=~/bin:$PATH

$ curl http://commondatastorage.googleapis.com/git-repo-downloads/repo > ~/bin/repo
$ chmod a+x ~/bin/repo

In addition, make sure you have Git user name and e-mail configured:

$ git config --global user.email "you@example.com"
$ git config --global user.name "Your Name"

Run repo init to bring down the latest stable version of Repo. You must specify a URL for the manifest, which specifies the various repositories that will be placed within your working directory. Furthermore, a branch must be specified.

TorizonCore version Branch
4.0.0 zeus
0.0.0 master

Note: master is maintained to monitor changes in the external layers. Builds can be broken for a longer period of time. This branch is not recommended for productization.

To check out the latest TorizonCore release:

$ repo init -u https://github.com/toradex/toradex-torizon-manifest -b zeus

A successful initialization will end with a message stating that Repo is initialized in your working directory. Your client directory should now contain a .repo directory where files such as the manifest will be kept.

To pull down the metadata sources to your working directory from the repositories as specified in the repo manifest, run:

$ repo sync --no-clone-bundle

When downloading from behind a proxy (which is common in some corporate environments), it might be necessary to explicitly specify the proxy that is then used by repo:

$ export HTTP_PROXY=http://<proxy_user_id>:<proxy_password>@<proxy_server>:<proxy_port>
$ export HTTPS_PROXY=http://<proxy_user_id>:<proxy_password>@<proxy_server>:<proxy_port>

More rarely, Linux clients experience connectivity issues, getting stuck in the middle of downloads (typically during "Receiving objects"). It has been reported that tweaking the settings of the TCP/IP stack and using non-parallel commands can improve the situation. You need root access to modify the TCP setting:

$ sudo sysctl -w net.ipv4.tcp_window_scaling=0
$ repo sync -j1

Setup Environment

Supported MACHINE targets (officially tested by Toradex):

  • colibri-imx7 - Colibri iMX7 (eMMC & raw NAND*)
  • colibri-imx6 - Colibri iMX6
  • colibri-imx6ull - Colibri iMX6ULL*
  • colibri-imx8x - Colibri iMX8X V1.0C or newer
  • colibri-imx8x-v10b - Colibri iMX8X V1.0B
  • apalis-imx6 - Apalis iMX6
  • apalis-imx8 - Apalis iMX8
  • apalis-imx8x - Apalis iMX8X
  • verdin-imx8mm - Verdin iMX8MM

Note: Raw NAND-based modules are currently supported only by experimental releases and they may not be supported in the stable versions of TorizonCore.

Note: Apalis iMX8X is phased out, and it is not available for purchase anymore. The latest supported BSP and TorizonCore version is 5.4.0.

Supported image targets:

  • torizon-core-docker
  • torizon-core-podman

The default distribution (DISTRO) variable is automatically set to torizon, which is provided by the meta-toradex-torizon layer.

Setup the work environment by using the setup-environment script:

$ [MACHINE=<MACHINE>] source setup-environment [BUILDDIR]

If MACHINE is not provided, the script will list all possible machines and force one to be selected.

Start Building

To build the TorizonCore image:

The above setup script should properly prepare the environment with correct defaults. Additionally you'll want to modify the local.conf file by adding ACCEPT_FSL_EULA="1".

$ bitbake torizon-core-docker