Introduction

Since SoC vendors in the ARM world implement very diverse ways to boot their platforms, system developers need to deploy highly hardware-specific firmware images rather than generic operating systems. In this regard, the lack of a standardized boot flow as found in Intel-based PCs/laptops (e.g. BIOS and more recently UEFI) or in the 64-bit ARM server market (e.g. SBSA and SBBR) has made it unfeasible for generic OS distributors such as SUSE, Redhat or Fedora to support every available platform reliably and consistently. As a result, the tendency has been for hardware vendors to create their own forked OS distributions (e.g. Raspbian, Bananian, Parabuntu), thus bounding the user to the vendor's software. Distro Boot is an attempt to standardize a way for distributions to boot in a board-agnostic manner. Distro Boot is actually an informal name that refers to the Generic Distro Configuration Concept that U-Boot developers came up with around 2014. When Distro Boot is enabled in U-Boot, you can boot any supported distribution for your platform simply by placing an image in a conveniently partitioned removable device with the corresponding boot configuration file. This article describes the procedure to boot images through the Distro Boot mechanism.

Note: While Distro Boot is a simple standard, it is currently only supported by U-Boot. It also requires distributions to supply and maintain specific configuration files not standardized in the PC/laptop world. In 2018 a new standardization proposal for embedded booting backed by ARM is gaining popularity. The Embedded Base Boot Requirements (EBBR) specifies a UEFI booting model that attempts to unify the boot interface of embedded systems with the well established one in desktop platforms.

Support on the Embedded Linux BSP

Toradex does not support distro booting from external media, such as USB and SD Card. One of the reasons is that it is very hard to maintain such a feature, especially on the newer NXP i.MX 8/8X SoCs with SECO/SCFW.

While such a feature may or may not work on specific SoMs and specific BSP releases, it is not guaranteed that it will always work as we don't maintain it. Keep this in mind while developing your product and avoid depending on such a feature, if possible.

Nevertheless, if you still want to try it at your own risk, read the article Load Easy Installer From External Media (SD Card/USB Stick). It is specific for distro booting Toradex Easy Installer, but you may find insights if you plan to distro boot something else.

Requirements

In order to take advantage of the Distro Boot features you will need:

1. A running U-Boot that is Distro Boot compatible. The Embedded Linux BSP supported by Toradex provides U-Boot and its default environment compliant with Distro Boot since version 2.8b4. Therefore, at least this version of our BSP is needed to use the Distro Boot features (older versions might work, but you may need to adapt some settings). You can use our Toradex Easy Installer to install the latest image, that contains a compatible U-Boot.

2. An image of the distribution that you want to boot. It obviously has to support the Toradex platform you are using and it has to adhere to the Distro Boot concept. Some generic distributions do not fully support all of our platforms or aren't Distro Boot compliant, so you are at your own risk.

Note: Toradex doesn't support either encourage the use of any non-partner distribution. Please carefully evaluate the maintenance efforts of using a distribution other than the one(s) supported by Toradex.

3. A boot configuration file named extlinux.conf or a U-Boot-specific script named boot.scr or boot.scr.uimg

4. A removable storage media such as an SD card or a USB stick formatted and partitioned as explained below

Understanding the Boot Sequence

By default, U-Boot (with Distro Boot feature enabled) will scan through all devices in search of a boot configuration file called extlinux.conf or, in its absence, a U-Boot-specific script called boot.scr or boot.scr.uimg. The following devices are scanned in order:

1. External SD card.
2. Internal flash memory (raw NAND or eMMC).
3. External USB storage.
4. External server whose location is fetched from a DCHP request at the address provided by the ${serverip} environment variable.

For each of these devices, U-Boot will search in the first partition marked with a bootable flag (via the MBR bootable flag, or GPT legacy_bios_bootable attribute). In its absence, it will fall back to looking at the first valid partition. The Disto Boot recognizes FAT and ext2/3/4 filesystems. In this regard, you have a lot of flexibility when preparing your media for Distro Boot. Inside the partition, U-Boot will further look for the extlinux.conf file in two directories: /extlinux/ and /boot/extlinux/. For the boot.scr and boot.scr.uimg, U-Boot will look in / and /boot/. You can override this default by setting the boot_prefixes U-Boot variable.

Once found, U-Boot will either execute a specific boot command that will boot the system according to the extlinux.conf or simply run the commands contained in boot.scr/boot.scr.uimg. The next section describes how these boot configuration files are generated.

Boot Configuration Files

As mentioned above, there are two types of configuration files available in a Distro Boot compliant system. extlinux.conf is a configuration file typically used in the ubiquitous Syslinux/Extlinux bootloaders that predate the x86_64 era. Most GNU/Linux distributions are familiar with it and likely have tools to automatically create them. The downside is that they are not very flexible in terms of how to boot images (e.g. they expect your kernel to be in a zImage format. Any other format such as the convenient FIT Images -see also here- will not work). On the other hand, you can provide a U-Boot specific boot.scr or boot.scr.uimg. These scripts leave you absolute control of the U-Boot environment (e.g. you can modify device trees, manually load images to specific addresses, etc.), but they are not used outside the U-Boot world. This section describes both approaches.

extlinux.conf

After U-Boot has found extlinux.conf it executes the sysboot command on it, which parses the file and boots the system accordingly. This command is a rough analogy to what the Syslinux/Extlinux bootloader uses to boot Linux systems. Here is an example from an Arch Linux distribution.

 boot/extlinux/extlinux.conf
ui menu.c32

menu autoboot Arch Boot. Automatic boot in # second{,s}. Press a key for options.
menu title Arch Boot Options.
menu hidden

timeout 50

default Arch

label Arch
    kernel /zImage
    append root=/dev/mmcblk0p2 rw rootfstype=ext4 rootwait consoleblank=0 no_console_suspend=1 console=ttymxc0,115200n8
    fdtdir /dtbs
    initrd /initramfs-linux.img

As you can see, the symbols are self-explanatory. The ui menu.c32 directive acts as an including for a menu module used in Syslinux. The important part here, though, is the label with the kernel image, the command line arguments, the device tree path, and the initramfs path. The paths used here refer to the same device and partition where the extlinux.conf is located. You can also use relative paths from the directory where extlinux.conf is located, as exemplified in the U-Boot documentation.

Note: For a comprehensive list of all possible directives you can use in extlinux.conf files see the Syslinux and the Comboot wikis.

boot.scr/boot.scr.uimg

After U-Boot has found a boot.scr/boot.scr.uimg (in absence of an extlinux.conf), it will simply execute its contents in the U-Boot shell. This means that you can (and should only) use any U-Boot command to customize your boot process. This is especially convenient if you want to circumvent the limitations of the extlinux.conf approach. Images provided in the FIT format may make it more evident since they contain the kernel, device trees, and rootfs (as a squashfs file) bundled together. Since the extlinux.conf doesn't provide a means to boot such images, you also need to ship a boot.scr that can be copied together with the FIT image in the first partition of the bootable media. You might also want to pass custom kernel parameters to the installer or modify the device trees. Refer to the tab Custom Boot in our Toradex Easy Installer page for an example script.

In order to make a boot.scr file, first create a file named boot.cmd with your desired U-Boot commands. For instance:

 boot.cmd
setenv bootargs console=ttymxc0,115200 quiet video=mxcfb0:dev=hdmi,640x480@60,if=RGB24 video=mxcfb1:dev=lcd,640x480@60,if=RGB666 rootfstype=squashfs root=/dev/ram autoinstall ${teziargs}

# Reenable fdt relocation since in place fdt edits corrupt the ramdisk
# in a FIT image...
setenv fdt_high

# Load FIT image from location as detected by distroboot
load ${devtype} ${devnum}:${distro_bootpart} ${ramdisk_addr_r} ${prefix}tezi.itb

bootm ${ramdisk_addr_r}

Now use the mkimage utility to create the script file as follows:

mkimage -A arm -O linux -T script -C none -a 0 -e 0 -n "Distro Boot Script" -d boot.cmd boot.scr

Your script file is now ready. Add it into the boot-flagged FAT or ext2/3/4 partition in your external media (or in the first partition if you are not using GPT flags) together with our tezi.itb FIT image.

Preparing the Media

The boot configuration files detailed in Configuration Files mainly determine how you will prepare your Distro Boot ready media. The minimal requirement is to format a single FAT or ext2/3/4 partition.

For a general-purpose Linux distribution

The standard partitioning scheme for general-purpose distributions is to set a boot partition of a few hundred megabytes (~500 MB) for the boot files and kernel, and an ext4 partition with the rest of the media size (1-2 GB should suffice) for the rootfs. Refer to Boot From an SD Card / USB stick / SATA drive for detailed steps on how to create such partitioning in removable media. You will need an image package that contains the kernel in zImage format, the devicetrees (usually .dtb files), and the initramfs files, as well as the rootfs. Refer to the documentation of your distribution for details on how to burn your image into external media. Usually copying the files to your boot partition should be enough. Remember to add your extlinux.conf in the /extlinux or the /boot/extlinux directory.

Booting

• Booting in BSP 5.x
• Predefined Boot Commands in BSP 5.x
• One Time Boot in BSP 3.x
• Distro Boot by Default in BSP 3.x

Colibri iMX7 # boot

Colibri iMX7 # run bootcmd_usb0
starting USB...
Bus usb@30b10000: USB EHCI 1.00
Bus usb@30b20000: USB EHCI 1.00
scanning bus usb@30b10000 for devices... 2 USB Device(s) found
scanning bus usb@30b20000 for devices... 2 USB Device(s) found
scanning usb for storage devices... 1 Storage Device(s) found
Device 0: Vendor: Verbatim Rev: 8.07 Prod: STORE N GO
Type: Removable Hard Disk
Capacity: 14910.0 MB = 14.5 GB (30535680 x 512)
... is now current device
Scanning usb 0:1...
Found U-Boot script /boot.scr
4807 bytes read in 4 ms (1.1 MiB/s)
## Executing script at 87000000
Loading DeviceTree: imx7d-colibri-emmc-eval-v3.dtb
63547 bytes read in 7 ms (8.7 MiB/s)
6696896 bytes read in 421 ms (15.2 MiB/s)
Bootargs: root=PARTUUID=aa182840-02 ro rootwait console=tty1 console=ttymxc0,115200n8 consoleblank=0
Kernel image @ 0x81000000 [ 0x000000 - 0x662fc0 ]
## Flattened Device Tree blob at 82000000
Booting using the fdt blob at 0x82000000
Loading Device Tree to 8ffcd000, end 8fffffff ... OK
Starting kernel ...
...

setenv fdtfile ${soc}-colibri-${fdt_board}.dtb && run distro_bootcmd

Colibri iMX7 # env print bootcmd
bootcmd=run ubiboot; setenv fdtfile ${soc}-colibri-${fdt_board}.dtb && run distro_bootcmd;
Colibri iMX7 # env set bootcmd 'setenv fdtfile ${soc}-colibri-${fdt_board}.dtb && run distro_bootcmd; run ubiboot;'
Colibri iMX7 # env save
Saving Environment to NAND...
Erasing NAND...
Erasing at 0x380000 -- 100% complete.
Writing to NAND... OK

Example: Preparing a Distro Boot Media for Archlinux

This section provides an example of how to prepare a media compliant with Distro Boot. The Linux distribution chosen for the example is Arch Linux ARM. Notice that instructions may vary among distributions and how they are provided.

Note: Toradex doesn't support Arch Linux ARM as its Embedded Linux BSP, neither recommends its usage with Toradex Computer on Modules nor any other Linux Distributions that may be provided for the ARM architecture. Please carefully evaluate the maintenance efforts of using a distribution other than the one(s) supported by Toradex.

Download a Linux Distribution

On your host PC, create a folder and download the Arch Linux ARM release from the Arch Linux ARM Downloads Section. Make sure you select an image compatible with the SoC architecture, e.g. for Colibri iMX7 which is equipped with the NXP i.MX7 SoC, choose any of the ARMv7 releases.

mkdir distro_boot && cd distro_boot
wget http://os.archlinuxarm.org/os/ArchLinuxARM-armv7-latest.tar.gz

Create two directories where you will mount the boot and rootfs partitions of the media:

mkdir boot rootfs

Prepare the Boot Media

Format the media with a primary partition formatted as FAT and size of 100MB, and a secondary partition formatted as EXT4 and using all the space left in the media device. You can use a GUI tool such as Gparted or do it from command-line using fdisk and mkfs, as described below:

Create partition table:

fdisk /dev/<path-to-media>
# For instance: fdisk /dev/mmcblk0
# Press "o" to wipe the current partition table,
# Press "n" to create a new partition, "1" to make it the first partition, "ENTER" to accept the default first sector, and type "+100M" to make it have 100MB.
# Press "t" and then "c" to make the first partition FAT formatted.
# Press "n" to create a new partition, "2" to make it the first partition, "ENTER" to accept the default first sector, and "ENTER" to accept the default last sector.
# Press "w" to save and exit.

Create filesystems:

mkfs.vfat /dev/<path-to-media-first-partition>
mkfs.ext4 /dev/<path-to-media-second-partition>
# For instance
mkfs.vfat /dev/mmcblk0p1
mkfs.ext4 /dev/mmcblk0p2

Mount the filesystems into the previously created directories:

mount /dev/<path-to-media-first-partition> boot
mount /dev/<path-to-media-second-partition> rootfs
# For instance
mount /dev/mmcblk0p1 boot
mount /dev/mmcblk0p2 rootfs

Extract the downloaded rootfs:

bsdtar -xpf ArchLinuxARM-armv7-latest.tar.gz -C rootfs
sync

Move the boot files to the first partition:

mv rootfs/boot/* boot

Create Distro Boot Configuration File

Create a directory extlinux and an extlinux.conf file in the boot partition. You may have to tweak the append parameter, which passes command-line options to the Linux kernel. See the bootparam documentation for more information, or inspect the default values set in the Toradex BSP by booting the board and using dmesg, for instance for Colibri iMX7D 512MB V1.1C with the Toradex Embedded Linux BSP 2.8b3:

The Angstrom Distribution colibri-imx7 ttymxc0
Angstrom v2017.12 - Kernel
Colibri-iMX7_LXDE-Image 2.8b3.111 20180627
colibri-imx7 login: root
root@colibri-imx7:~# dmesg | grep "Kernel command line"
[    0.000000] Kernel command line: user_debug=30 ubi.mtd=ubi root=ubi0:rootfs rw rootfstype=ubifs ubi.fm_autoconvert=1 console=tty1 console=ttymxc0,115200n8 consoleblank=0 video=mxsfb:640x480M-16@60

Create the file:

 boot/extlinux/extlinux.conf
ui menu.c32

menu autoboot Arch Boot. Automatic boot in # second{,s}. Press a key for options.
menu title Arch Boot Options.
menu hidden

timeout 50

default Arch

label Arch
    kernel /zImage
    append root=/dev/mmcblk0p2 rw rootfstype=ext4 rootwait consoleblank=0 no_console_suspend=1 console=ttymxc0,115200n8
    fdtdir /dtbs
    initrd /initramfs-linux.img

Note: In this example, the kernel, device-tree, and initramfs being used are the default ones that came with the distribution. You may choose to replace them with custom ones. See Build U-Boot and Linux Kernel from Source Code and Initramfs and tmpfs for instructions on how to build from source.

Boot on the Toradex Board

Unmount, remove the media device from your computer and plug it into the Toradex carrier board:

umount boot rootfs

Power on the board and issue the command:

run distro_bootcmd