Search by Tags

Torizon IDE Backend Command-line Interface

 

Article updated at 03 Mar 2022
Compare with Revision

Introduction

The features of the Torizon IDE Backend that are used to implement the functionality of the Torizon extensions for Visual Studio 2019 and Visual Studio Code can be accessed using a command-line interface (CLI).

This interface makes available most of the functions exposed via REST API in a way that is easy to integrate into scripts used for tests or automated builds.

The source code of the IDE backend and command-line interface (CLI) is open source, available on github.com/toradex/moses. The IDE backend name is Moses. The CLI tool is named tdskt.

This article complies to the Typographic Conventions for the Toradex Documentation.

Intended Audience

While this article is useful for anyone doing development with the Visual Studio Extension For Torizon and the Visual Studio Code Extension for Torizon, it is most useful for those interested in setting up a CI/CD environment.

Why Integrate the IDE Backend in a CI/CD Pipeline

Development in an IDE usually happens in an iterative way. The developers change, deploy and debug the code until they are happy with the results. Our IDE extensions are designed to support this approach and the usage of an interactive debugger.

At some point, usually after the experimentation and prototyping phase, code is moved into production, and CI/CD tools are used to build, test, and release it.

Using the IDE backend CLI in this scenario makes sense because you:

  • Keep the application configuration in a single place and consequently reduce duplication of source code.
  • Re-use the IDE features instead of creating something from scratch.
  • Can automate the deployment of new application versions and perform tests on devices before a release.
  • Make sure that the application you test and ship is built in the same way you built it locally on your machine during interactive debugging.

How to Integrate the IDE Backend in a CI/CD Pipeline

Usually, building in a CI setup happens by invoking a script or defining a sequence of operations. For a Torizon application, developers can use the IDE built-in project system of Visual Studio 2019 (via ms-build) or execute build tools directly (in VS Code, depending on the language used).

For C/C++ applications, tdskt can be used to start the SDK container with the toolchain required to build the code.

When the application is ready to be shipped, its binaries can be copied to the /work subfolder of the application configuration directory. This operation is automated in the IDE extensions, but can be done using:

  • dotnet publish for .NET apps.
  • rsync for Python apps.
  • Make/CMake install for C/C++ apps.

At this point, the IDE backend command-line interface (CLI) can be used to:

  • Build images.
  • Deploy to test devices and perform additional testing.
  • Publish images on a docker registry and release them as updates with the Torizon Remote Updates.

CI/CD Pipeline Examples

We want to know more about your needs. If you are interested in a reference implementation of a CI/CD pipeline with the IDE backend CLI, let us know in the Toradex Community. Our decisions are driven by customer feedback, and you can help us understand what do you need.

Prerequisites

Installation of the ide-backend

The command-line interface is part of the IDE-backend and it is automatically installed when the VS2019 or VS Code extensions are installed.

For VS Code: - Extensions are installed under $HOME/.vscode/extension by default. This may be different if you use an insiders build or if you pass the --extensions-dir parameter when starting VS Code. - The extension folder is named toradex.torizon+<version> where the version part depends on the type (stable vs early access) and release of the extension. - The executables are found inside the extension directory. The folder is named moses-linux with the executables of the backend moses and CLI tool tdskt.

For VS 2019: - The extension will be installed under %LocalAppData%\Microsoft\VisualStudio\<visual studio version>\Extensions\Toradex\Toradex Torizon C\C++ Support\<extension version> - Inside this Folder you'll find a folder named Server - The executables are found inside the extension directory. The folder is named moses-windows and the executables are moses.exe for the backend and tdskt.exe for the CLI tool.

Using the command line interface

Launching the tool with no arguments shows the available commands:

$ ./tdskt
usage: tdskt [-h] [-p] [-b BACKEND] {devices,device,eulas,eula,platforms,platform,application,detect,create,load,pull,enableemulation,version,dockerversion} ...

On windows the executable name is tdskt.exe, this is the only difference between the Linux and Windows versions of the tool, so in the following samples we will use the Linux file name, but parameters and output will match those of the Windows release.

$ ./tdskt.exe
usage: tdskt [-h] [-p] [-b BACKEND] {devices,device,eulas,eula,platforms,platform,application,detect,create,load,pull,enableemulation,version,dockerversion} ...

To better understand the meaning of some of those commands and related objects, please refer to the Torizon IDE Backend Architecture and Internals documentation.

Before attempting to run commands you need to start the ide-backend main executable (moses or Linux, moses.exe on Windows) and keep it running in the background.

By default, the command line interface tries to connect to a running instance of the IDE backend on port 5000. You can pass a different host:port combination using the --backend command line parameter.

Some long-running commands (ex: building or deploying an application) return progress information. By default, this information is not printed on the console to keep the output simple for people parsing that in their scripts, but it can be enabled by passing the -p command-line option before the selected command.

Working with devices

Devices are identified using their unique ID (serial number for Toradex devices) and they can be used only after they have been detected by the back-end. Detecting a new device means collecting information about it, but also configuring some services in a way that will allow the ide-backend to operate with the device in an automated way.

Detecting a device

The detect subcommand may be used to detect a new device or refresh a device configuration after an OS-reinstall.

$ ./tdskt detect                                                                                                      ✔
usage: tdskt detect [-h] [--network] [--uart] target username password
tdskt detect: error: the following arguments are required: target, username, password

A device can be detected using the console on its serial port or via SSH. After first detection all the communication with the device will happen via ssh. To detect a device using serial port you need to specify the --uart command line switch and provide a name of the local port used for connection and the credential of a user that is part of the "sudoers" group.

For example, to detect a device on Windows using the default credentials you can use:

$ ./tdskt detect --uart COM1: torizon torizon
Attepting to detect serial device, this may take a couple of minutes, after detection the device will reboot.
Check that your device is powered on an connected to your PC via serial/USB cable.
At the end of detection your device will reboot.

On Linux (notice that serial ports are referenced in a different way):

$ ./tdskt detect --uart /dev/ttyUSB1 torizon torizon
Attepting to detect serial device, this may take a couple of minutes, after detection the device will reboot.
Check that your device is powered on an connected to your PC via serial/USB cable.
At the end of detection your device will reboot.

If detection is successful, some information is shown:

Device successfully detected.
Property           Value
id                 06827787
name               Toradex Verdin iMX8M Mini WB on Verdin Development Board(06827787)
model              0055
hwrev              V1.1A
kernelversion      #1-TorizonCore SMP PREEMPT Mon Jul 19 18:19:07 UTC 2021
kernelrelease      5.4.129-5.4.0-devel+git.9443377ef940
distroversion      TorizonCore 5.4.0-devel-202108+build.15 (dunfell)
hostname           verdin-imx8mm-06827787.local
username           torizon
homefolder         /home/torizon
runningtorizon     True
cpu_architecture   aarch64
model_description  Verdin iMX8M Mini Quad 2GB WB IT

Information about devices is stored in the user's home folder in a subdirectory named ".moses".

List available devices

The "devices" command can be used to show a list of available devices.

$ ./tdskt devices                                                                                                     ✔
      Id  Name                                                                         Model  HW release
06772320  Toradex Apalis iMX8QM V1.1 on Apalis Evaluation Board(06772320)               0037  V1.1C
03112636  Toradex Colibri iMX7D 1GB (eMMC) on Colibri Evaluation Board V3(03112636)     0039  V1.1A
06827787  Toradex Verdin iMX8M Mini WB on Verdin Development Board(06827787)            0055  V1.1A

You can get more detailed information using the "info" subcommand of the "device" command:

Property           Value
id                 06827787
name               Toradex Verdin iMX8M Mini WB on Verdin Development Board(06827787)
model              0055
hwrev              V1.1A
kernelversion      #1-TorizonCore SMP PREEMPT Mon Jul 19 18:19:07 UTC 2021
kernelrelease      5.4.129-5.4.0-devel+git.9443377ef940
distroversion      TorizonCore 5.4.0-devel-202108+build.15 (dunfell)
hostname           verdin-imx8mm-06827787.local
username           torizon
homefolder         /home/torizon
runningtorizon     True
cpu_architecture   aarch64
model_description  Verdin iMX8M Mini Quad 2GB WB IT

The info command works by reading information stored on the local machine, so it can't be used to check if a device is actually online.

Collect dynamic information from devices

The ide-backend can be used also to get some realtime information from the device. To be able to collect this kind of information the device must be turned on and connected to the network.

Resources (processes, memory, storage)

If you want to have a list of the processes currently running on a device you can use the "ps" subcommand:

$ ./tdskt device 06827787 ps                                                                                                                                      ✔
  PID    PPID  User      Time        Nice  State    Args
    1       0  root      00:00:02       0  Ss       /sbin/init
    2       0  root      00:00:00       0  S        [kthreadd]
    3       2  root      00:00:00     -20  I<       [rcu_gp]
    4       2  root      00:00:00     -20  I<       [rcu_par_gp]
...
 1221       2  root      00:00:00       0  I        [kworker/2:0-events]
 1223    1220  torizon   00:00:00       0  R        sshd: torizon@notty
 1224    1223  torizon   00:00:00       0  Rs       ps -A -o pid,ppid,user,time,nice,stat,args

If you want to keep memory usage under control you may use the "mem" subcommand that wil return total, available and free memory:

$ ./tdskt device 06827787 mem                                                                                                                                     ✔
Property          Value
total       2.02504e+06
available   1.61596e+06
free        1.45358e+06

If you are interested in mass-storage usage on the target device you can collect information using the "storage" subcommand:

$ ./tdskt device 06827787 storage                                                                                                                                 ✔
Mount poinf     FS                              Size    Available
/dev            devtmpfs                      743912       743912
/dev/shm        tmpfs                        1012516      1012516
/run            tmpfs                        1012516      1002928
/run/user/1000  tmpfs                         202500       202500
/sys/fs/cgroup  tmpfs                        1012516      1012516
/sysroot        /dev/disk/by-label/otaroot  15226800     10146152
/tmp            tmpfs                        1012516      1012504
/var/volatile   tmpfs                        1012516      1012512

Container images

For example, it's possible to collect a list of all the container images currently stored on the device using the images subcommand:

$ ./tdskt device 06827787 images                                                                                    2 ↵
Id                                                                       Tags                                                                                                            Created
sha256:6cee1083be7909fdcd76d4d9a612189aea23b8f22fc947d86c238f126d670c61  ['aspnetwebapp_arm64v8-aspdotnet-50_bullseye_debug_ed61174b-2f6a-46b4-ab2d-9039cde48f57:latest']                2021-07-16T12:00:10.767823355Z
sha256:00e2e9440c08ff1311d8f5dae8abc032ed8f9cc1082a4940919f572ef53c8784  ['torizon/torizon-provisioner:0.0.9']                                                                           2021-04-22T22:12:46.697446671Z
sha256:c3ad9c9da7a31b543ee246f0e30162ccfedce67498986fd5cec0e19437135855  ['torizon/weston-vivante:2']                                                                                    2021-08-24T08:37:12.986272161Z

Information about a specific image can be retrieved using the "info" subcommand and the image's sha256 id:

$ ./tdskt device 06827787 image sha256:c3ad9c9da7a31b543ee246f0e30162ccfedce67498986fd5cec0e19437135855 info   (-127) ↵
Property          Value
id                sha256:c3ad9c9da7a31b543ee246f0e30162ccfedce67498986fd5cec0e19437135855
parent
comment
created           2021-08-24T08:37:12.986272161Z
container         f93dc422ce9af6f97e69a565088fe33b994e096ed3490acf2f15fd3bf62cfe6b
docker_version    20.10.8
author
architecture      arm64
os                linux
size              424189109
virtual_size      424189109
graph_driver      name  overlay2
                  data  LowerDir   /var/lib/docker/overlay2/d2633886eb4060ffb2a445f32b948eccbe89d91e296826bc956bed2289e9da93/diff:/var/lib/docker/overlay2/67e41a806d7e2935438487d45dc2db0cf46d45c4c5167feff0d03916aa06a46b/diff:/var/lib/docker/overlay2/00385ef0f080b8576c365402272f7c4e7feed641ff2b0bad64c631a8433e5e67/diff:/var/lib/docker/overlay2/ae73158a99f6a2c5462dbdfb4a7a10d21086ed2cecb6ecff2eada54da234ebea/diff:/var/lib/docker/overlay2/2ad98064aec8889f52c3f1925eff5af7ea000d6f53d92359aab4c2a00d6fd7fa/diff:/var/lib/docker/overlay2/91ef1218186177fb83f794f403f0a62a98153db7d274aa222a44f6f3a2c0716e/diff:/var/lib/docker/overlay2/211479d2a70057f0b6612cf2404aeda7085ed52bb890e0a8d75361819d4ec1a4/diff:/var/lib/docker/overlay2/28921d4eb9b189149515d6509b60ba0b7469916e86e2014b186fdd2136246e2c/diff:/var/lib/docker/overlay2/ff0b6a2cb3a1a2db1d5b62a4da67127a0f86576b14b2fc0b2b59bc61d22fc163/diff:/var/lib/docker/overlay2/f5567df51055805cc3ef61309bf8bb33c4e21c0b26de8ee8b6e2193da2fca0c6/diff:/var/lib/docker/overlay2/7d6cedbd99819599966c02affe55d399898b66219ed9d30bae321aa92f193ce9/diff:/var/lib/docker/overlay2/367e3869a5f872a05f7f390ee92bd0a7e94a5a17ff7de93127b6f929d56b182d/diff:/var/lib/docker/overlay2/80e6c3c9c1201f3ece5f513d8ee2658aa68a66eaedc5a208db70f0bfb36bf135/diff:/var/lib/docker/overlay2/cb9b1ff8b04099fbd12c9514bcb2cc677455d9cf71be4faa805ae4c381e24cb3/diff:/var/lib/docker/overlay2/9dc970b6243554a985c0dcfb7168980a926c9f285579f7cbfdcd5a4d7af7fdae/diff:/var/lib/docker/overlay2/12f5d03ac02398f4803810cbdb463885641ab4b0f1f8425b8800d2d77f63d497/diff:/var/lib/docker/overlay2/6c717914f5d514a0996c2d3fa30aac50cc92ca068437717b1964b702d07f9cd0/diff:/var/lib/docker/overlay2/aa1b40749d8c483c9bf3e0c3135eb4eb887962c696b614e69662bd564b1dfdca/diff:/var/lib/docker/overlay2/03fc241087b86fa2beee86c8fbdf4d2af9ee58851f13d3eaf660bb00035658a6/diff:/var/lib/docker/overlay2/249547edf579c7b9f7628642a8d33a808b51b7d749a82f6b58d3d851086acf16/diff:/var/lib/docker/overlay2/fa42ff58dda5bcaf7682b74d5f2087fa666d54d65bcff0f5f35cf759a780cce6/diff:/var/lib/docker/overlay2/00c9c1074ecec3b5aa69aeeacd10d0f20eba86595eb19ecbc4a69bcc68bb0b07/diff:/var/lib/docker/overlay2/7faeb03181fd6db7b6c9244050fe3f2af26af8d18af6ba9a86f4b5e12c439ce9/diff:/var/lib/docker/overlay2/07be5be03b44b46865c6a123b5da27d3f2e1870ec85f93f7143cd1488a0dfa47/diff
                        MergedDir  /var/lib/docker/overlay2/bc6f8ff3762b7bcea982ca4c1a7943eb5430529bd8df04978785bcf34ff19ed0/merged
                        UpperDir   /var/lib/docker/overlay2/bc6f8ff3762b7bcea982ca4c1a7943eb5430529bd8df04978785bcf34ff19ed0/diff
                        WorkDir    /var/lib/docker/overlay2/bc6f8ff3762b7bcea982ca4c1a7943eb5430529bd8df04978785bcf34ff19ed0/work
root_fs           type        layers
                  layers      sha256:86cd85a03b6d3b0eed0957346e6d6996b40f5ac1a1da62b63b2c71293a936649
                              sha256:0b3620f246387ee25613a97dfd1ddbdffad5ebee48698e0c5047bac487379b7b
                              sha256:de3c3fc2b41c70157148878788d19795d04fea6c1bb60414cd99ba6983033c25
                              sha256:969a289d250d3e3f0c3801c2490096af467b643c53dfe3e04c8970a314aa2eb7
                              sha256:2f546a025b5d82eccf4d01326420143f83649db09c967581a10329f2782d64f6
                              sha256:fa59f1f5217a353e1399c63f61ac2222510404536ff555033072a41373c9c0a2
                              sha256:9198a4de59c2905805c69de70b19e052a579efee895d7674699c4d6872e810f9
                              sha256:8abf42fe64caead2cd7a19dd80988066ae38b1fbcdc8f1c53a5e745970d74080
                              sha256:7149cfb870852322778fac611e78483f7ae05f9769e46fd14c13b0ed6dbe3f53
                              sha256:9e5ea0390dff1d5be5eec38a5a9ec69edfd118dbf5f787e5f383b1f56b3fa65b
                              sha256:85f500478a8e11ec3843e78cb2a16571c90a01920aa8cec7b83714e78aa1e39c
                              sha256:77d8131fd2853f7e81c2a5156a9a5747ac3e6024ad33689c9d0eabc9b81c1a0c
                              sha256:d42edf5b500254db1eb902f4d33f7f2da8e44ef74ef590000931fce0c675dc2b
                              sha256:08276752e118c19f01616e5057efe39b60a7e12c4300f54497c33d6b8d5cc9b9
                              sha256:b808f74f36cc1f53b8bf8d8a7eb6843e5c0f67fe5b15c6327da2f368aa267659
                              sha256:3e1fd3039e5c25cc8f893e8d26ede21d651386aec260d3ccf8a2716ae8b9a35c
                              sha256:38f4fe53f26f1f8674e8621f60a179e5b21db7044aab1c6e1880ef2d5e4aae3b
                              sha256:c49574efebd6f23525740aaf2714e8b402cbc9265e179525d760e40e3adc7de0
                              sha256:28a878677a93b057e55c841a17be324b5ac39166a4233caacf5678a9d9f1287c
                              sha256:3222f5429d1d80efdfba9622fccdce44039b4b96cae2f015edcab43d52aa2062
                              sha256:eadcb8ca680c1afde0aa96d98000af70f2f40075583220478809e1fca6bb7f99
                              sha256:8a0eaffd36d95345e04861ba85388644539625dfbf94f1d283210082ec050b74
                              sha256:fed1a9b89709babace200749ab29a186b67f5b7352c9d20cc1afbc5751198276
                              sha256:7bf8b1da7ed90c4df908f590f1bae9bb39145493a1b89b23522f8074379fe7a0
                              sha256:e212981a9ebdbe8e7d893fe8449dcdb735016397805af72110d9f08d289273b6
                  base_layer
repo_tags         torizon/weston-vivante:2
repo_digests      torizon/weston-vivante@sha256:a485503fb0334d8f71a127b93f428339d744ff15349f2ac1061cdf30e1866833
container_config  hostname          f93dc422ce9a
                  domainname
                  user
                  attach_stdin      False
                  attach_stdout     False
                  attach_stderr     False
                  exposed_ports
                  tty               False
                  open_stdin        False
                  stdin_once        False
                  env               PATH=/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin
                                    LC_ALL=C.UTF-8
                                    WAYLAND_USER=torizon
                                    XDG_RUNTIME_DIR=/tmp/1000-runtime-dir
                                    WAYLAND_DISPLAY=wayland-0
                                    DISPLAY=:0
                  cmd               /bin/sh
                                    -c
                                    #(nop)
                                    LABEL pipeline.id=20758
                  healthcheck       {}
                  args_escaped      False
                  image             sha256:82ef133944ceab63427c0e6da484969fdec6598f2632bb25138a17d56dd24060
                  volumes
                  working_dir       /home/torizon
                  entrypoint        /usr/bin/entry.sh
                  network_disabled  False
                  mac_address
                  on_build
                  labels            container.name     weston-vivante
                                    container.version  2.5.0-20210824
                                    git.branch         bullseye
                                    git.hash           bf832271831d076f292462ada9b0996547939ed0
                                    pipeline.id        20758
                  stop_signal
                  stop_timeout      0
                  shell
config            hostname
                  domainname
                  user
                  attach_stdin      False
                  attach_stdout     False
                  attach_stderr     False
                  exposed_ports
                  tty               False
                  open_stdin        False
                  stdin_once        False
                  env               PATH=/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin
                                    LC_ALL=C.UTF-8
                                    WAYLAND_USER=torizon
                                    XDG_RUNTIME_DIR=/tmp/1000-runtime-dir
                                    WAYLAND_DISPLAY=wayland-0
                                    DISPLAY=:0
                  cmd
                  healthcheck       {}
                  args_escaped      False
                  image             sha256:82ef133944ceab63427c0e6da484969fdec6598f2632bb25138a17d56dd24060
                  volumes
                  working_dir       /home/torizon
                  entrypoint        /usr/bin/entry.sh
                  network_disabled  False
                  mac_address
                  on_build
                  labels            container.name     weston-vivante
                                    container.version  2.5.0-20210824
                                    git.branch         bullseye
                                    git.hash           bf832271831d076f292462ada9b0996547939ed0
                                    pipeline.id        20758
                  stop_signal
                  stop_timeout      0
                  shell
os_version
metadata          last_tag_time  0001-01-01T00:00:00Z

A specific container image can be deleted using the "delete" subcommand (this can be useful to avoid filling the device storage space when running tests):

$  ./tdskt device 06827787 image sha256:c3ad9c9da7a31b543ee246f0e30162ccfedce67498986fd5cec0e19437135855 delete
Image sha256:c3ad9c9da7a31b543ee246f0e30162ccfedce67498986fd5cec0e19437135855 has been successfully deleted.

Containers

A list of containers can be retrieved using the "containers" subcommand:

./tdskt device 06827787 containers
Id                                                                Name              State
aba0c00281f7f63ab645734a179f96e762e9a14e5e5038be203f16ac4b75538d  /goofy_heyrovsky  {'dead': False,
                                                                                     'error': '',
                                                                                     'exit_code': 0,
                                                                                     'finished_at': '2021-08-25T11:21:36.460296724Z',
                                                                                     'oom_killed': False,
                                                                                     'paused': False,
                                                                                     'pid': 0,
                                                                                     'restarting': False,
                                                                                     'running': False,
                                                                                     'started_at': '2021-08-25T11:21:35.455432588Z',
                                                                                     'status': 'exited'}
aba0c00281f7f63ab645734a179f96e762e9a14e5e5038be203f16ac4b75538d  /goofy_heyrovsky  {'dead': False,
                                                                                     'error': '',
                                                                                     'exit_code': 0,
                                                                                     'finished_at': '2021-08-25T11:21:36.460296724Z',
                                                                                     'oom_killed': False,
                                                                                     'paused': False,
                                                                                     'pid': 0,
                                                                                     'restarting': False,
                                                                                     'running': False,
                                                                                     'started_at': '2021-08-25T11:21:35.455432588Z',
                                                                                     'status': 'exited'}
281254c5baba9d0bf98eb2a61b4f8826265022d1320ce607e9ce559933ef5375  /peaceful_yonath  {'dead': False,
                                                                                     'error': '',
                                                                                     'exit_code': 0,
                                                                                     'finished_at': '0001-01-01T00:00:00Z',
                                                                                     'oom_killed': False,
                                                                                     'paused': False,
                                                                                     'pid': 1457,
                                                                                     'restarting': False,
                                                                                     'running': True,
                                                                                     'started_at': '2021-10-11T14:14:09.357570366Z'

The "info" subcommand can be used to retrieve information about a specific container (please notice that you must use the container-id and not mnemonics):

$ ./tdskt device 06827787 container aba0c00281f7f63ab645734a179f96e762e9a14e5e5038be203f16ac4b75538d info                                                       2 ↵
Property           Value
id                 aba0c00281f7f63ab645734a179f96e762e9a14e5e5038be203f16ac4b75538d
created            2021-08-25T11:21:35.002846039Z
path               provision-device
args               https://app.torizon.io/api/accounts/create-device
                   https://ota-ce.torizon.io
                   eyJhbGciOiJSUzI1NiIsInR5cCI6IkpXVCJ9.eyJleHAiOjE2Mjk4OTAxOTksImlzcyI6ImFjY291bnRzLXN2Yy1wdWIta2V5IiwibmFtZXNwYWNlIjoiODNjMTkwMWYtYWJhMS00ZGU1LWFjZTYtNDA2N2ZlMGFjNmIzIiwibmJmIjoxNjI5ODg4Mzk5LCJzdWIiOiI0Y2VkMTViNS1lMTAyLTRmZmMtYjlkZS1jZTQzZDU2NDdiMjIifQ.kUz2R90sObT4d0sOnCUeQ5nvrxKrr8YQ0-ove1QMt9BSih5AkhPDYEF4QogePzjIeLPeTSto-yfHjTRq-zukzv4eiKHqgJXg-YLIS7UPf_TodVhbZ8iqD2f4dkN4u38zaNqypodJnRsCgTlNsSdtl3u76-ElUg_JZhUeAYE1EuftusQfLK_fuatoJvNzHkRgDTUy4hRj2YQC86N7fVHJzxbmFP7fGoDrfOs7MJcTWNIIhY-BJIKN70Zk2TWrl-M26VZ3RRy6ezvmo73RBQclQslbkruAaBsniXku3w7X75GO2_LNVWmBj6TnUATIt7mACRP3wF1gS55dhTnzakf5EQ
state              status       exited
                   running      False
                   paused       False
                   restarting   False
                   oom_killed   False
                   dead         False
                   pid          0
                   exit_code    0
                   error
                   started_at   2021-08-25T11:21:35.455432588Z
                   finished_at  2021-08-25T11:21:36.460296724Z
image              sha256:00e2e9440c08ff1311d8f5dae8abc032ed8f9cc1082a4940919f572ef53c8784
resolv_conf_path   /var/lib/docker/containers/aba0c00281f7f63ab645734a179f96e762e9a14e5e5038be203f16ac4b75538d/resolv.conf
hostname_path      /var/lib/docker/containers/aba0c00281f7f63ab645734a179f96e762e9a14e5e5038be203f16ac4b75538d/hostname
hosts_path         /var/lib/docker/containers/aba0c00281f7f63ab645734a179f96e762e9a14e5e5038be203f16ac4b75538d/hosts
log_path
node
name               /goofy_heyrovsky
restart_count      0
driver             overlay2
mount_label
process_label
app_armor_profile
exec_ids
host_config        {'auto_remove': False,
                    'binds': ['/var/sota:/data', '/etc/hostname:/etc_host/hostname'],
                    'blkio_device_read_bps': [],
                    'blkio_device_read_i_ops': [],
                    'blkio_device_write_bps': [],
                    'blkio_device_write_i_ops': [],
                    'blkio_weight': 0,
                    'blkio_weight_device': [],
                    'cap_add': [],
                    'cap_drop': [],
                    'cgroup': '',
                    'cgroup_parent': '',
                    'container_id_file': '',
                    'cpu_count': 0,
                    'cpu_percent': 0,
                    'cpu_period': 0,
                    'cpu_quota': 0,
                    'cpu_realtime_period': 0,
                    'cpu_realtime_runtime': 0,
                    'cpu_shares': 0,
                    'cpuset_cpus': '',
                    'cpuset_mems': '',
                    'device_cgroup_rules': [],
                    'devices': [],
                    'disk_quota': 0,
                    'dns': [],
                    'dns_options': [],
                    'dns_search': [],
                    'extra_hosts': [],
                    'group_add': [],
                    'init': None,
                    'io_maximum_bandwidth': 0,
                    'io_maximum_i_ops': 0,
                    'ipc_mode': 'private',
                    'kernel_memory': 0,
                    'links': [],
                    'log_config': {'Config': {}, 'Type': 'journald'},
                    'memory': 0,
                    'memory_reservation': 0,
                    'memory_swap': 0,
                    'memory_swappiness': None,
                    'mounts': [],
                    'nano_cpus': 0,
                    'network_mode': 'host',
                    'oom_kill_disable': False,
                    'oom_score_adj': 0.0,
                    'pid_mode': '',
                    'pids_limit': None,
                    'port_bindings': {},
                    'privileged': True,
                    'publish_all_ports': False,
                    'readonly_rootfs': False,
                    'restart_policy': {'MaximumRetryCount': 0, 'Name': 'no'},
                    'runtime': 'runc',
                    'security_opt': [],
                    'shm_size': 67108864.0,
                    'storage_opt': {},
                    'sysctls': {},
                    'tmpfs': {},
                    'ulimits': [],
                    'userns_mode': '',
                    'uts_mode': '',
                    'volume_driver': '',
                    'volumes_from': []}
graph_driver       name  overlay2
                   data  LowerDir   /var/lib/docker/overlay2/c826d31237220e566643dc8602c54ffa134444548a4cfa9fb61f3fbc53d08a9f-init/diff:/var/lib/docker/overlay2/3ef6ab003b79964aececb05f163e94e5929c84509c4563868259f2d8d49060cc/diff:/var/lib/docker/overlay2/af9939f0842454853137e6b735c8ba97ebded1aef3021fc3c303f8a425e9ad7c/diff:/var/lib/docker/overlay2/5b63dde5d9fe36a23d6c148344d061e5482b18836f07acd0cb7d6807055a647c/diff
                         MergedDir  /var/lib/docker/overlay2/c826d31237220e566643dc8602c54ffa134444548a4cfa9fb61f3fbc53d08a9f/merged
                         UpperDir   /var/lib/docker/overlay2/c826d31237220e566643dc8602c54ffa134444548a4cfa9fb61f3fbc53d08a9f/diff
                         WorkDir    /var/lib/docker/overlay2/c826d31237220e566643dc8602c54ffa134444548a4cfa9fb61f3fbc53d08a9f/work
size_rw            0
size_root_fs       0
mounts             {'destination': '/data',
                    'driver': '',
                    'mode': '',
                    'name': '',
                    'propagation': 'rprivate',
                    'rw': True,
                    'source': '/var/sota',
                    'type': 'bind'}
                   {'destination': '/etc_host/hostname',
                    'driver': '',
                    'mode': '',
                    'name': '',
                    'propagation': 'rprivate',
                    'rw': True,
                    'source': '/etc/hostname',
                    'type': 'bind'}
config             hostname          verdin-imx8mm-06827787
                   domainname
                   user
                   attach_stdin      False
                   attach_stdout     True
                   attach_stderr     True
                   exposed_ports
                   tty               False
                   open_stdin        False
                   stdin_once        False
                   env               PATH=/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin
                   cmd               provision-device
                                     https://app.torizon.io/api/accounts/create-device
                                     https://ota-ce.torizon.io
                                     eyJhbGciOiJSUzI1NiIsInR5cCI6IkpXVCJ9.eyJleHAiOjE2Mjk4OTAxOTksImlzcyI6ImFjY291bnRzLXN2Yy1wdWIta2V5IiwibmFtZXNwYWNlIjoiODNjMTkwMWYtYWJhMS00ZGU1LWFjZTYtNDA2N2ZlMGFjNmIzIiwibmJmIjoxNjI5ODg4Mzk5LCJzdWIiOiI0Y2VkMTViNS1lMTAyLTRmZmMtYjlkZS1jZTQzZDU2NDdiMjIifQ.kUz2R90sObT4d0sOnCUeQ5nvrxKrr8YQ0-ove1QMt9BSih5AkhPDYEF4QogePzjIeLPeTSto-yfHjTRq-zukzv4eiKHqgJXg-YLIS7UPf_TodVhbZ8iqD2f4dkN4u38zaNqypodJnRsCgTlNsSdtl3u76-ElUg_JZhUeAYE1EuftusQfLK_fuatoJvNzHkRgDTUy4hRj2YQC86N7fVHJzxbmFP7fGoDrfOs7MJcTWNIIhY-BJIKN70Zk2TWrl-M26VZ3RRy6ezvmo73RBQclQslbkruAaBsniXku3w7X75GO2_LNVWmBj6TnUATIt7mACRP3wF1gS55dhTnzakf5EQ
                   healthcheck       {}
                   args_escaped      False
                   image             torizon/torizon-provisioner:0.0.9
                   volumes
                   working_dir
                   entrypoint
                   network_disabled  False
                   mac_address
                   on_build
                   labels
                   stop_signal
                   stop_timeout      0
                   shell
network_settings   bridge
                   sandbox_id                  56cb69c28d046fd2bec7a452bb37fa8ce759d17f9dfd5feba9de0072701bde42
                   hairpin_mode                False
                   link_local_ipv6_address
                   link_local_ipv6_prefix_len  0
                   ports
                   sandbox_key                 /var/run/docker/netns/default
                   secondary_ip_addresses
                   secondary_ipv6_addresses
                   endpoint_id
                   gateway
                   global_ipv6_address
                   global_ipv6_prefix_len      0
                   ip_address
                   ip_prefix_len               0
                   ipv6_gateway
                   mac_address
                   networks                    host  {'aliases': [],
                                                      'driver_opts': {},
                                                      'endpoint_id': '',
                                                      'gateway': '',
                                                      'global_ipv6_address': '',
                                                      'global_ipv6_prefix_len': 0,
                                                      'ip_address': '',
                                                      'ip_prefix_len': 0,
                                                      'ipam_config': {},
                                                      'ipv6_gateway': '',
                                                      'links': [],
                                                      'mac_address': '',
                                                      'network_id': '2933a01b622ef7234aedffb65996bf9bd0a4762cca8c2a4de6db84ce886333b0'}

You can (re)start and stop containers using the "start" and "stop" commands. If a container is in the running state you can use the "ps", "mem" and "storage" commands, as described in the above chapter, to retrieve information that is specific for the container:

$ ./tdskt device 06827787 container b648801cc191f448d140220fee93c8e16958da8f1d9b3183bf0c11df4ee2ff40 ps
  PID    PPID  User    Time        Nice  State    Args
    1       0  root    00:00:00       0  Ss+      bash
  346       0  root    00:00:00       0  Rs       ps -A -o pid,ppid,user,time,nice,stat,args
$ ./tdskt device 06827787 container b648801cc191f448d140220fee93c8e16958da8f1d9b3183bf0c11df4ee2ff40 mem
Property          Value
total       2.02504e+06
available   1.54931e+06
free        1.05221e+06
$ ./tdskt device 06827787 container b648801cc191f448d140220fee93c8e16958da8f1d9b3183bf0c11df4ee2ff40 storage
Mount poinf     FS                              Size    Available
/               overlay                     15226800      9905112
/dev            tmpfs                          65536        65536
/dev/shm        shm                            65536        65536
/etc/hosts      /dev/disk/by-label/otaroot  15226800      9905112
/proc/asound    tmpfs                        1012516      1012516
/sys/firmware   tmpfs                        1012516      1012516
/sys/fs/cgroup  tmpfs                        1012516      1012516

To get process information the container must have the "ps" tool installed (on debian it's part of the procps package), if it's not installed you will get an empty list.

The "logs" subcommand can be used to collect a container's logs (if the container is still running the output will be updated in sync with output on the container's console):

$ ./tdskt device 06827787 container b648801cc191f448d140220fee93c8e16958da8f1d9b3183bf0c11df4ee2ff40 logs                                                       2 ↵
root@b648801cc191:/# ls -la

total 84

drwxr-xr-x   1 root root 4096 Oct 11 14:18 .

drwxr-xr-x   1 root root 4096 Oct 11 14:18 ..

-rwxr-xr-x   1 root root    0 Oct 11 14:18 .dockerenv

drwxr-xr-x   1 root root 4096 Oct 11 14:21 bin

drwxr-xr-x   2 root root 4096 Apr 10  2021 boot

...

drwxr-xr-x   1 root root 4096 Sep 27 00:00 var

Additional device information

If you are running the command-line interface inside a container, it may not be able to resolve the device hostname to a valid ip. To overcome this issue, if you have the backend running on the host system, you can query the device ip:

$  ./tdskt device 06827787 ip
192.168.1.128

to allow the backend to connect securely over SSH without using a password a set of private/public keys is generated during device detect and the public key is configured to let the configured user access via ssh. If you need to estabilish an ssh connection you may need the key. Its path can be returned by the "key" subcommand:

$ ./tdskt device 06827787 key                                                                                       ✔
/home/myuser/.moses/devices/06827787/id_rsa

If you need to activate an ssh connection to the remote device to run some commands etc. you may use this command:

$ ssh -i $(./tdskt device 06827787 key) torizon@$(./tdskt device 06827787 ip) ls / -la                              1 ↵
total 28
drwxr-xr-x  12 root root 4096 Aug 25 12:15 .
drwxr-xr-x  12 root root 4096 Aug 25 12:15 ..
lrwxrwxrwx   3 root root    7 Jul  1 01:56 bin -> usr/bin
drwxr-xr-x   4 root root 4096 Aug 25 12:15 boot
...
drwxr-xr-x  10 root root 4096 Aug 25 12:15 var

Copying files to the device

The "sync" subcommand allows you to synchronize a source folder on your machine with a destination folder on the device. This synchronization is performed using rsync and so it's quite efficent, transferring only new or modified files.

For example, this command synchronizes the contents of current directory to a folder named /home/torizon/dummy on the target device:

./tdskt device 06827787 sync $(pwd)/platforms /home/torizon/dummy

Working with platforms

Platforms are used to define an application type. This includes an architecture (arm32 or arm64) a base distribution (usually debian) and additional runtimes required to run the user's app (for example Python or Qt). Custom platforms can be defined by users and stored in the .moses/platform directory inside the user's home folder. Platforms can't be modified using the API.

List available platforms

The "platforms" command lists all the available platforms on your system:

$ ./tdskt platforms                                                                                                                                               ✔
Id                                            Name                                    Version  Custom
arm32v7-dotnet-uno_bullseye                   .NET 5.0 Uno App arm32v7 bullseye             1  True
arm64v8-dotnet-uno_bullseye                   .NET 5.0 Uno App arm64v8 bullseye             1  True
...
arm64v8-debian-base-no-ssh_buster             debian arm64v8 buster                         1  True
arm64v8-debian-base_buster                    debian arm64v8 buster                         1  True

Additional information about a specific platform can be retrieved using the "info" subcommand:

$  ./tdskt platform arm64v8-debian_bullseye info                                                                                                                 2 ↵
Property              Value
id                    arm64v8-debian_bullseye
name                  debian arm64v8 bullseye
standard              True
version               1.0
runtimes              c-cpp
sdkcontainerusername  build
sdkcontainerpassword  build
dockercomposefile     common
                      debug
                      release
startupscript         common
                      debug
                      release
shutdownscript        common
                      debug
                      release
ports                 common   {}
                      debug    {'2222/tcp': ''}
                      release  {}
volumes               common   {}
                      debug    {}
                      release  {}
devices               common   []
                      debug    []
                      release  []
networks              common   []
                      debug    []
                      release  []
extraparms            common   {}
                      debug    {}
                      release  {}
props                 common   {'linkeroptions': '-Wl,--dynamic-linker=/lib/ld-linux-aarch64.so.1', 'prefix': 'aarch64-linux-gnu-'}
                      debug    {}
                      release  {}
description           Platform for headless applications with a minimal set of libs for arm64v8 on debian-buster
tags                  console
architecture          linux/arm64
deprecated            False

A platform may not be compatible with all the devices. For example, arm64 platforms require a 64 bits SOC. You can get a list of configured devices that are compatible with a specific platform using the "compatible" subcommand.

$ ./tdskt platform arm64v8-debian_bullseye compatible                                                                                                             ✔
      Id  Name                                                                  Model  HW release
06772320  Toradex Apalis iMX8QM V1.1 on Apalis Evaluation Board(06772320)        0037  V1.1C
06827787  Toradex Verdin iMX8M Mini WB on Verdin Development Board(06827787)     0055  V1.1A

EULAs

EULAs (End User License Agreement) are connected to some platforms that may include proprietary code that requires an additional license (for example for the GPU drivers of imx8-based devices). If the user does not accept the license those platform can't be used to build, deploy and run application. Since accepting EULAs in a non-interactive environment may be problematic, the command line interface provides a way to do this from you automation scripts.

You can list available EULAs with the "eulas" command:

$ ./tdskt eulas
Id             Title                                        Accepted    Visualized
nxp-la-opt-v5  LA_OPT_NXP_Software_License v5 January 2019  False       False

You can find the path of the EULA text file using the "info" subcommand:

$ ./tdskt eula nxp-la-opt-v5 info                                                                                                                                 ✔
Property    Value
id          nxp-la-opt-v5
title       LA_OPT_NXP_Software_License v5 January 2019
question    To use software provided by NXP on modules based on i.MX8 you have to accept "LA_OPT_NXP_Software_License v5 January 2019". Do you want to accept this license?
filepath    /home/valter/Work/ide-plugins/moses/moses/eulas/nxp-la-opt-v5/nxp-eula.txt
visualized  True
accepted    False

Accept an EULA

To accept an eula you can set its "Accepted" property to True:

$ ./tdskt eula nxp-la-opt-v5 setprop accepted true

Work with applications

Application objects store the information required to build, deploy and run containers. Application information is usually stored together with the user's application code and it's used by the IDE extensions to deploy, run and debug it on the target device. This information may be stored in a git repository together with the application code. The configuration files are stored in a folder named appconfig_* (the directories are numbered since you may store multiple configurations for the same codebase). A folder named "work" is created inside this directory during some operations, this folder should not be added to source repositories.

Create a new application

To create a new application object you can use the "create" command, providing a valid platform id and a directory where the appconfig_* subfolder will be created. The command returns an application id that could be used to reference the application in subsequent commands.

./tdskt create arm32v7-debian-no-ssh_bullseye $(pwd)/dummy                                                          ✔
ce365e8d-e6f4-4ce3-9dfe-e5565ed01986

Load an existing application

When the backend start it can't load any application object. Those are stored together with code and their filesystem location is not known so, if you need to work on an application configuration that you created in a past session you have to load it first.

This can be done using the "load" command:

$ ./tdskt load $(pwd)/dummy/appconfig_0                                                                        (-127) ↵
ce365e8d-e6f4-4ce3-9dfe-e5565ed01986

You can get detailed information about your application using the "info" subcommand:

$  ./tdskt application ce365e8d-e6f4-4ce3-9dfe-e5565ed01986 info                                                     2 ↵
Property           Value
id                 ce365e8d-e6f4-4ce3-9dfe-e5565ed01986
platformid         arm32v7-debian-no-ssh_bullseye
folder             /home/valter/Work/ide-plugins/moses/moses/dist/moses-linux/dummy/appconfig_0
props              common   {'arg': '', 'buildcommands': '', 'buildfiles': '', 'command': '', 'devpackages': '', 'env': '', 'expose': '', 'extrapackages': '', 'preinstallcommands': '', 'sdkpostinstallcommands': '', 'sdkpreinstallcommands': '', 'targetcommands': '', 'targetfiles': ''}
                   debug    {'arg': 'ARG SSHUSERNAME=#%application.username%#\n'}
                   release  {}
dockercomposefile  common
                   debug
                   release
startupscript      common
                   debug
                   release
shutdownscript     common
                   debug
                   release
ports              common   {}
                   debug    {}
                   release  {}
volumes            common   {}
                   debug    {}
                   release  {}
devices            common   []
                   debug    []
                   release  []
networks           common   []
                   debug    []
                   release  []
extraparms         common   {}
                   debug    {}
                   release  {}
username           torizon
images             debug
                   release
sdkimages          debug
                   release
imagetags          debug    arm32v7-debian-no-ssh_bullseye_debug_ce365e8d-e6f4-4ce3-9dfe-e5565ed01986
                   release  arm32v7-debian-no-ssh_bullseye_release_ce365e8d-e6f4-4ce3-9dfe-e5565ed01986
sdkimagetags       debug    arm32v7-debian-no-ssh_bullseye_debug_ce365e8d-e6f4-4ce3-9dfe-e5565ed01986_sdk_image
                   release  arm32v7-debian-no-ssh_bullseye_release_ce365e8d-e6f4-4ce3-9dfe-e5565ed01986_sdk_image
otapackagename
otapackageversion  1.0.0

Build, deploy and run an application

An application object has two configuration: debug and release. This is required by the fact that debugging an application may require additional tools inside the container (typically a debugger client) and those may need to expose additional ports or enable other services like SSH. Shipping such a container to production can be a security problem and will use more resources than those actually needed by the user's application. That's why two configuration make sense for many kinds of applications.

The first step you need to perform to be able to run your application on a device is building it. Build will happen on the local machine, using docker. To build your application you can use the "build" subcommand and select the configuration (debug/release) that you want to build (passing -p will enable progress information and return some information about what's happening during build):

$ ./tdskt -p application ce365e8d-e6f4-4ce3-9dfe-e5565ed01986 build release                                           ✔
Building application, this may take a few minutes...
Step 1/7 : FROM --platform=linux/arm torizon/debian:2-bullseye
 ---> 2471538edb28
Step 2/7 : ENV DEBIAN_FRONTEND="noninteractive"
 ---> Using cache
 ---> b3ce4188943b
Step 3/7 : RUN if [ ! -z "" ]; then     apt-get -q -y update     && apt-get -q -y install      && rm -rf /var/lib/apt/lists/* ;     fi
 ---> [Warning] The requested image's platform (linux/arm/v7) does not match the detected host platform (linux/amd64) and no specific platform was requested
 ---> Running in b5dabb41253c
 ---> a5959e502432
...
 ---> Running in 8967a81790db
 ---> bfed8e4c32be
Successfully built bfed8e4c32be
Successfully tagged arm32v7-debian-no-ssh_bullseye_release_ce365e8d-e6f4-4ce3-9dfe-e5565ed01986:latest
Application ce365e8d-e6f4-4ce3-9dfe-e5565ed01986 successfully built.
operation completed successfully

The warning about image platform not maching the host architecture is not an issue, since we are actually "cross-compiling" our image. On a linux-based machine you have to enable emulation to be able to cross-compile containers, check the paragraph below for more information.

Once you have built an image you can then deploy it to a device using the "deploy" subcommand (in this case you need to provide a target device for the operation and -p returns progress information):

$ ./tdskt application ce365e8d-e6f4-4ce3-9dfe-e5565ed01986 deploy release 06827787                                    ✔
Deploying application, this may take a few minutes...
Application ce365e8d-e6f4-4ce3-9dfe-e5565ed01986 successfully deployed.

Once your application has been deployed you can run it on the target using the "run" subcommand:

$ ./tdskt application ce365e8d-e6f4-4ce3-9dfe-e5565ed01986 run release 06827787                                     2 ↵
Starting application...
Started container 315005993abb2d3b3282923e5097f9b7bb6d15fb20a9fb61fcf3ef593f3aa76f

If the application was already running on the device it will be restarted. Running an application for the first time may require some time if it requires additional containers to be started (for example weston for graphical UI apps) since those container may have to be dowloaded first.

If you want to monitor the execution of your application you can use the device-containers features described previously and the "container" subcommand may be used to retrieve information about the current instance of your application's container, including its id:

$ ./tdskt -p application ce365e8d-e6f4-4ce3-9dfe-e5565ed01986 container release 06827787                              ✔
Property           Value
id                 298efc99448fd355baf9417390d2780ed785c13c0a7a18b22c77bbc606d88979
created            2021-10-12T08:22:27.539254144Z
path               /bin/sh
args               -c
                   //
state              status       exited
                   running      False
                   paused       False
                   restarting   False
                   oom_killed   False
                   dead         False
                   pid          0
                   exit_code    126
                   error
                   started_at   2021-10-12T08:22:28.820484343Z
                   finished_at  2021-10-12T08:22:28.869419684Z
image              sha256:bfed8e4c32bea435a25fc312a30db11478a700214f9ec3d07d87d6f8923d787f
resolv_conf_path   /var/lib/docker/containers/298efc99448fd355baf9417390d2780ed785c13c0a7a18b22c77bbc606d88979/resolv.conf
hostname_path      /var/lib/docker/containers/298efc99448fd355baf9417390d2780ed785c13c0a7a18b22c77bbc606d88979/hostname
hosts_path         /var/lib/docker/containers/298efc99448fd355baf9417390d2780ed785c13c0a7a18b22c77bbc606d88979/hosts
log_path
node
name               /arm32v7-debian-no-ssh_bullseye_release_ce365e8d-e6f4-4ce3-9dfe-e5565ed01986_latest_instance
restart_count      0
driver             overlay2
mount_label
process_label
app_armor_profile
exec_ids
host_config        {'auto_remove': False,
                    'binds': [],
                    'blkio_device_read_bps': [],
                    'blkio_device_read_i_ops': [],
                    'blkio_device_write_bps': [],
                    'blkio_device_write_i_ops': [],
                    'blkio_weight': 0,
                    'blkio_weight_device': [],
                    'cap_add': [],
                    'cap_drop': [],
                    'cgroup': '',
                    'cgroup_parent': '',
                    'container_id_file': '',
                    'cpu_count': 0,
                    'cpu_percent': 0,
                    'cpu_period': 0,
                    'cpu_quota': 0,
                    'cpu_realtime_period': 0,
                    'cpu_realtime_runtime': 0,
                    'cpu_shares': 0,
                    'cpuset_cpus': '',
                    'cpuset_mems': '',
                    'device_cgroup_rules': [],
                    'devices': [],
                    'disk_quota': 0,
                    'dns': [],
                    'dns_options': [],
                    'dns_search': [],
                    'extra_hosts': [],
                    'group_add': [],
                    'init': None,
                    'io_maximum_bandwidth': 0,
                    'io_maximum_i_ops': 0,
                    'ipc_mode': 'private',
                    'kernel_memory': 0,
                    'links': [],
                    'log_config': {'Config': {}, 'Type': 'journald'},
                    'memory': 0,
                    'memory_reservation': 0,
                    'memory_swap': 0,
                    'memory_swappiness': None,
                    'mounts': [],
                    'nano_cpus': 0,
                    'network_mode': 'default',
                    'oom_kill_disable': False,
                    'oom_score_adj': 0.0,
                    'pid_mode': '',
                    'pids_limit': None,
                    'port_bindings': {},
                    'privileged': False,
                    'publish_all_ports': False,
                    'readonly_rootfs': False,
                    'restart_policy': {'MaximumRetryCount': 0, 'Name': ''},
                    'runtime': 'runc',
                    'security_opt': [],
                    'shm_size': 67108864.0,
                    'storage_opt': {},
                    'sysctls': {},
                    'tmpfs': {},
                    'ulimits': [],
                    'userns_mode': '',
                    'uts_mode': '',
                    'volume_driver': '',
                    'volumes_from': []}
graph_driver       name  overlay2
                   data  LowerDir   /var/lib/docker/overlay2/fe86513d0d448b99fde260e3097abaa7e957b044383fb7afa67e096cbe13ad9a-init/diff:/var/lib/docker/overlay2/08dd6b9c60fb235b639af8e25f0537ba02f3b83c3aed0019042d5d0b9110014c/diff:/var/lib/docker/overlay2/d51509dfde8509f7ee4be83f093db02f73c10e17a58149cd84ca8252b9c1035d/diff:/var/lib/docker/overlay2/152f0e4ca0a39704fd07548b125120faed3f206f9cea7a0b01052545eb973630/diff:/var/lib/docker/overlay2/eea82fb59f87bfe01fd1d4f5beeaf8de4e65b5d10256d69a93274ca47602d9ee/diff:/var/lib/docker/overlay2/87eaa06c98ceb0ed6643cb7ae1e34727fca4deb089c8d03aaddd758527fd3c95/diff:/var/lib/docker/overlay2/26bd92067f6ea056aa1d43be3c0223e71c0392c4967993168b7aeb6000be1fe6/diff:/var/lib/docker/overlay2/1ea937cb51bce32d314d642c2f1fc0bd2a449e31005ff0fc200ebcd85bdd5f7b/diff:/var/lib/docker/overlay2/6f654a67f10ab9aab3eb0274d4c88afc023e735411c5cb4d66b5cd401f4378cc/diff:/var/lib/docker/overlay2/d186af37e1d4328ddc0ee69acad4fa43993342b141eb37377df4fb6efcbb22f0/diff:/var/lib/docker/overlay2/8d5606df66c0f32b80c6fe3a71f774a68e0eca76d6039942ddc255a49b4de807/diff:/var/lib/docker/overlay2/4cdfcb063f094250cc29514b57961e2f460a8e15bd6ff52a6c8782b39cd6252d/diff
                         MergedDir  /var/lib/docker/overlay2/fe86513d0d448b99fde260e3097abaa7e957b044383fb7afa67e096cbe13ad9a/merged
                         UpperDir   /var/lib/docker/overlay2/fe86513d0d448b99fde260e3097abaa7e957b044383fb7afa67e096cbe13ad9a/diff
                         WorkDir    /var/lib/docker/overlay2/fe86513d0d448b99fde260e3097abaa7e957b044383fb7afa67e096cbe13ad9a/work
size_rw            0
size_root_fs       0
mounts
config             hostname          298efc99448f
                   domainname
                   user              torizon
                   attach_stdin      False
                   attach_stdout     False
                   attach_stderr     False
                   exposed_ports
                   tty               False
                   open_stdin        False
                   stdin_once        False
                   env               PATH=/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin
                                     LC_ALL=C.UTF-8
                                     DEBIAN_FRONTEND=noninteractive
                   cmd               /bin/sh
                                     -c
                                     //
                   healthcheck       {}
                   args_escaped      False
                   image             sha256:bfed8e4c32bea435a25fc312a30db11478a700214f9ec3d07d87d6f8923d787f
                   volumes
                   working_dir       /
                   entrypoint
                   network_disabled  False
                   mac_address
                   on_build
                   labels            container.name     arm32v7-debian-base
                                     container.version  2.3.1-20210920
                                     git.branch         bullseye
                                     git.hash           1ef7e8ff62aec45db967ea7443db8dfa763f0b33
                                     pipeline.id        22318
                   stop_signal
                   stop_timeout      0
                   shell
network_settings   bridge
                   sandbox_id                  10ddb94150ac3373554574ac66a00754920ea82abb7399b61610aae8c5276c35
                   hairpin_mode                False
                   link_local_ipv6_address
                   link_local_ipv6_prefix_len  0
                   ports
                   sandbox_key                 /var/run/docker/netns/10ddb94150ac
                   secondary_ip_addresses
                   secondary_ipv6_addresses
                   endpoint_id
                   gateway
                   global_ipv6_address
                   global_ipv6_prefix_len      0
                   ip_address
                   ip_prefix_len               0
                   ipv6_gateway
                   mac_address
                   networks                    bridge  {'aliases': [],
                                                        'driver_opts': {},
                                                        'endpoint_id': '',
                                                        'gateway': '',
                                                        'global_ipv6_address': '',
                                                        'global_ipv6_prefix_len': 0,
                                                        'ip_address': '',
                                                        'ip_prefix_len': 0,
                                                        'ipam_config': {},
                                                        'ipv6_gateway': '',
                                                        'links': [],
                                                        'mac_address': '',
                                                        'network_id': '827f2d964f10ad1cdc0a341a71559a1e289e714b7e61ef2a52d025231212e6fb'}

You can stop a running istance of your application using the "stop" subcommand:

$ ./tdskt -p application ce365e8d-e6f4-4ce3-9dfe-e5565ed01986 stop release 06827787
$

The "logs" subcommand may be used to collect container's output for the current instance or for one that just terminated (in this case execution will return to the prompt):

$ ./tdskt -p application ce365e8d-e6f4-4ce3-9dfe-e5565ed01986 logs release 06827787
Hello World!
$

SDK operations

Some platform (typically those using C/C++ as development language) provide an SDK that can be used to build the application code using the right toolchain and libraries.

The SDK is provided as a container that can run on a developer's machine.

To build and start the container you can use the "updatesdk" subcommand (also the SDK support debug and release configuration to easily support different options to enable/disable, for example, code optimization or instrumentation):

$ ./tdskt -p application ce365e8d-e6f4-4ce3-9dfe-e5565ed01986 updatesdk release                                     2 ↵
Updating SDK, this may require a few minutes...
Step 1/4 : FROM torizon/debian-cross-toolchain-armhf:2-bullseye
 ---> 9dc71b89193e
Step 2/4 : RUN apt-get -q -y update     && apt-get -q -y upgrade     && apt-get -q -y install gdb-multiarch procps rsync openssh-client            && rm -rf /var/lib/apt/lists/*
 ---> Running in 61ffba77f5d4
...
---> Running in d6f75d6eebe2
 ---> 983efa93c58c
Successfully built 983efa93c58c
Successfully tagged arm32v7-debian-no-ssh_bullseye_release_ce365e8d-e6f4-4ce3-9dfe-e5565ed01986_sdk_image:latest
SDK for application ce365e8d-e6f4-4ce3-9dfe-e5565ed01986 successfully updated.
operation completed successfully

The "runsdk" subcommand will just start the container or return information about the running instance, without rebuilding it.

$ ./tdskt -p application ce365e8d-e6f4-4ce3-9dfe-e5565ed01986 runsdk release                                        2 ↵
operation completed successfully
Property    Value
host_ip
host_port

Some SDK containers expose an SSH interface (using build:build by default as user/pass), for those who don't the returned properties will be empty, as in the sample.

export information

In some scenarios you may want to run your application's container from the command line or from a docker compose file. The command-line interface provides a couple of features to make this easy.

The "cmdline" subcommand can be used to generate a valid docker command line to run your container:

$ ./tdskt -p application ce365e8d-e6f4-4ce3-9dfe-e5565ed01986 cmdline release                                         ✔
docker run arm32v7-debian-no-ssh_bullseye_release_ce365e8d-e6f4-4ce3-9dfe-e5565ed01986

The generated command line will include all the extra properties you can set via application configuration in the IDEs.

The "composefile" command allows you to generate a docker-compose file. This will include also additional containers required by your application, if so configured:

$ ./tdskt -p application ce365e8d-e6f4-4ce3-9dfe-e5565ed01986 composefile release                                     ✔
services:
  arm32v7-debian-no-ssh_bullseye_release_ce365e8d-e6f4-4ce3-9dfe-e5565ed01986:
    depends_on: []
    devices: []
    image: arm32v7-debian-no-ssh_bullseye_release_ce365e8d-e6f4-4ce3-9dfe-e5565ed01986
    ports: []
    volumes: []
version: '2.4'

Change application's configuration

Usually application configuration is modified using the UI provided by the IDE extensions, but in some cases it may be useful to do this programmatically. To do this the "setprop" command can be used to change configuration for debug, release or common parameters. For examle, this command:

$ ./tdskt -p application 421ad07f-5bfe-4cd3-83eb-8266e1d08312 setprop extraparms.privileged true common

Will run the application in privileged mode for both debug and release mode. This one instead:

$ ./tdskt -p application 421ad07f-5bfe-4cd3-83eb-8266e1d08312 setprop ports.8080/tcp 80 debug

Will map port 8080 to container's port 80 only for debug configuration. Some of the application properties are array or dictionaries. For dictionaries (like ports or extraparms above) a single entry can be changed or addded, for array you need to pass the whole sequence:

$ ./tdskt -p application 421ad07f-5bfe-4cd3-83eb-8266e1d08312 setprop networks '[network1,network2]' release

After the changes you may ue the "info" subcommand to verify that your changes have been applied.

$ ./tdskt -p application 421ad07f-5bfe-4cd3-83eb-8266e1d08312 info                                                  ✔
Property           Value
id                 421ad07f-5bfe-4cd3-83eb-8266e1d08312
platformid         arm32v7-debian-no-ssh_bullseye
folder             /home/valter/Work/ide-plugins/moses/moses/dist/moses-linux/dummy/appconfig_0
props              common   {'arg': '', 'buildcommands': '', 'buildfiles': '', 'command': '', 'devpackages': '', 'env': '', 'expose': '', 'extrapackages': '', 'preinstallcommands': '', 'sdkpostinstallcommands': '', 'sdkpreinstallcommands': '', 'targetcommands': '', 'targetfiles': ''}
                   debug    {'arg': 'ARG SSHUSERNAME=#%application.username%#\n'}
                   release  {}
dockercomposefile  common
                   debug
                   release
startupscript      common
                   debug
                   release
shutdownscript     common
                   debug
                   release
ports              common   {}
                   debug    {'8080/tcp': '80'}
                   release  {}
volumes            common   {}
                   debug    {}
                   release  {}
devices            common   []
                   debug    []
                   release  []
networks           common   []
                   debug    []
                   release  ['network1', 'network2']
extraparms         common   {'privileged': 'true'}
                   debug    {}
                   release  {}
username           torizon
images             debug
                   release
sdkimages          debug
                   release
imagetags          debug    arm32v7-debian-no-ssh_bullseye_debug_421ad07f-5bfe-4cd3-83eb-8266e1d08312
                   release  arm32v7-debian-no-ssh_bullseye_release_421ad07f-5bfe-4cd3-83eb-8266e1d08312
sdkimagetags       debug    arm32v7-debian-no-ssh_bullseye_debug_421ad07f-5bfe-4cd3-83eb-8266e1d08312_sdk_image
                   release  arm32v7-debian-no-ssh_bullseye_release_421ad07f-5bfe-4cd3-83eb-8266e1d08312_sdk_image
otapackagename
otapackageversion  1.0.0

You may also check how your changes impact docker command line:

$  ./tdskt -p application 421ad07f-5bfe-4cd3-83eb-8266e1d08312 cmdline debug
docker run --privileged --publish 6502/tcp --publish 80:8080/tcp arm32v7-debian-no-ssh_bullseye_debug_421ad07f-5bfe-4cd3-83eb-8266e1d08312

Push applications to container registries and Torizon OTA

If you want to push your application to a docker registry you can use the "push" subcommand. Before doing that you need to ensure that you built the image you plan to upload and that you set the "tag" property to the repository:tag you plan to use for the push operation (repository may include the address of a server if you don't plan to upload your image to docker hub).

$  ./tdskt -p application 421ad07f-5bfe-4cd3-83eb-8266e1d08312 setprop props.tag myrepository/mytag common
$ ./tdskt -p application 421ad07f-5bfe-4cd3-83eb-8266e1d08312 push release myuser mypassword              ✔
The push refers to repository [docker.io/myrepository/mytag]
01ff42507560: Preparing
cc9427b89fd9: Preparing
c114edb48deb: Preparing
...
latest: digest: sha256:17f5583a93f390914c9549274af4ee157cc198228493f5fc2e961281055fab1e size: 2610
100%
operation completed successfully

If you want to publish your application on Torizon OTA you have to register on the OTA Application and obtain the credential.zip file required to sign and publish your updates. You also have to set the "otapackagename" property to a valid package name and provide username and password required to publish your container on docker registry.

$ ./tdskt -p application 8057ba96-528e-447d-b56d-fb0d0e6970e3 setprop otapackagename mypackage0000
$ ./tdskt -p application 8057ba96-528e-447d-b56d-fb0d0e6970e3 publish ./credentials.zip myusername mypassword
Publishing application to Torizon OTA...
Generating docker-compose base file...
Pushing release container to docker registry...
The push refers to repository [docker.io/myrepository/mytag]
01ff42507560: Preparing
...
latest: digest: sha256:17f5583a93f390914c9549274af4ee157cc198228493f5fc2e961281055fab1e size: 2610
Fixing image IDs in compose file...
Checking that images are accessible...
...
Generating OTA-compatible docker-compose file...
Pushing new package to OTA server...
3.1: Pulling from torizon/torizoncore-builder
Digest: sha256:5ac1981670e227259a115fed6e69577ab953fe761916ae07f3e563dbd1f7a280
Status: Image is up to date for torizon/torizoncore-builder:3.1
operation completed successfully

Other application-specific subcommands

Some containers may support SSH connectivity for debugging or remote access purposes. The "key" subcommand allow you to get the path of the SSH key you can use to connect.

$ ./tdskt -p application 8057ba96-528e-447d-b56d-fb0d0e6970e3 key
/home/user/dummy/appconfig_0/id_rsa

In the same way the sync command can be used to sync a folder from the host PC (or SDK container) to the target container. This requires that the target platform exposes an SSH interface and supports rsync.

$ ./tdskt -p application 8057ba96-528e-447d-b56d-fb0d0e6970e3 sync                                                 25 ↵
usage: tdskt application application-id sync [-h] [--sdk] source-folder configuration device-id destination-folder
tdskt application application-id sync: error: the following arguments are required: source-folder, configuration, device-id, destination-folder

An application configuration object contains a unique id and security keys. Sharing them on an internal repository may not be an issue, but sharing them between different teams or in a public open-source code base may be a problem. The ide-backend provides a feature to remove those IDs from configuration. This makes the application object unusable, but new IDs and keys will be recreated when the configuration is loaded. This means that a different user rebuilding the application from code and configuration will get the same functional results but different IDs, preventing him from overwriting the original release.

The "reseal" command can be used to remove ids and keys from an application before publishing its code:

$ ./tdskt -p application 8057ba96-528e-447d-b56d-fb0d0e6970e3 reseal
Application has been resealed. It should not be used for any further operation, otherwise keys will be regenerated.

Validate parameters

Some of the configuration objects (devices, applications) have properties and parameters that can be configured by users. The API provides functions to check if a value is valid for a specific parameter. This avoids the need of saving and applying the changes just to check if a specific value is valid or not. The "validate" subcommand allows you to check if a value is valid for device properties.

$ ./tdskt device 06827787 validate hostname bla@blabla
Value does not appear to be a valid hostname or IP address.
$ ./tdskt device 06827787 validate hostname 192.168.1.34
$

Applications support more complex parameters and so the options of "validate" subcommand change depending on the kind of parameter to be validated. You can have simple entries, like dockercomposefile that can be validated as parameters:

$ ./tdskt application f7ab6402-aec5-44ac-8a09-708fbb226c21 validate parameter common dockercomposefile docker-compose.yml
$ ./tdskt application f7ab6402-aec5-44ac-8a09-708fbb226c21 validate parameter common dockercomposefile /##docker-compose.yml
Value does not appear to be a valid relative file path.

Some parameters (ex: networks and devices) are defined as lists of items, so you can validate an item and specify it's index (or -1 one for an item that will be appended to the list).

$ ./tdskt application f7ab6402-aec5-44ac-8a09-708fbb226c21 validate item common networks network01 -1
$ ./tdskt application f7ab6402-aec5-44ac-8a09-708fbb226c21 validate item common networks network@@01 -1
Value does not appear to be a valid docker object name.
$ ./tdskt application f7ab6402-aec5-44ac-8a09-708fbb226c21 validate item common devices /bla/blabla
Device name should start with /dev

Other elements are defined as dictionaries (props, ports, extraparms). In this case you'll have to use the entry method and specify also the key.

$ ./tdskt application f7ab6402-aec5-44ac-8a09-708fbb226c21 validate entry common extraparms privileged true
$ ./tdskt application f7ab6402-aec5-44ac-8a09-708fbb226c21 validate entry common extraparms privileged blabla
YAML validation error: 'blabla' is not of type 'boolean'

Failed validating 'type' in schema:
    {'type': 'boolean'}

On instance:
    'blabla'

Additional features

The API supports also some generic features that can be useful for developing a more complete and easy to use IDE extension.

Pull updates for base containers

The "pull" command can be used to pull base containers for the different platforms. This will require some time but will then speed-up further build operations. It can also be used to ensure that all base containers are up to date.

$ ./tdskt -p pull
Downloading torizon/binfmt
Downloading torizon/dotnet-debug:2-3.1
2%
...
100%
operation completed successfully

Enable ARM emulation

ARM (and other architectures) emulation is enabled by default when docker runs on Windows. On Linux you have to enable it explicitely using binfmt. This operation can be automated using the "enableemulation" command.

$  ./tdskt -p enableemulation                                                                                          ✔
operation completed successfully

If emulation is already enabled the success message won't be printed out, but the tool will not return an error, since this will not generate any side effect.

Version information

The "version" command can be used to check the version of the backend:

$ ./tdskt version                                                                                                   ✔
Property     Value
api_version  1.1.4
app_version  1.0.0

the "dockerversion" command can be used to check the version of the docker runtime and can also be used to quickly check that the backend can communicate with docker with no issues:

$ ./tdskt dockerversion                                                                                               ✔
Property         Value
platform         name  Docker Engine - Community
components       {'details': {}, 'name': 'Engine', 'version': '20.10.9'}
                 {'details': {}, 'name': 'containerd', 'version': '1.4.11'}
                 {'details': {}, 'name': 'runc', 'version': '1.0.2'}
                 {'details': {}, 'name': 'docker-init', 'version': '0.19.0'}
version          20.10.9
api_version      1.41
min_api_version  1.12
git_commit       79ea9d3
go_version       go1.16.8
os               linux
arch             amd64
kernel_version   5.4.0-87-generic
experimental     True
build_time       2021-10-04T16:06:34.000000000+00:00