From 8bd489804ebf9b6fab81da05e451d72619413405 Mon Sep 17 00:00:00 2001 From: Katrina Prosise Date: Wed, 13 Dec 2023 10:28:29 -0500 Subject: [PATCH 1/2] Cleanup ota reference manual Grammar and style changes made. Updates to content may be required. QA: Ran linter in editor. Checked rendered output in browser, no rendering issues spotted. Linkcheck encountering issues not believed to be associated with any changes made in this commit. This commit addresses FFTK-2792 This commit applies to FFTK-2730 This commit applies to FFTK-2732 This commit applies to FFTK-2731 This commit applies to FFTK-1702 This commit applies to FFTK-988 Signed-off-by: Katrina Prosise --- .../reference-manual/ota/advanced-tagging.rst | 71 +++++---- .../reference-manual/ota/aktualizr-lite.rst | 38 +++-- source/reference-manual/ota/ci-targets.rst | 52 +++---- source/reference-manual/ota/configuring.rst | 10 +- source/reference-manual/ota/device-tags.rst | 35 +++-- source/reference-manual/ota/fioconfig.rst | 77 +++++----- .../reference-manual/ota/ota-architecture.rst | 57 ++++---- source/reference-manual/ota/ota.rst | 2 + .../ota/production-targets.rst | 136 +++++++++--------- source/reference-manual/ota/static-deltas.rst | 68 ++++----- source/reference-manual/ota/targets.rst | 67 +++++---- .../reference-manual/ota/update-rollback.rst | 45 +++--- 12 files changed, 325 insertions(+), 333 deletions(-) diff --git a/source/reference-manual/ota/advanced-tagging.rst b/source/reference-manual/ota/advanced-tagging.rst index cec5a66bc..e5c5d390c 100644 --- a/source/reference-manual/ota/advanced-tagging.rst +++ b/source/reference-manual/ota/advanced-tagging.rst @@ -3,26 +3,26 @@ Advanced Tagging ================ -Some users incorporate non-trivial workflows that can require advanced tagging -techniques. These workflows can be handled in the :ref:`ref-factory-definition`. +You may sometimes need to incorporate a non-trivial workflow requiring advanced tagging techniques. +These workflows are handled in the :ref:`ref-factory-definition`. Terminology ----------- -**Platform Build** - A build created by a change to the LmP (lmp-manifest.git -or meta-subscriber-overrides.git). This is the base OS image. +**Platform Build**: A build created by a change in ``lmp-manifest.git`` or ``meta-subscriber-overrides.git``. +This is the base OS image. -**Container Build** - A build created by a change to containers.git. +**Container Build**: A build created by a change to ``containers.git``. -**Target** - This an entry in a factory's TUF targets.json file. It represents -what should be thought of as an immutable combination of the Platform build's -OSTree hash + the output of a Container build. +**Target**: An entry in a Factory's TUF ``targets.json``. +It represents an immutable combination of the Platform build's OSTree hash with the output of a container build. -**Tag** - A Target has a "custom" section where with a list of Tags. The -tags can be used to say things like "this is a development build" -or this is a "production" build. +**Tag**: A Target has a "custom" section where with a list of Tags. +**Tag:**: A user defined attribute of a Target designating its intended usage. +Tags are defined in the "custom" section of a Target. +They can be used to e.g. distinguish between "development" versus "production" builds. -Scenario 1: A new platform build that re-uses containers +Scenario 1: A New Platform Build That Re-Uses Containers -------------------------------------------------------- A Factory is set up with the normal ``main`` branch:: @@ -36,8 +36,8 @@ A Factory is set up with the normal ``main`` branch:: refs/heads/main: - tag: main -You'd like to introduce a new ``stable`` branch from the LmP but have it use -the latest containers from master. This can be done with:: +You want to introduce a new ``stable`` branch from the LmP, but have it use the latest containers from ``main``. +This can be done with:: lmp: tagging: @@ -52,7 +52,7 @@ the latest containers from master. This can be done with:: - tag: main - tag: stable -Consider this pseudo targets example:: +Consider this pseudo Targets example:: targets: build-1: @@ -64,10 +64,10 @@ Consider this pseudo targets example:: compose-apps: foo:v2, bar:v2 tags: main -If a change to the stable branch was pushed to the LmP, a new -target, build-3, would be added. The build logic would then look through -the targets list to find the most recent ``main`` target so that -it can copy those compose-apps. This would result in a new target:: +If a change to the stable branch was pushed to the LmP, a new Target, ``build-3``, would be added. +The build logic would then look through the Targets list to find the most recent ``main`` Target. +It can then copy the compose-apps from that most recent Target. +This would result in a new Target:: build-3: ostree-hash: NEWHASH @@ -75,7 +75,7 @@ it can copy those compose-apps. This would result in a new target:: tags: stable On the other hand, there might also be a new container build for ``main``. -In this case the build logic will produce two new targets:: +In this case, the build logic will produce two new Targets:: build-4: # for stable it will be based on build-3 ostree-hash: NEWHASH @@ -87,13 +87,13 @@ In this case the build logic will produce two new targets:: compose-apps: foo:v3, bar:v3 tags: main -Scenario 2: Multiple container builds using the same platform +Scenario 2: Multiple Container Builds Using the Same Platform ------------------------------------------------------------- -This scenario is the reverse of the previous one. A factory might have a -platform build tagged with ``main``. However, there are two versions of -containers being worked on: ``main`` and ``foo``. This could be handled -with:: +This scenario is the reverse of the first one. +A Factory might have a platform build tagged with ``main``. +However, there are two versions of containers being worked on: ``main`` and ``foo``. +This could be handled with:: lmp: tagging: @@ -108,13 +108,13 @@ with:: - tag: foo inherit: main -Scenario 3: Multiple teams, different cadences +Scenario 3: Multiple Teams, Different Cadences ---------------------------------------------- -Some organizations may have separate core platform and application teams. In -this scenario, it may be desirable to let each team move at their own decoupled -paces. Furthermore, the application team might have stages(branches) of -development they are working on. This could be handled with something like:: +Your organization may have separate core platform and application teams. +In this scenario, it may be desirable to let each team move at their own pace. +Furthermore, the application team might have stages(branches) of development they are working on. +This can be handled with something like:: lmp: tagging: @@ -131,12 +131,11 @@ development they are working on. This could be handled with something like:: - tag: qa inherit: main -This scenario is going to produce ``main`` tagged builds that have no -containers, but can be generically verified. Then each containers.git branch -will build Targets and grab the latest ``main`` tag to base its platform -on. **NOTE:** Changes to ``main`` don't cause new container builds. In -order to get a container's branch updated to the latest ``main`` a user -would need to push an empty commit to containers.git to trigger a new build:: +This will produce ``main`` tagged builds that have no containers, but can be generically verified. +Then, each ``containers.git`` branch will build Targets and grab the latest ``main`` tag to base its platform on. + +It is important to note that changes to ``main`` do not cause new container builds. +In order to get a container's branch updated to the latest ``main``, push an empty commit to ``containers.git`` to trigger a new build:: # from branch qa git commit --allow-empty -m 'Pull in latest platform changes from main' diff --git a/source/reference-manual/ota/aktualizr-lite.rst b/source/reference-manual/ota/aktualizr-lite.rst index 5882d767b..e68941307 100644 --- a/source/reference-manual/ota/aktualizr-lite.rst +++ b/source/reference-manual/ota/aktualizr-lite.rst @@ -1,9 +1,11 @@ .. _ref-aktualizr-lite: -aktualizr-lite +Aktualizr-Lite ============== -The default OTA client shipped with the Linux microPlatform is ``aktualizr-lite``. This client is a build variant of the Aktualizr project. It is targeting users who wish to have the security aspects of TUF but do not want the complexity of Uptane. +The default OTA client shipped with the Linux® microPlatform is ``aktualizr-lite``. +This client is a build variant of the Aktualizr project. +It is for those who wish to have the security aspects of TUF, but without the complexity of Uptane. .. figure:: /_static/diagrams/aktualizr-lite/aktualizr-lite.png :align: center @@ -14,7 +16,9 @@ There are two modes ``aktualizr-lite`` supports. Daemon Mode (Default) --------------------- -This is the default mode of ``aktualizr-lite`` in the Linux microPlatform. It is a systemd service, which is enabled by default on Community Factory images. Additionally, the daemon will only be enabled in a Personal or Enterprise factory after ``lmp-device-register`` has sucessfully registered your device. The daemon will periodically check for new updates, and apply them when found. +This is the default mode of ``aktualizr-lite`` in the Linux microPlatform. +The daemon will only be enabled in a Factory after ``lmp-device-register`` has successfully registered your device. +The daemon periodically checks for new updates, and applies them when found. To disable daemon mode: @@ -49,49 +53,55 @@ Disabling daemon mode is not recommended nor supported, but running ``aktualizr- View Current Status ~~~~~~~~~~~~~~~~~~~ -You can run ``sudo aktualizr-lite status`` to view the current status of the device. +To view the current status of the device:: + + sudo aktualizr-lite status Fetch and List Updates ~~~~~~~~~~~~~~~~~~~~~~ -This command will refresh the targets metadata from the OTA server, and present you with a list of available targets which can be applied. +This will refresh the Targets metadata from the OTA server, and present you with a list of available Targets:: -``sudo aktualizr-lite list`` + sudo aktualizr-lite list Apply Latest Update ~~~~~~~~~~~~~~~~~~~ -This command will apply the latest available update to the device. This includes both OSTree and Docker app targets. +This will apply the latest available update to the device. +This includes both OSTree and Docker app Targets:: -``sudo aktualizr-lite update`` + sudo aktualizr-lite update Apply Specific Update ~~~~~~~~~~~~~~~~~~~~~ -If you would like to update to a specific build number, you can use the following command. +To update to a specific build number:: ``sudo aktualizr-lite update --update-name `` .. note:: - This can only be performed when the original and update targets are under the same tag. In case the update is tagged differently, it is required to switch tags before running this command. + This can only be performed when the original and update Targets are under the same tag. + In case the update is tagged differently, it is required to switch tags before running this command. Configuration ------------- -Configuration update methods +Configuration Update Methods ~~~~~~~~~~~~~~~~~~~~~~~~~~~~ * Editing ``/var/sota/sota.toml`` on a device -* Adding or editing an existing configuration snippet e.g. ``/etc/sota/conf.d/z-50-fioctl-01.toml`` on a device -* Running *fioctl* from any host ``fioctl devices config ``, see :ref:`ref-configuring-devices` for more details +* Adding or editing an existing configuration snippet, e.g. ``/etc/sota/conf.d/z-50-fioctl-01.toml`` on a device +* Running ``fioctl devices config `` from a host. + See :ref:`ref-configuring-devices` for more details. .. _ref-aktualizr-lite-params: Parameters ~~~~~~~~~~ -The following are aktualizr-repo's configuration parameters that can be useful to play with, the presented values are the default one. +The following are aktualizr-repo's configuration parameters that can be useful to modify. +The presented values are the default one. .. code-block:: diff --git a/source/reference-manual/ota/ci-targets.rst b/source/reference-manual/ota/ci-targets.rst index 892b61fce..380f812cc 100644 --- a/source/reference-manual/ota/ci-targets.rst +++ b/source/reference-manual/ota/ci-targets.rst @@ -3,29 +3,25 @@ CI Targets ========== -The point of FoundriesFactory is to create Targets. The magic -is how a ``git push`` can make this all happen. Because -of how easy these are to create, there is another type of Target, -:ref:`Production Targets `, that are intended -to be used for production devices. However, it's almost always -originally created by CI when: +The point of a Factory is to create Targets. +A ``git push`` is all that is required to trigger a Target to be built. - * A change is pushed to source.foundries.io - * A CI job is triggered in ci.foundries.io - * The CI job signs the resulting TUF ``targets.json`` with the Factory's - "online" targets signing key. +There is another type of Target, :ref:`Production Targets `, that are intended to be used for production devices. +However, it is almost always originally created by the CI when: -The online targets signing key ID can be seen in the TUF root -metadata: + * A change is pushed to ``source.foundries.io`` + * A CI job is triggered in ``ci.foundries.io`` + * The CI job signs the resulting TUF ``targets.json`` with the Factory's "online" Targets signing key. + +The online Targets signing key ID can be seen in the TUF root metadata: .. code-block:: bash $ fioctl get https://api.foundries.io/ota/repo//api/v1/user_repo/root.json \ | jq '.signed.roles.targets.keyids[0]' -Due to the number of changes and development branches a typical -customer may have, the TUF targets metadata can grow to include large -numbers of Targets. There are two ways these are dealt with: +Due to all the changes and branches you may have, the TUF Targets metadata can grow to include a large number of Targets. +There are two ways this can be dealt with: * Condensed Targets * Target Pruning @@ -35,23 +31,20 @@ numbers of Targets. There are two ways these are dealt with: Condensed Targets ----------------- -Each device is configured to take updates for Targets that include -a specific tag. Because of this, the most of the Targets in the -CI ``targets.json`` aren't relevant and can be ignored by the device. -In order to provide smaller TUF metadata payloads, the Foundries -back-end employs a trick referred to as "condensed targets". +Each device is configured to take updates for Targets that include a specific tag. +Because of this, most of the Targets in ``targets.json`` are not relevant for any given device and can be ignored by it. +In order to provide smaller TUF metadata payloads, the backend employs what is referred to as "condensed Targets". -Condensed Targets are produced by taking the raw CI version and then -producing condensed versions for each unique tag. For example, the -raw targets.json might include:: +Condensed Targets are produced by taking the raw CI version, and then producing condensed versions for each unique tag. +For example, a raw ``targets.json`` might include:: version=1, tag=master version=2, tag=devel version=3, tag=devel version=4, tag=devel,experimental -The back-end will actually produce three different condensed versions -that are each signed with the Factory's online targets signing key:: +The back-end will actually produce three different condensed versions. +Each one is signed with the Factory's online Targets signing key:: # targets-master.json version=1, tag=master @@ -64,14 +57,13 @@ that are each signed with the Factory's online targets signing key:: version=3, tag=experimental version=4, tag=experimental -The :ref:`device gateway ` is then able to serve -an optimized targets.json to each CI device. +The :ref:`device gateway ` is then able to serve an optimized ``targets.json`` to each CI device. Target Pruning -------------- -Each successful build appends a Target to targets.json. Eventually -it grows too large and users will see an error in CI:: +Each successful build appends a Target to ``targets.json``. +Eventually it can grow too large, and you would see an error in CI:: Publishing local TUF targets to the remote TUF repository == 2022-03-24 00:44:18 Running: garage-sign targets push --repo /root/tmp.gkfCEF @@ -79,4 +71,4 @@ it grows too large and users will see an error in CI:: | com.advancedtelematic.libtuf.http.CliHttpClient$CliHttpClientError: ReposerverHttpClient|PUT|http/413|https://api.foundries.io/ota/repo/andy-corp/api/v1/user_repo/targets%7C | 413 Request Entity Too Large -When this happens, it's time to :ref:`prune targets `. +When this happens, it is time to :ref:`prune targets `. diff --git a/source/reference-manual/ota/configuring.rst b/source/reference-manual/ota/configuring.rst index 2111ef3c4..1b324b6e7 100644 --- a/source/reference-manual/ota/configuring.rst +++ b/source/reference-manual/ota/configuring.rst @@ -82,10 +82,16 @@ Two interesting things that can be done with this include ``on-changed`` and ``u } EOF -The ``on-changed`` parameter allows to run commands upon receiving configuration fragments. By default, it only runs commands from ``/usr/share/fioconfig/handlers``, so a custom handler should be created in this folder, see `fioconfig_git.bb `_ for reference. +The ``on-changed`` parameter allows to run commands upon receiving configuration fragments. +The `fioconfig` may only run "trusted" scripts from the ``/usr/share/fioconfig/handlers/`` folder to protect devices from arbitrary script execution. +As such, a custom handler should be created in this folder. +See `fioconfig_git.bb `_ for reference. .. tip:: - It is possible to use the ``on-changed`` parameter to run commands outside of the ``/usr/share/fioconfig/handlers`` folder by running ``fioconfig daemon --unsafe-handlers``. This would allow running configurations as:: + For testing purposes, it is possible to use the ``on-changed`` parameter to run commands outside of the ``/usr/share/fioconfig/handlers`` folder. + This is done by running ``fioconfig daemon --unsafe-handlers``. + We do not recommend doing that in production. + This allows running configurations as:: cat >tmp.json <``. This is tmpfs and is only available when the -the device is running. +Configuration data is assumed to be sensitive unless ``unencrypted`` is explicitly set to true in a configuration JSON payload. +Fleet-wide configuration is uploaded to the server unencrypted, then encrypted with a secret symmetric key at back-end. +Note: even in unencrypted form the configuration is still protected by the TLS encryption at both upload and fetch time. +When a device requests this information, the server will decrypt it and deliver it over its TLS encrypted connection. + +Device configuration has a better approach. +The contents of each config file gets encrypted with the device's public key, using the `Elliptical Curve Integrated Encryption Scheme`_ by Fioctl. +This means only someone owning the device's private key, and the device itself, can decrypt the value; +Foundries.io, without the symmetric key, can not. + +Your Factory tracksthe last 10 config changes made to both the fleet, and each device. +Your Factory keeps track of the last 10 config changes made to each device or device group config. + +.. important:: + The maximum size of a total "config" (files and data combined) is 1Mb. + +A device stores its config persistently under ``/var/sota/config.encrypted``. +Device specific config options get encrypted. +At boot, each file in the config is extracted into ``/var/run/secrets/``. +This is tmpfs, only available when the device is running. Implementation ~~~~~~~~~~~~~~ -Configuration is managed by fioctl which communicates via the -Foundries.io `REST API`_. +Configuration is managed by Fioctl, which communicates via the Foundries.io `REST API`_. + +LmP devices run a configuration daemon, `fioconfig`_. +This daemon checks in with the server every 5 minutes. +The HTTP request includes a timestamp of its current configuration. +The server will send back one of two responses: -LmP devices run a configuration daemon, `fioconfig`_. This daemon checks in -with the server every 5 minutes. The HTTP request includes a timestamp of its -current configuration. The server will send back one of two responses: + * **304** — This means nothing has changed since the last request. + There will be an empty response body. - * **304** - This means nothing has changed since the last request. There will - be an empty response body. + * **200** — Something has changed since the last request. + The response body will be the updated configuration. - * **200** - Something has changed since the last request. The response body will - be the updated configuration. + * **204** — No config for a given device. Fioconfig’s logic for a configuration change is: - * Save "encrypted" file to /var/sota (fleet-wide and "unencrypted" values - aren't encrypted). + * Save "encrypted" file to `/var/sota` (fleet-wide and "unencrypted" values are not encrypted). * For each file in the new configuration: * Decrypt if needed - * If different from ``/var/run/secrets/`` + * If different from ``/var/run/secrets/``, then: - * Write new file - * Call the the on-changed handler (if provided) + * Write a new file + * Call the on-changed handler (if provided). -Since ``/var/run/secrets`` is lost between boots, a systemd oneshot job is -invoked during the boot process to extract the "encrypted" persistent -configuration. It will call "onchanged" handlers. +Since ``/var/run/secrets`` is lost between boots, a systemd oneshot job is invoked during the boot process, to extract the "encrypted" persistent configuration. +It will call "onchanged" handlers. Diagram ~~~~~~~ + :: fioctl diff --git a/source/reference-manual/ota/ota-architecture.rst b/source/reference-manual/ota/ota-architecture.rst index 661dfa809..804dbf350 100644 --- a/source/reference-manual/ota/ota-architecture.rst +++ b/source/reference-manual/ota/ota-architecture.rst @@ -5,7 +5,7 @@ Architecture Overview ===================== -At a high level the system consists of three entities: +At a high level, the system consists of three entities: * LmP Devices @@ -16,35 +16,32 @@ At a high level the system consists of three entities: * The device gateway * The Rest API - - tooling like fioctl and app.foundries.io use + - tooling like Fioctl and ``app.foundries.io`` use. .. figure:: /_static/ota-arch.png :align: center :scale: 70 % :alt: OTA architecture diagram -Devices talk to the device gateway using `mutual TLS`_. The device gateway -provides a set of REST APIs to support -:ref:`aktualizr-lite `, -:ref:`fioconfig `, -:ref:`device testing `, and Docker authentication. Aktualizr-lite -and fioconfig run as separate daemons that are periodically polling the -device gateway with HTTP GET requests on configurable intervals. +Devices talk to the device gateway using `mutual TLS`_. +The device gateway provides a set of REST APIs to support: -Due to the fact devices are polling the server, REST API changes requested by -tooling like fioctl happen asynchronously. +* :ref:`aktualizr-lite ` +* :ref:`fioconfig ` +* :ref:`device testing ` +* Docker authentication. -How A Device Uses Security Hardware -~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ +Aktualizr-lite and fioconfig run as separate daemons, periodically polling the device gateway with ``HTTP GET`` requests at configurable intervals. -An LmP device uses Hardware Security Modules, Trusted Platform Module (TPM) devices or -Trusted Execution Environments via the Public-Key Cryptography Standards #11 (PKCS#11) API. +Due to the fact devices are polling the server, REST API changes requested by Fioctl® tooling happen asynchronously. -They provide the guarantee that secrets will not be leaked and that -communications will be secure. Certain keys will be provisioned during device -manufacturing. +How A Device Uses Security Hardware +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ +An LmP device uses Hardware Security Modules (HSM), Trusted Platform Module (TPM) devices, or Trusted Execution Environments (TEE) via the Public-Key Cryptography Standards #11 (PKCS#11) API. +They provide the guarantee that secrets will not be leaked and that communications will be secure. +Certain keys will be provisioned during device manufacturing. .. figure:: /_static/lmp-device-arch.png :align: center @@ -55,23 +52,21 @@ manufacturing. How A Device Finds Updates ~~~~~~~~~~~~~~~~~~~~~~~~~~ -Aktualizr-lite uses `TUF`_ to find and validate available Targets that -a device may install. Aktualizr-lite will periodically check-in using -this high level logic: +Aktualizr-lite uses `TUF`_ to find and validate the available Targets that a device may install. +Aktualizr-lite will periodically check-in using this high level logic: - * Ask if a new root.json exists. This allows a device to know about - key rotations before going further. This call is almost always going - to result in an HTTP 404 response. + * Asks if a ``new root.json`` exists. + This allows a device to know about key rotations before going further. + This call is almost always going to result in an HTTP 404 response. - * Ask for the timestamp.json metadata. If this file hasn't changed, - there's no need to ask for more metadata - nothing has changed. + * Asks for the ``timestamp.json`` metadata. + If this file has not changed, there is no need to ask for more metadata—nothing has changed. - * Ask for the snapshot.json metadata. If this file hasn't changed, - there's no need to ask for more metadata - the targets have - not changed. + * Asks for the ``snapshot.json`` metadata. + If this file has not changed, there is no need to ask for more metadata—the targets have not changed. - * Ask for the targets.json metadata. At this point the device can - see if a new Target is available for installation. + * Ask for the ``targets.json`` metadata. + At this point, the device can see if a new Target is available for installation. .. _mutual TLS: https://codeburst.io/mutual-tls-authentication-mtls-de-mystified-11fa2a52e9cf diff --git a/source/reference-manual/ota/ota.rst b/source/reference-manual/ota/ota.rst index 8c0f10541..32eb68554 100644 --- a/source/reference-manual/ota/ota.rst +++ b/source/reference-manual/ota/ota.rst @@ -3,6 +3,8 @@ Over the Air Updates ==================== +This section provides reference information and in depth details on Over the Air (OTA) Updates for your Factory. + .. toctree:: :maxdepth: 1 diff --git a/source/reference-manual/ota/production-targets.rst b/source/reference-manual/ota/production-targets.rst index 2b990372c..32555654e 100644 --- a/source/reference-manual/ota/production-targets.rst +++ b/source/reference-manual/ota/production-targets.rst @@ -3,75 +3,76 @@ Production Targets ================== -As a part of going to production, a user needs to define two distinct sets of devices: test versus production. -A **production device** is defined by a presence of a special attribute in its public certificate, -which is set at the :ref:`device registration time `. -If that attribute is absent, it designates a **test device**. +As a part of going to production,you need to define two distinct sets of devices: *test* versus *production*. +A **production device** is defined by a presence of a special attribute in its public certificate. +This is set at the :ref:`device registration time `. +If the attribute is absent, it designates a **test device**. -As noted in the :ref:`offline keys documentation`, -FoundriesFactory® leverages The Update Framework (TUF) to deliver Over-the-Air (OTA) Software Updates to devices. +As noted in the :ref:`offline keys documentation `, +FoundriesFactory® leverages The Update Framework (TUF) to deliver Over-the-Air (OTA) software updates to devices. Production versus test devices receive different copies of the TUF metadata: - Test devices receive :ref:`CI targets `. - Production devices receive **production targets**, managed through so-called **waves** — a FoundriesFactory way to release updates to production devices. -Production targets provide an increased security compared to CI targets. +Production Targets provide increased security compared to CI Targets. They must be signed by the user-owned targets signing key, in addition to the Foundries.io™ owned signing key. Normally, when the TUF targets metadata is updated, all devices see it and can start updating themselves. Waves allow Factory operators to control an exact time when devices see a new version of the TUF targets metadata. -.. note:: +.. important:: - Production devices can only be updated to production targets. - The opposite is also true, production targets can only be installed on production devices. - From the TUF perspective CI versus production TUF targets comprise two isolated sets of updates. + Production devices can only be updated to production Targets. + The opposite is also true, production Targets can only be installed on production devices. + From the TUF perspective CI versus production TUF Targets comprise two isolated sets of updates. - Within a scope of below paragraphs a term "device" always means a "production device". + Within the scope of the below paragraphs, "device" refers exclusively to a "production device". .. _ref-rm-wave: Performing a Production OTA --------------------------- -A customer should define a process to select CI builds which need to be delivered to production devices. +First define a process to select CI builds which need to be delivered to production devices. Every user performing production OTAs should generate their personal :ref:`offline TUF targets key ` to sign production Targets. -Let's assume a user selected a CI build version 42 as ready to be run in production. -To start the production release process, a user would create a new wave using the below command:: +Let's assume you selected CI build version 42 as ready to be run in production. +To start the production release process, create a new wave using the below command:: # fioctl waves init -k fioctl waves init -k /absolute/path/to/targets.only.key.tgz v2.0-update 42 production This creates a new TUF targets role version for production devices which listen to OTA updates for the ``production`` tag. -That TUF targets role only includes a single Target from CI build (in above example, that target version is 42). +That TUF targets role only includes a single Target from CI build (in the above example, that Target version is 42). .. note:: - We recommend that a user generates :ref:`OSTree static deltas` before rolling out waves to devices. + We recommend generating :ref:`OSTree Static Deltas ` before rolling out waves to devices. Static Deltas will optimize your OTA update download. -Once created, a new wave can be rolled out to Factory production devices, all at once or in phases. +Once created, a new wave can be rolled out to Factory production devices. +This can be either all at once, or in phases. There are several ways how a wave can be rolled out: - To a subset of devices in a specific device group. - To all devices in a specific device group. -- To a subset of devices in a Factory (potentially, across several device groups, and including group-less devices). +- To a subset of devices in a Factory (potentially, across several device groups, including group-less devices). - To all devices in a Factory. -We recommend to first roll out a wave to a dedicated device group, which contains a small number of production devices. +We recommend you first roll out a wave to a dedicated device group, which contains a small number of production devices. Another good option is to roll out a wave to a small subset of devices in a bigger device group. -Let's assume a user wants to first roll out a new ``v2.0-update`` wave to a device group called ``canary``. +Let us assume you want to first roll out a new ``v2.0-update`` wave to a device group called ``canary``. This can be done using the below command:: fioctl waves rollout v2.0-update canary -A user can roll out a wave in as many phases as their workflow requires, +You can roll out a wave in as many phases as your workflow requires, before making the software update generally available. -For example, a user may decide to roll out a wave to device group ``us-east-1`` in several chunks, +For example, you may decide to roll out a wave to device group ``us-east-1`` in several chunks, after enough devices in group ``canary`` were updated successfully. -To do that, a user would run the below command sequence:: +To do that, you would run the below command sequence:: fioctl waves rollout v2.0-update us-east-1 --limit=10 fioctl waves rollout v2.0-update us-east-1 --limit=50 @@ -80,12 +81,11 @@ To do that, a user would run the below command sequence:: The above command chain rolls out a wave to 10 devices in the ``us-east-1`` group, then to 50 more devices (60 total), and finally to all remaining devices in that group. -A user may also want to roll out a wave to a subset of devices in entire fleet, across several device groups. -That can be accomplished by the below command:: +You may also want to roll out a wave to a subset of devices in entire fleet, across several device groups:: fioctl waves rollout v2.0-update --limit=5 -It is possible to examine a list of devices that would be updated by a rollout command, without actually performing it:: +It is possible to examine a list of devices that *would* be updated by a rollout command, without actually performing it:: fioctl waves rollout v2.0-update --limit=5 --dry-run --print-uuids @@ -93,25 +93,25 @@ It is possible to examine a list of devices that would be updated by a rollout c Keep in mind that the device selection is pseudo-random, and can vary from one command run to another. -A user can then inspect and amend that list of devices, and pass it back to the rollout command. -Alternatively, a user can provide their own choice of device UUIDs to update, like in the below command:: +You can then inspect and amend that list of devices, and pass it back to the rollout command. +Alternatively, you can provide the device UUIDs to update:: fioctl waves rollout v2.0-update --uuids=ab8ecb00-8ed4-42ff-90b2-815b371c0f86,7a733e81-f948-43a9-a358-56f3deb5f184 -Please, check the ``fioctl waves rollout --help`` command for all available options, +Check the ``fioctl waves rollout --help`` command for all available options, or look at the :ref:`Advanced Usage ` for more complex workflows. Hopefully, they should suit your specific production release lifecycle needs. -To monitor the status of your Factory OTA updates status use the ``fioctl status`` command. -FoundriesFactory also provides a dedicated command to monitor the wave OTA updates status — ``fioctl wave status``. +To monitor the status of your Factory OTA updates, use the ``fioctl status`` command. +FoundriesFactory also provides a dedicated command to monitor the wave status — ``fioctl wave status``. -Eventually, a user may decide that a new software release (represented by a wave) is good enough to be generally available. +Eventually, you may decide that a new software release (represented by a wave) is fit be generally available. In this case, wave TUF targets need to be copied into production TUF targets for a specific tag. In our example that is accomplished by using the below command:: fioctl waves complete v2.0-update -Alternatively, if a wave progresses badly, a user can cancel it using the below command (unless a wave is already completed):: +Alternatively, if a wave progresses badly, you can cancel it using the below command (unless a wave is already completed):: fioctl waves cancel v2.0-update @@ -128,24 +128,24 @@ Advanced Usage -------------- The FoundriesFactory ``fioctl waves rollout`` command allows implementing various release workflows. -This section focuses on some popular scenarios which are supported by it. +This section focuses on supported, popular scenarios. Releasing to Canary Devices +++++++++++++++++++++++++++ -First, let's consider the most trivial use case — a Factory with a small device fleet. -In this case, the two most convenient ways to deliver updates to your devices in a controlled way are: +Consider the most trivial use case — a Factory with a small device fleet. +In this case, the two most convenient ways to deliver updates in a controlled way are: - Delivering updates to device groups, defined before the rollout. - Delivering updates to ad-hoc device sets, generated at the rollout time. -Each of the above options has its pros and cons. -Either way, the idea is to incrementally deliver the updates to your device fleet in chunks, which gradually increase in size. +Each option has pros and cons. +In both, the idea is to incrementally deliver the updates to your fleet in chunks, which gradually increase in size. Using device groups, a typical setup would look this way. Assume you have a fleet of 100 devices. -We recommend to split up 2 device group out of that fleet: e.g. "canary" having 5 devices, "beta" having 20 devices. -Canary devices would usually be those that are easier to reach out to in case of any troubles during an update. +We recommend splitting up 2 device groups out of that fleet: e.g. "canary" having 5 devices, "beta" having 20 devices. +Canary devices would be those that are easier to reach out to in case of any issues during an update. Having done that, a regular update rollout process would look like this:: fioctl waves rollout v2.0-update --group canary @@ -154,22 +154,22 @@ Having done that, a regular update rollout process would look like this:: .. note:: - It is up to the user to define the acceptance criterion for going to the next phase of the rollout. - Usually, a user would run the ``fioctl waves status`` command several times during that period. - That allows a user to watch a nearly real-time picture of how the update is going on fleet-wide. - Apparently, there must be a *wait and watch* period after each rollout command before proceeding to the next one. + It is up you to define the acceptance criterion for going to the next phase of the rollout. + Usually, you would run the ``fioctl waves status`` command several times during that period. + That allows you to watch a nearly real-time picture of how the update is going fleet-wide. + There must be a *wait and watch* period after each rollout command before proceeding to the next one. -For the same example, a user might opt to not use device groups but still rollout the update in phases. +For the same example, you might opt to not use device groups, but still rollout the update in phases. An equivalent way of doing this using randomized device fleet partitions may look like this:: fioctl waves rollout v2.0-update --limit 5 fioctl waves rollout v2.0-update --limit 20 fioctl waves complete -Alternatively, a user may create a file containing a comma-separated list of "canary" device UUIDs. -For example, let's assume the user created a file ``canary-devices.lst`` for those "canary" devices, -and prefers to keep the next rollout phase randomized. -This way is even closer to the use of device groups but does not necessitate their management:: +Alternatively, you may create a file containing a comma-separated list of "canary" device UUIDs. +For example, assume you created a file ``canary-devices.lst``, +and you prefer to keep the next rollout phase randomized. +This way is even closer to the use of device groups, but does not necessitate their management:: fioctl waves rollout v2.0-update --uuids @/path/to/canary-devices.lst fioctl waves rollout v2.0-update --limit 20 @@ -177,38 +177,38 @@ This way is even closer to the use of device groups but does not necessitate the When using dynamic randomized device partitions for the rollout process, Foundries.io APIs prioritize recently active devices over the offline devices. -It also filters out those devices, which already participate in a wave rollout. -That helps getting early response about how the rollout progresses, -and streamline the gradual update of your device fleet to a newer version. +It also filters out devices which already participated in a wave rollout. +That helps getting an early response about how the rollout is progressing, +and streamlines the gradual update of your device fleet to a newer version. Releasing to Large Device Fleets ++++++++++++++++++++++++++++++++ -Now let's look at a more complex examples of a rolling out an update to a large device fleet. -Let's assume that your Factory has 100'000 production devices to be updated within a wave. +Now let us look at a more complex example of rolling out an update to a large device fleet. +Assume that your Factory has 100'000 production devices to be updated within a wave. Normally, these devices will be split into a couple of groups according to your criteria. -Let's assume your device fleet is arranged according to geographic presence, -e.g. device groups are "us-east", "us-west", "eu-emea", and "apac". -Apparently, some or all of these device groups would still contain a large number of devices. -So, from a safety perspective, it is risky to deliver an update to the entirety of any group like that. +Also assume your device fleet is arranged according to geographic presence, +e.g. device groups are ``us-east``, ``us-west``, ``eu-emea``, and ``apac``. +Some, or all of these device groups, would still contain a large number of devices. +From a safety perspective, it is risky to deliver an update to the entirety of any group like that. A usual practice would be to apply the "canary" approach (described above) to every individual device group. -For example, commands below would roll out a wave to the "us-east" group in 4 incremental chunks:: +For example, commands below would roll out a wave to the ``us-east`` group in 4 incremental chunks:: fioctl waves rollout v2.0-update --group us-east --limit 5 fioctl waves rollout v2.0-update --group us-east --limit 20 fioctl waves rollout v2.0-update --group us-east --limit 100 fioctl waves rollout v2.0-update --group us-east -A user can then use the same technique to roll out an update to other device groups. +You can then use the same technique to roll out an update to other device groups. Integration with External Device Management Systems +++++++++++++++++++++++++++++++++++++++++++++++++++ An ability to specify the exact list of device UUIDs to the rollout command allows you to integrate it with your device management system. -For example, let's assume that your "eu-emea" device group is the biggest, containing 40'000 devices. +For example, let us assume that your ``eu-emea`` device group is the biggest, containing 40'000 devices. You might use your device management system to split that fleet into several partitions. -For that, you'd export the appropriate subsets of device UUIDs into one or more files in a Comma Separated Values (CSV) format. +For that, you would export the appropriate subsets of device UUIDs into one or more files in a Comma Separated Values (CSV) format. We support various characters as separators: a comma, a semicolon, and all sorts of newlines and white space. For example, let's assume a user prepared the following lists of device UUIDs:: @@ -234,10 +234,10 @@ Going Beyond Limits At Foundries.io, we care a lot about the speed of our APIs and scaling to large device fleets. That strategy binds us to define certain limits for specific device management operations. - One such limit is that users cannot pass more than 10'000 device UUIDs to a single rollout command. + One such limit is that you cannot pass more than 10'000 device UUIDs to a single rollout command. That constraint also implies that the ``--limit`` argument does not accept a value bigger than 10'000. It is still possible to pass more than 10'000 device UUIDs using several rollout commands. - Also, a user can roll out to the entire device group. + Also, you can roll out to the entire device group. When rolling out to a subset of devices using ``--limit`` argument, the "randomized" sample will exclude devices that were already updated to a wave version. @@ -248,7 +248,7 @@ Going Beyond Limits In particular, the same (not yet updated) device can be selected for the rollout several times. That precision loss allows us to keep the decision making speed reasonable, regardless of the number of devices in your Factory, theoretically scaling to infinity. - A user can restore a lossless precision by specifying both ``--uuids`` and ``--limit`` arguments, as described in an example above. + You can restore a lossless precision by specifying both ``--uuids`` and ``--limit`` arguments, as described in an example above. The techniques described above can be applied without using the ``--group`` argument. In this case, the rollout command will be applied to a subset of the entire device fleet. @@ -259,10 +259,10 @@ For example, the below commands roll out a wave to 5'000 devices in a ``pre-sele fioctl waves rollout v2.0-update --limit 1000 --uuids @/path/to/pre-selected.lst fioctl waves rollout v2.0-update --limit 3500 --uuids @/path/to/pre-selected.lst -A user can also dump a pre-selected device list into a file; then inspect, amend, and push it back to the rollout command:: +You can also dump a pre-selected device list into a file; then inspect, amend, and push it back to the rollout command:: fioctl waves rollout v2.0-update --limit 1000 --print-uuids >/path/to/pre-selected.lst # Open and edit /path/to/pre-selected.lst using your editor of choice. fioctl waves rollout v2.0-update --uuids >/path/to/pre-selected.lst -One way or another, the Fioctl® allows you to implement various processes to roll out updates to your Factory device fleet. +One way or another, Fioctl® allows you to implement various processes to roll out updates to your Factory's device fleet. diff --git a/source/reference-manual/ota/static-deltas.rst b/source/reference-manual/ota/static-deltas.rst index 97e94fa83..38c4ba782 100644 --- a/source/reference-manual/ota/static-deltas.rst +++ b/source/reference-manual/ota/static-deltas.rst @@ -3,32 +3,28 @@ OSTree Static Deltas ==================== -OSTree is implemented as a `Content Addressable Storage`_ system -inspired by Git. Both systems organize their objects in a tree-based -hierarchy. The client's job is to work through these trees of objects -and apply the correct changes. While Git has `smart protocol`_, -OSTree `does not`_. This can lead to certain types of OTAs being -really inefficient - the client will be requesting a large number -of files via HTTP requests. - -OSTree has a solution for this problem called static deltas. OSTree -can produce static deltas that are good balance between number -of files to download and the size of each file. For instance, -a 1.5G OTA might be split up into about 38 files that are each -about 30Mb. - -Generating static deltas +OSTree is implemented as a `Content Addressable Storage`_ system inspired by Git. +Both systems organize their objects in a tree-based hierarchy. +The client's job is to work through these trees of objects and apply the correct changes. +While Git has `smart protocol`_, OSTree `does not`_. +This can lead to certain types of OTAs being really inefficient — the client will be requesting a large number of files via HTTP requests. + +OSTree has a solution for this problem called static deltas. +OSTree can produce static deltas that are good balance between number of files to download and the size of each file. +For instance, a 1.5G OTA might be split up into about 38 files that are each about 30Mb. + +Generating Static Deltas ------------------------ -Fioctl® includes a command to help generate static deltas. Since -these deltas can sometimes require a bit of processing power and -network bandwidth, the actual work is performed as CI Job in -the Factory. -Understanding what static deltas are needed -~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ -An operator will be planning an update to a new Target. This Target -will have been produced by CI and already have a tag that was used -by CI devices. For example, the targets.json might include:: +Fioctl includes a command to help generate static deltas. +Since these deltas can sometimes require a bit of processing power and network bandwidth, the actual work is performed as CI Job in the Factory. + +Understanding Why Static Deltas are Needed +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ + +An operator will be planning an update to a new Target. +This Target will have been produced by CI and already have a tag that was used by CI devices. +For example, the ``targets.json`` might include:: "raspberrypi4-64-lmp-9" : { "custom" : { @@ -36,9 +32,8 @@ by CI devices. For example, the targets.json might include:: "tags" : ["main"], ... -In this example, Target #9 has passed CI and needs to be deployed to -devices following the ``promoted`` tag. The operator can determine -what static deltas are needed by running: +In this example, Target #9 has passed CI and needs to be deployed to devices following the ``promoted`` tag. +The operator can determine what static deltas are needed by running: .. prompt:: bash host:~$, auto @@ -47,29 +42,24 @@ what static deltas are needed by running: 7 -> 9 5 -> 9 -In this case, fioctl has looked at all the devices configured to the -``promoted`` tag and found the Target versions they are running, -Target #5 and Target #7. So, to produce the most efficient OTAs, -two static deltas need to be created. +In this case, Fioctl has looked at all the devices configured to the ``promoted`` tag, and found the Target versions #5 and #7. +To produce the most efficient OTAs, two static deltas need to be created. -Creating the static deltas -~~~~~~~~~~~~~~~~~~~~~~~~~~ -Running the same command without ``--dryrun`` will produce the -static deltas via a CI Job: +Creating Static Deltas +~~~~~~~~~~~~~~~~~~~~~~ +Running the same command without ``--dryrun`` will produce the static deltas via a CI Job: .. prompt:: bash host:~$, auto host:~$ fioctl targets static-deltas --by-tag promoted 9 -Once the static deltas are in place, Target #9 can be re-tagged so -that "promoted" devices will apply the update: +Once the static deltas are in place, Target #9 can be re-tagged so that "promoted" devices will apply the update: .. prompt:: bash host:~$, auto host:~$ fioctl targets tag --tags main,promoted --by-version 9 -When the CI job completes, devices on the promoted tag will start -performing OTA updates that will use the static deltas. +When the CI job completes, devices on the promoted tag will start performing OTA updates that will use the static deltas. .. _Content Addressable Storage: https://en.wikipedia.org/wiki/Content-addressable_storage diff --git a/source/reference-manual/ota/targets.rst b/source/reference-manual/ota/targets.rst index d38320aec..ff4d000ea 100644 --- a/source/reference-manual/ota/targets.rst +++ b/source/reference-manual/ota/targets.rst @@ -3,12 +3,12 @@ Targets Overview ================ -Perhaps the most important goal of a FoundriesFactory is to deliver -immutable software updates to devices. This is achieved by using the -`The Update Framework`_ (TUF). The central piece to TUF is the notion -of a Target. A Target defines a cryptographically verifiable -description of the software a device should run. A simplified example -could be:: +Perhaps the most important goal of a Factory is to deliver immutable software updates to devices. +This is achieved by using the `The Update Framework`_ (TUF). +The central piece to TUF is the notion of a Target. +A Target defines a cryptographically verifiable description of the software a device should run. + +For a simplified example:: "raspberrypi3-64-lmp-42" : { "hashes" : {"sha256" : "0xdeadbeef"}, @@ -28,42 +28,40 @@ This Target specifies some important bits of information: * This is build ``42`` * The immutable OSTree hash of the base image is ``0xdeadbeef`` - * The Compose App, ``shellhttpd``, is part of the Target. + * The Compose app, ``shellhttpd``, is part of the Target. + +The Target is the goal. +Developers push changes to Git so as to build a Target. +Devices look to the OTA system for the latest Target they should run. +Operators oversee the intersection of these goals. -The Target is the goal. Developers push changes to Git in hopes of -building a Target. Devices look to the OTA system for the latest -Target they should run. Operators oversee the intersection of these -goals. +The point of a Factory is to create Targets. +The interesting thing about a Factory, is how with a ``git push``, you can make this all happen. +This is where you can start to visualize a Factory. +The :ref:`Factory Definition ` file instructs CI what to build when changes hit ``source.foundries.io``. -The point of the Factory is to create Targets. The magic of the -Factory is how a simple ``git push`` can make this all happen and -is where you can start to visualize a Factory. The -:ref:`Factory Definition ` file instructs -CI what it's supposed to build when changes hit source.foundries.io. -The default factory-config.yml tells CI: +The default ``factory-config.yml`` tells CI: - * If an LmP change (lmp-manifest.git or meta-subscriber-overrides.git) - comes in on the master branch, do a platform build and tag it with - "master". + * If an LmP change (``lmp-manifest.git`` or ``meta-subscriber-overrides.git``) is made to the default(i.e., "main") branch, + do a platform build and tag it with "default". - * If a container change (containers.git) comes in on master branch, - do a container build and tag it with master. + * If a container change (``containers.git``) is made to the default branch, do a container build and tag it with "default". However, this can grow much :ref:`more complex `. -CI must also take into account that Targets require both an OSTree -image **and** Compose Apps. This turns out to be a fairly simple -calculation. CI looks at the previous Target for a given tag. In -the case of a platform build, it will copy the Compose Apps defined for it. -In the case of a container build, it will copy the OSTree hash. In -this way, there aren't "container targets" and "platform targets". There -are only Targets. +CI must also take into account that Targets require both an OSTree image **and** Compose apps. +This turns out to be a fairly simple calculation. +CI looks at the previous Target for a given tag. +In the case of a platform build, it will copy the Compose apps defined for it. +In the case of a container build, it will copy the OSTree hash. +In this way, there are not "container targets" and "platform targets"; there are only Targets. Visualizing a Factory --------------------- -Start with ``factory-config.yml``. The ``tagging`` and ``ref_options`` -stanzas describe the intent. Then take a high-level view of the fleet: +Start with ``factory-config.yml``. +The ``tagging`` and ``ref_options`` stanzas describe the intent. +Then take a high-level view of the fleet: .. code-block:: bash @@ -80,8 +78,8 @@ stanzas describe the intent. Then take a high-level view of the fleet: 46 1 `fioctl targets show 46` 112 1 `fioctl targets show 112` -This will show all Targets active in the field. Now take a look -at a specific Target: +This will show all Targets active in the field. +Now take a look at a specific Target: .. code-block:: bash @@ -102,8 +100,7 @@ at a specific Target: --- ---- shellhttpd sha256:e4a7b3a31c0126d28aaf75e1b8b6e83c7afd160b110267530b8572ce192160da -This command gives the exact details of the Target, including the CI -change that produced it. +This command gives the exact details of the Target, including the CI change that produced it. .. _The Update Framework: https://theupdateframework.com/ diff --git a/source/reference-manual/ota/update-rollback.rst b/source/reference-manual/ota/update-rollback.rst index b7f54aada..8fc24f0a0 100644 --- a/source/reference-manual/ota/update-rollback.rst +++ b/source/reference-manual/ota/update-rollback.rst @@ -3,15 +3,15 @@ Update Rollback =============== -The Foundries.io OTA update service is intended for updating of software referenced by :ref:`Target `. +The Foundries.io™ OTA update service is intended for updating software referenced by a :ref:`Target `. Installation of artifacts for a new ``Target`` version on a device may fail for some reason or another. -When that happens the :ref:`OTA Client ` and the bootloader performs an "Update Rollback" procedure that will reinstall -and actualize the previous version of ``Target`` artifacts. +When this happens, the :ref:`OTA Client ` and the bootloader performs an "Update Rollback" procedure. +This will reinstall and actualize the previous version of ``Target`` artifacts. -``Target`` describes two updatable items: an `OSTree`_-managed rootfs and the :ref:`Compose Apps `. +``Target`` describes two updatable items: an `OSTree`_ — managed rootfs and the :ref:`Compose Apps `. Therefore, versions (hashes) of the ``Target`` components are bound by a specific ``Target`` version (number). -If one updated component fails to install, then both components should be rolled back to the previous version, -regardless of the installation status of the other component. +If one updated component fails to install, then both components should be rolled back to the previous version. +This is regardless of the installation status of the other component. During an update, ``aktualizr-lite`` (the :ref:`OTA Client `) tries to install a new version of rootfs at first, if found, and then it installs ``Compose Apps``. @@ -19,12 +19,12 @@ During an update, ``aktualizr-lite`` (the :ref:`OTA Client ` Rollback Driven by Rootfs Update Failure ________________________________________ -An installation of an `OSTree`_-managed rootfs consists of two phases: +An installation of an `OSTree`_ — managed rootfs consists of two phases: 1. Deployment of a new ostree version. - Effectively, it is creating hard links for each file of a specific ostree commit and corresponding drop-in boot snippet. + Effectively, it is creating hard links for each file of a specific ostree commit, and the corresponding drop-in boot snippet. See `OSTree Deployment Doc`_ for more details. -2. Booting a device on the deployment of the new rootfs version. +2. Booting a device upon deployment of the new rootfs version. If the first phase fails then the installation is interrupted and Update Rollback happens: @@ -34,17 +34,20 @@ If the first phase fails then the installation is interrupted and Update Rollbac If a boot on a new version of rootfs fails three times then: -1. the bootloader boots the previous deployment of rootfs; +1. The bootloader boots the previous deployment of rootfs; 2. ``aktualizr-lite`` re-installs the previous version of ``Compose Apps``; 3. ``aktualizr-lite`` prunes unused Compose Apps data (container images/layers of the new version). Rollback Driven by a Bootloader Update Failure ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ -The Foundries.io OTA update service supports bootloader update. Effectively, boot artifacts are included into `OSTree`_-managed rootfs. -Therefore a rollback procedure in the case of bootloader update failure is the same as in the case of the ostree update failure. + +The FoundriesFactory® OTA update service supports bootloader update. +Effectively, boot artifacts are included into `OSTree`_ — managed rootfs. +Therefore, a rollback procedure in the case of a bootloader update failure, is the same as in the case of the ostree update failure. The only difference is how and when the rollback is triggered. -In the case of a regular ostree update the rollback is managed by the bootloader and is triggered after a boot fails three times. -In the case of bootloader update failure the rollback triggering depends on a specific hardware, see :ref:`Boot software updates ` for more details. +In the case of a regular ostree update, the rollback is managed by the bootloader, and is triggered after a boot fails three times. +In the case of bootloader update failure, the triggering of the rollback depends on a specific hardware. +See :ref:`Boot software updates ` for more details. Rollback Driven by Compose Apps Update Failure @@ -52,24 +55,24 @@ ______________________________________________ The rollback procedure depends on the two factors: -1. An update type, whether it is just an Apps update or a composite update that includes both rootfs and Apps update. +1. An update type—whether it is just an Apps update, or a composite update—that includes both rootfs and Apps update. 2. A value of the :ref:`aktualizr-lite parameter ` called ``create_containers_before_reboot``. Apps Driven Rollback in the Case of a Composite Update ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ -If the first phase of an `OSTree`_-managed rootfs update is successful and ``create_containers_before_reboot`` is set to 1 (default), -then Apps' containers are (re-)created before a device reboot (needed to apply a new version of rootfs). -The Apps' containers (re-)creation may fail, if that happens then Apps driven rollback is triggered: +If the first phase of an `OSTree`_ — managed rootfs update is successful, and ``create_containers_before_reboot`` is set to 1 (default), then Apps' containers are (re-)created before a device reboot. +This is needed to apply a new version of rootfs. +However, the Apps' containers (re-)creation may fail. +If that happens, then Apps driven rollback is triggered: 1. ``Aktualizr-lite`` redeploys the previous version of rootfs. 2. ``Aktualizr-lite`` (re-)creates and starts the previous version of Apps' containers. 3. ``Aktualizr-lite`` prunes unused Compose Apps data (container images/layers of the new version). -If ``create_containers_before_reboot`` is set to 0 and a device is successfully booted on a new version of rootfs just after an update, -then ``aktualizr-lite`` (re-)creates and starts Apps' containers on its start. If one of these actions (creation or start) fails, -then another type of Apps driven rollback is triggered: +If ``create_containers_before_reboot`` is set to 0, and a device is successfully booted on a new version of rootfs just after an update, then ``aktualizr-lite`` (re-)creates and starts Apps' containers on its start. +If one of these actions (creation or start) fails, then another type of Apps driven rollback is triggered: 1. ``Aktualizr-lite`` redeploys the previous version of rootfs. 2. ``Aktualizr-lite`` initiates a device reboot. From 42c851934577d422a358bd9cdd103cf21e077aaf Mon Sep 17 00:00:00 2001 From: Katrina Prosise Date: Fri, 22 Dec 2023 05:32:18 -0500 Subject: [PATCH 2/2] Add suggestions Added additional suggestions Signed-off-by: Katrina Prosise --- source/reference-manual/ota/production-targets.rst | 14 +++++++------- 1 file changed, 7 insertions(+), 7 deletions(-) diff --git a/source/reference-manual/ota/production-targets.rst b/source/reference-manual/ota/production-targets.rst index 32555654e..290837b25 100644 --- a/source/reference-manual/ota/production-targets.rst +++ b/source/reference-manual/ota/production-targets.rst @@ -3,7 +3,7 @@ Production Targets ================== -As a part of going to production,you need to define two distinct sets of devices: *test* versus *production*. +As a part of going to production, you need to define two distinct sets of devices: *test* versus *production*. A **production device** is defined by a presence of a special attribute in its public certificate. This is set at the :ref:`device registration time `. If the attribute is absent, it designates a **test device**. @@ -28,7 +28,7 @@ Waves allow Factory operators to control an exact time when devices see a new ve The opposite is also true, production Targets can only be installed on production devices. From the TUF perspective CI versus production TUF Targets comprise two isolated sets of updates. - Within the scope of the below paragraphs, "device" refers exclusively to a "production device". + Within the scope of the below paragraphs, "device" refers exclusively to a "production device". .. _ref-rm-wave: @@ -38,7 +38,7 @@ Performing a Production OTA First define a process to select CI builds which need to be delivered to production devices. Every user performing production OTAs should generate their personal :ref:`offline TUF targets key ` to sign production Targets. -Let's assume you selected CI build version 42 as ready to be run in production. +Let us assume you selected CI build version 42 as ready to be run in production. To start the production release process, create a new wave using the below command:: # fioctl waves init -k @@ -61,7 +61,7 @@ There are several ways how a wave can be rolled out: - To a subset of devices in a Factory (potentially, across several device groups, including group-less devices). - To all devices in a Factory. -We recommend you first roll out a wave to a dedicated device group, which contains a small number of production devices. +We recommend that you first roll out a wave to a dedicated device group, which contains a small number of production devices. Another good option is to roll out a wave to a small subset of devices in a bigger device group. Let us assume you want to first roll out a new ``v2.0-update`` wave to a device group called ``canary``. This can be done using the below command:: @@ -187,9 +187,9 @@ Releasing to Large Device Fleets Now let us look at a more complex example of rolling out an update to a large device fleet. Assume that your Factory has 100'000 production devices to be updated within a wave. Normally, these devices will be split into a couple of groups according to your criteria. -Also assume your device fleet is arranged according to geographic presence, -e.g. device groups are ``us-east``, ``us-west``, ``eu-emea``, and ``apac``. -Some, or all of these device groups, would still contain a large number of devices. +Also assume your device fleet is arranged according to geographic presence. +For example, having device groups ``us-east``, ``us-west``, ``eu-emea``, and ``apac``. +Some—or all—of these device groups would still contain a large number of devices. From a safety perspective, it is risky to deliver an update to the entirety of any group like that. A usual practice would be to apply the "canary" approach (described above) to every individual device group.