diff --git a/dictionary-octopus.txt b/dictionary-octopus.txt index 8e659475ac..bb45d34164 100644 --- a/dictionary-octopus.txt +++ b/dictionary-octopus.txt @@ -6,6 +6,7 @@ bootstrap bootstrapped bootstrapper cicd +comms COMPUTERNAME createartifact combinational @@ -96,6 +97,7 @@ WCAG webfonts WEBSVR WIXUI +workerpool workertools xlarge xmark diff --git a/src/pages/docs/infrastructure/workers/built-in-worker.md b/src/pages/docs/infrastructure/workers/built-in-worker.md index b48a4b16ca..ff88efe991 100644 --- a/src/pages/docs/infrastructure/workers/built-in-worker.md +++ b/src/pages/docs/infrastructure/workers/built-in-worker.md @@ -1,7 +1,7 @@ --- layout: src/layouts/Default.astro pubDate: 2023-01-01 -modDate: 2023-01-01 +modDate: 2023-11-17 title: Built-in Worker description: Octopus Server comes with a built-in worker which enables you to conveniently run parts of your deployment process on the Octopus Server without the need to install a Tentacle or other deployment target. This page describes how to configure the built-in worker for a variety of scenarios. navOrder: 2 @@ -11,6 +11,10 @@ Octopus Server comes with a built-in worker which enables you to conveniently ru This page describes how to configure the built-in worker for a variety of scenarios. +:::div{.hint} +The built-in worker is only available on [self-hosted Octopus](/docs/getting-started#self-hosted-octopus) instances. [Octopus Cloud](/docs/octopus-cloud) customers have access to [dynamic worker pools](/docs/infrastructure/workers/dynamic-worker-pools), which provides a pre-configured worker on-demand. +::: + ## Built-in Worker When the built-in Worker is executed, the Octopus Server spawns a new process for Calamari. This conveniently allows a default Octopus set up to enable features like running script steps on the server and Azure deployments. However, this convenience comes at a cost: **security**. diff --git a/src/pages/docs/infrastructure/workers/dynamic-worker-pools.md b/src/pages/docs/infrastructure/workers/dynamic-worker-pools.md index a06bd3bac2..5a6dc75245 100644 --- a/src/pages/docs/infrastructure/workers/dynamic-worker-pools.md +++ b/src/pages/docs/infrastructure/workers/dynamic-worker-pools.md @@ -7,11 +7,13 @@ description: Dynamic Worker pools are used in our cloud product to dynamically c navOrder: 50 --- -Dynamic Worker Pools are a special type of [worker pool](/docs/infrastructure/workers/worker-pools/) used by Octopus Cloud. They use [workers](/docs/infrastructure/workers) provided by Octopus, and they don't require users to configure any infrastructure. +Dynamic Worker Pools provide a quick and easy way to use [workers](/docs/infrastructure/workers) for your deployments. They are a special type of [worker pool](/docs/infrastructure/workers/worker-pools) available on [Octopus Cloud](/docs/octopus-cloud). + +Dynamic workers are isolated virtual machines, created on-demand to run your deployments and runbook steps. They run on Azure and are created and managed by Octopus Cloud, which means you don't need to configure or maintain additional infrastructure. ## On demand -Dynamic workers are created on demand and leased to an Octopus Cloud instance for a limited time before being destroyed. +A dynamic worker is created on demand and leased to an Octopus Cloud instance for a limited time before being destroyed. :::div{.info} Octopus Cloud will automatically destroy dynamic workers as soon as one of these conditions is met: @@ -24,24 +26,42 @@ Please reach out to [support@octopus.com](mailto:support@octopus.com) if you nee ## Isolated -Each worker is provisioned exclusively to a specific customer, and is completely isolated from other customers. +Each worker VM is provisioned exclusively to a specific customer, and is completely isolated from other customers. ## Dynamic Worker Images -Each dynamic worker pool can specify the worker image used. Ubuntu Linux 22.04 is the default. Windows Server Core 2019 (end-of-life, see [below](#windows-server-core-2019)) and 2022 worker images are also available. +Each dynamic worker pool uses a specific worker image. This is a VM image which determines the operating system (and OS version) running on the worker. You can edit a dynamic worker pool to change which image is used. + +When you sign up to [Octopus Cloud](/docs/octopus-cloud) (or create a new [space](/docs/administration/spaces)), you automatically receive a worker pool for the `Ubuntu (default)` image, and a worker pool for the `Windows (default)` image. + +The full list of available worker images includes both specific operating system versions (e.g., `Ubuntu Linux 22.04`), and also generic "default" options such as `Ubuntu (default)`. Choosing the default option means that your worker will use the latest stable worker image when it is released. + +The current default images are: + +- `Ubuntu (default)` ➜ `Ubuntu Linux 22.04` +- `Windows (default)` ➜ `Windows Server Core 2019` + +:::div{.warning} +The `Windows (default)` image will change to Windows 2022 on 4 December 2023, and Windows 2019 images will be deprecated on 9 January 2024. You are advised to test your deployment processes with our Windows 2022 images. Please refer to [Windows 2019 end-of-life](/docs/infrastructure/workers/dynamic-worker-pools/windows-2019-end-of-life) for further details. +::: + +### Choosing an Image + +The default image is a good option to choose if you are: -Editing a dynamic worker pool allows you to modify the image used. +- Running a simple script that doesn't require specific tools or operating system versions +- Running a step [inside a container](/docs/projects/steps/execution-containers-for-workers) -The available worker images list specific operating system versions (e.g., `Ubuntu Linux 22.04`) but also generic "default" options such as `Ubuntu (default)`. Choosing the default option means that your worker will get the latest stable worker image released. This is a good option to choose if you're running a basic script that doesn't have any dependencies on specific tool or operating system versions. +If you're writing a script that relies on a specific version of tooling (e.g., helm), then we recommend using [execution containers for workers](/docs/projects/steps/execution-containers-for-workers) to run the script in a Docker container with the tool versions you need. -If you're writing a script that relies on a specific version of tooling (e.g., helm), then we recommend choosing a specific worker image, instead of the "default" options, to prevent worker image upgrades from impacting your deployments. +Alternatively, you can choose a specific worker image, instead of the "default" options, to prevent worker image upgrades from impacting your deployments. |Type | Pros | Cons | |-----|------|------| -| Default (eg `Ubuntu (default)`) | Automatically uses the latest image. Deployments will continue to work even when a worker image is marked as deprecated or decommissioned.| The versions of dependencies (e.g., helm) are not fixed. Deployments that rely on specific versions of dependencies or operating system specific features may break during upgrades. | +| Default (eg `Ubuntu (default)`) | Automatically uses the latest image. Deployments will continue to work even when a worker image is marked as deprecated or decommissioned.| The versions of dependencies (e.g. helm) are not fixed. Deployments that rely on specific versions of dependencies or operating system specific features may break during upgrades. | | Specific (e.g., `Ubuntu Linux 22.04`) | The version of the operating system and dependencies are fixed and can be relied upon. | When a worker image is marked as deprecated, warnings will start to appear in your deployment logs. When a worker image is decommissioned, you will need to take action to update your worker pool or deployments will fail. | -## Deprecation +### Deprecation When an image is marked as deprecated, you will see warnings in the Octopus UI, and in the deployment log. After a suitable deprecation period, deployments will start to fail if they target an image that has hit end-of-life. diff --git a/src/pages/docs/infrastructure/workers/index.mdx b/src/pages/docs/infrastructure/workers/index.mdx index c4afb36055..01ef282317 100644 --- a/src/pages/docs/infrastructure/workers/index.mdx +++ b/src/pages/docs/infrastructure/workers/index.mdx @@ -1,7 +1,7 @@ --- layout: src/layouts/Default.astro pubDate: 2023-01-01 -modDate: 2023-01-01 +modDate: 2023-11-17 title: Workers description: External workers are machines that can execute steps that don't need to be performed on the Octopus Server or deployment targets. navOrder: 30 @@ -13,7 +13,7 @@ import ConfigureSshConnectionWorker from 'src/shared-content/tentacle/configure- import ConfigureWindowsListeningWorker from 'src/shared-content/tentacle/configure-windows-listening-worker.include.md'; import ConfigureWindowsPollingWorker from 'src/shared-content/tentacle/configure-windows-polling-worker.include.md'; import InstallTentacleManager from 'src/shared-content/tentacle/install-tentacle-manager.include.md'; -import PoweShellNamedMutex from 'src/shared-content/tentacle/powershell-named-mutex.include.md'; +import PowerShellNamedMutex from 'src/shared-content/tentacle/powershell-named-mutex.include.md'; import Workers from 'src/shared-content/infrastructure/workers.include.md'; import WorkersIndividualProcessExclusively from 'src/shared-content/tentacle/workers-individual-process-exclusively.include.md'; import WorkersMultipleSimultaneousProcesses from 'src/shared-content/tentacle/workers-multiple-simultaneous-processes.include.md'; @@ -38,15 +38,20 @@ The following steps always run inside the Octopus Server process (and do not run A worker receives instruction from the Octopus Server to execute a step, it executes the step using Calamari and returns the logs and any collected artifacts to the Octopus Server. -There are two kinds of workers you can use in Octopus: - -1. [The built-in worker](#built-in-worker) (default) -1. [External workers](#external-workers) - :::div{.hint} Workers are assigned at the start of a deployment or runbook, not at the time the individual step executes. ::: +There are three kinds of workers you can use in Octopus: + +1. [The built-in worker](#built-in-worker) - available on self-hosted Octopus +1. [Dynamic workers](#dynamic-workers) - available on Octopus Cloud +1. [External workers](#external-workers) - manually configured + +:::div{.success} +[Octopus Cloud](/docs/octopus-cloud) uses [dynamic workers](/docs/infrastructure/workers/dynamic-worker-pools) by default, which provides an on-demand worker running on an Ubuntu or Windows VM. Dynamic workers are managed by Octopus Cloud, and are included with your Octopus Cloud subscription. +::: + ## Ignoring Workers \{#ignoring-workers} Octopus works out-of-the-box without setting up workers. You can run all deployment processes, run script steps on the built-in worker, deploy to Azure and run AWS and Terraform steps, without further setup. The built-in worker is available in a default Octopus set up, and Octopus workers are designed so that, if you aren't using external workers, none of your deployment processes need to be worker aware. @@ -55,7 +60,7 @@ The choices of built-in worker, built-in worker running in a separate account, a ## Migrating to Workers \{#migrating-to-workers} -Octopus workers provide a way to move work off the built-in worker. This lets you move the work away from the Octopus Server and onto external workers without the need to update the deployment process. Learn [how to use the default worker pool to move steps off the Octopus Server](/docs/infrastructure/workers/worker-pools/#Using-the-default-pool-to-stop-running-scripts-on-the-server). +Octopus workers provide a way to move work off the built-in worker. This lets you move the work away from the Octopus Server and onto external workers without the need to update the deployment process. Learn [how to use the default worker pool to move steps off the Octopus Server](/docs/infrastructure/workers/worker-pools/#default-worker-pool). ## Built-in Worker \{#built-in-worker} @@ -67,18 +72,34 @@ Adding a worker to the default worker pool will disable the built-in worker, and Learn about the security implications and how to configure the [built-in worker](/docs/infrastructure/workers/built-in-worker). +:::div{.hint} +The built-in worker is only available on [self-hosted Octopus](/docs/getting-started#self-hosted-octopus) instances. [Octopus Cloud](/docs/octopus-cloud) customers have access to [dynamic worker pools](/docs/infrastructure/workers/dynamic-worker-pools), which provides a pre-configured worker on-demand. +::: + +## Dynamic Workers \{#dynamic-workers} + +**Dynamic workers** are on-demand workers managed by [Octopus Cloud](/docs/octopus-cloud), which means you don't need to configure or maintain additional infrastructure. Dynamic workers provides an Ubuntu or Windows VM running as a pre-configured tentacle worker. + +Dynamic worker pools are included with all Octopus Cloud instances, and are the default option when creating new worker steps in your deployments and runbooks. + +Learn more about configuring and using [dynamic worker pools](/docs/infrastructure/workers/dynamic-worker-pools) and selecting an OS image for your worker tasks. + ## External Workers \{#external-workers} An **External Worker** is either: - A [Windows](/docs/infrastructure/deployment-targets/tentacle/windows/) or [Linux](/docs/infrastructure/deployment-targets/tentacle/linux) Tentacle. - An [SSH machine](/docs/infrastructure/deployment-targets/linux/ssh-target) that has been registered with the Octopus Server as a worker. -The setup of a worker is the same as setting up a deployment target as a [Windows Tentacle target](/docs/infrastructure/deployment-targets/tentacle/windows/) or an [SSH target](/docs/infrastructure/deployment-targets/linux/ssh-target), except that instead of being added to an environment, a worker is added to a worker pool. +The setup of a worker is the same as setting up a deployment target as a [Windows Tentacle target](/docs/infrastructure/deployment-targets/tentacle/windows/) or an [SSH target](/docs/infrastructure/deployment-targets/linux/ssh-target), except that instead of being added to an environment, a worker is added to a [worker pool](/docs/infrastructure/workers/worker-pools/). Using external workers allows delegating work to a machine other than the Octopus Server. This can make the server more secure and allow scaling. When Octopus executes a step on an external worker, it's the external worker that executes Calamari; no user-provided script executes on the Octopus Server itself. Workers have machine policies, are health checked, and run Calamari, just like deployment targets. +:::div{.success} +[Octopus Cloud](/docs/octopus-cloud) customers can choose to use the included [dynamic worker pools](/docs/infrastructure/workers/dynamic-worker-pools) (enabled by default), and/or register their own external workers. +::: + ## Registering an External Worker \{#registering-an-external-worker} Once the Tentacle or SSH machine has been configured, workers can be added using the [Web Portal](#registering-workers-in-the-web-portal), the [Octopus Deploy REST API](/docs/octopus-rest-api/), the [Octopus.Clients library](/docs/octopus-rest-api/octopus.client) or with the Tentacle executable. Only a user with the `ConfigureServer` permission can add or edit workers. @@ -132,14 +153,14 @@ The new Polling Tentacle will automatically show up in the Workers list. Tentacle workers can also register with the server using the Tentacle executable (version 3.22.0 or later), for example: ``` -.\Tentacle.exe register-worker --instance MyInstance --server "https://example.com/" --comms-style TentaclePassive --apikey "API-CS0SW5SQJNLUBQCUBPK8LZY3KYO" --workerpool "Default Worker Pool" +.\Tentacle.exe register-worker --instance MyInstance --server "https://example.com/" --comms-style TentaclePassive --apikey "API-YOUR-KEY" --workerpool "Default Worker Pool" ``` Use `TentacleActive` instead of `TentaclePassive` to register a polling Tentacle worker. The Tentacle executable can also be used to deregister workers, for example: ``` -.\Tentacle.exe deregister-worker --instance MyInstance --server "https://example.com/" --apikey "API-CS0SW5SQJNLUBQCUBPK8LZY3KYO" +.\Tentacle.exe deregister-worker --instance MyInstance --server "https://example.com/" --apikey "API-YOUR-KEY" ``` :::div{.hint} @@ -164,7 +185,7 @@ Default pools attached to cloud targets allow co-location of workers and targets - + ### Workers in HA setups diff --git a/src/pages/docs/infrastructure/workers/worker-pools.md b/src/pages/docs/infrastructure/workers/worker-pools.md index e12ab6d8cc..35d56b7c59 100644 --- a/src/pages/docs/infrastructure/workers/worker-pools.md +++ b/src/pages/docs/infrastructure/workers/worker-pools.md @@ -1,19 +1,21 @@ --- layout: src/layouts/Default.astro pubDate: 2023-01-01 -modDate: 2023-01-01 +modDate: 2023-11-17 title: Worker pools description: Worker pools are used to group workers and allow targeting steps at the pool of workers best equipped to execute the step. This page describes how to configure worker pools for a variety of scenarios. navOrder: 40 --- -Worker pools are groups of [workers](/docs/infrastructure/workers), when a task is assigned to a worker, the task will be executed by one of the workers in the worker pools you've configured. +Worker pools are groups of [workers](/docs/infrastructure/workers). When a task is assigned to a worker pool, the task will be executed by one of the workers in that pool. -## Default Worker Pool +You can manage your worker pools by navigating to **Infrastructure ➜ Worker Pools** in the Octopus Web Portal. + +## Multiple Pools -There is always a default worker pool, and the default Pool can't be deleted, but you can swap which pool is the default. Worker pools are global resources which can be scoped using [worker pool variables](/docs/projects/variables/worker-pool-variables). All users can see what pools are available and if there are workers in the pools. Only a user with the `ConfigureServer` permission can see the worker machines or edit workers or pools. +You can create multiple worker pools for different purposes. Worker pools are available to all projects in a [space](/docs/administration/spaces), and can be dynamically selected using [worker pool variables](/docs/projects/variables/worker-pool-variables). All users can see which pools are available and if there are workers in the pools. Only a user with the `ConfigureServer` permission can see the worker machines or edit workers or pools. -Using multiple worker pools allows you to configure the workers in your pools for the tasks they will be assigned. For instance, depending on your teams needs you might configure worker pools in the following ways: +Using multiple worker pools allows you to configure the workers in your pools for the tasks they will be assigned. For instance, depending on your teams needs you might configure worker pools in the following ways: - Pools for scripts that expect different operating systems (Linux vs. Windows). - Pools for scripts that rely on different versions of cloud CLI tools: Azure CLI, AWS CLI, Terraform CLI, Kubernetes CLI (kubectl) etc. @@ -21,7 +23,20 @@ Using multiple worker pools allows you to configure the workers in your pools f For your default pool it might be enough that the workers are Tentacles running PowerShell 5, but you might have two teams working with different version of an SDK and so provision worker pools with workers running the appropriate SDK for each team. -## Using Workers +## Default Worker Pool + +You can specify a worker pool which will be selected by default when creating new worker steps, and will be used for any worker steps that don't specify a pool. + +There is always a default worker pool for each [space](/docs/administration/spaces). The default pool can't be deleted, but you can swap which pool is the default. + +On [self-hosted Octopus](/docs/getting-started#self-hosted-octopus), an empty `Default Worker Pool` is provided. + +- Initially this pool is empty, which means the [built-in worker](/docs/infrastructure/workers/built-in-worker) will be used +- When you add workers to the default worker pool, the built-in worker will be disabled. This means any deployment processes that previously used the built-in worker on the Octopus Server will automatically start using workers in the worker pool. + +On [Octopus Cloud](/docs/octopus-cloud), the initial default pool is a [dynamic worker pool](/docs/infrastructure/workers/dynamic-worker-pools) using the `Windows (default)` image. A pool using the `Ubuntu (default)` image is also provided. + +## How Workers are Selected When a step that requires a worker is executed, Octopus first determines what worker pool the step should use, and then selects a worker from that pool to execute the step. @@ -43,10 +58,6 @@ The step will fail for lack of a worker if: - There are no healthy workers in the pool. - Octopus selects a healthy worker from the pool, but during the deployment process can't contact the worker. -## Using the Default Pool for scripts - -When you add workers to the default worker pool the built-in worker will be disabled. This means any deployment processes that previously used the built-in worker on the Octopus Server, will automatically move from using the built-in worker to workers in the worker pool. - ## Add new Worker Pools Only users with the `ConfigureServer` permission can add or edit worker pools.