Peas

Definition

A 'pea' is a special term in garden for a process that is run alongside an existing container, sharing most namespaces and cgroups, but with its own mount namespace and root filesystem.

How to Create a Pea

There is no special API endpoint for creating pea processes, as distinct from ordinary processes. The normal POST to /containers/:handle/processes is used and this ends up calling Run() in the gardener/container code. The containerizer is invoked and will check if Image in the process spec is non-empty. If so, a pea is created, rather than a process Execed in the container.

You can create a pea using the gaol CLI as follows:

# create a container
gaol create -n main

# run and attach to a process in 'main' using the image 'busybox'
gaol run main -ai docker://busybox -c 'echo hello'

> hello

Why Peas?

A good example is Envoy. Cloudfoundry uses Envoy for container networking. Envoy has its own container image, and the envoy process is launched alongside each main container to give it network ingress and egress. Envoy needs to be in the same network namespace as the main container in order to do its networking job, but it can't be Execed in the container, as its binary and configuration is not included there. Therefore a pea is a good solution: Envoy has its own root filesystem so it can execute, but it shares networking with the container.

Healthchecks are also performed in peas for similar reasons. The healthcheck will often check for an OK response from an endpoint on the container. The checking code is not included in the running container, as it is a cloudfoundry concern, so a separate rootfs is required. Yet being in the same network namespace is advantageous, and means localhost addresses can be used, and potentially healthz endpoints need not be exposed publicly.

Pea Mechanics

Internally, peas are represented differently depending on which of the garden modes is being used. Common code runs in the pea_creator.

Notice how the namespaces are cloned from the main container (identified by the sandboxHandle) here and how the mount namespace is left empty so a new one will be created, distinct from the main container.

Since a pea is nearly a fully-fledged container, the first step is to create its container image using the volumizer. In fact, pea creation is very similar to container creation. Compare the image creation in the gardener.

It is regretable that the current model has the gardener comprising of the volumizer, containerizer and networker, and then the containerizer itself then contains a pea creator, which itself needs a volumizer and an exec-runner, which does much of the work of the containerizer. Moving the pea-creator to the level of the gardener could create a simpler structure.

After a process spec is created, the pea_creator invokes the ExecRunner to RunPea(). In runc mode, and container mode, the pea is created from garden with dadoo and runc. In containerd for processes mode, containerd is used.

Runc Mode / Containerd (without processes) Mode

The dadoo ExecRunner is used to run the pea. This creates a directory for the pea process in the processes directory of the main container in the depot, and writes the process spec to config.json there. Then runProcess() takes care of running the process. Note that it is passed run as the runMode. Normal process execution in the dadoo ExecRunner uses exec as the mode. This is passed through to the runc command, so a new container is created rather than a process running in an existing one.

Container for Processes Mode

When running in full containerd mode, RunContainerPea creates the pea. The peaManager is wired as runcontainerd/runcontainerd, just like the containerizer's runtime in containerd mode. The pea is created the same as a normal container, apart from having its process set as the requested process, and the shared namespaces.

Both normal process and pea processes end up as a BackingProcess object in runcontainerd. The first is a containerd Process; the other a containerd Task. A Task is a superset of a Process therefore we can wrap the Task (i.e. running pea container) as a process in runcontainerd code and treat them the same regarding attaching, streaming, etc.

Listing Containers and Peas

In runc mode, we list containers using the depot. The main containers are the top level directories in the depot. Peas appear as directories within the process subdirectory, and differ from normal process by containing a config.json file, describing the pea container.

In containerd for processes mode, containerd does the container management. Although container directories are still created in the depot, the depot is not used to list containers. There is nothing fundamental to distinguish pea containers from main containers in containerd, so we use annotations. Pea containers have the container-type annotation set to pea, whereas main containers have it set to garden-init. Pea containers also have a sandbox-container annotation set to the main container handle. This way we can list main containers excluding pea containers, and list the peas associated with a main container.