Services

Why comp?

The multi-staged nature and repetitive commands for setting up complex compositions (e.g., containers with host user's UID:GID, shared networks across compositions, etc.) is the raison d'être for comp. Some of the key features are highlighted below.

For running non-root containers with the host user's UID:GID (e.g., to avoid permission issues with mounted volumes), one must grab these from the host, and pass them as env variables, for docker compose to pick them up. The same applies to TZ and other host-side configs as well. Not all of these configs are absolute constants that maybe added to a .env and never be updated again. In such cases, we ideally want a "hook" that runs before docker compose is run, and updates / generates the .env as necessary.

Similar hooks are also desirable to set up external networks that are shared across multiple compositions. In such cases, the hook would execute a docker network command before running docker compose. comp automatically grabs all external networks in YAML files, and has a hook to create them if they do not already exist.

comp also has several safeguards that docker compose doesn't provide. For instance, any non-existent host directories that are mounted to containers are automatically created by Docker and are owned by root, even when the --user flag is set! comp errors out in such cases, requesting the user for explicit action.

Why not use Ansible / Kubernetes / XYZ?

Seemed like overkill, when I initially wrote comp.

Why bash?

Why not?

Usage

Typical workflows:

• (Re)start compositions:

  ./comp down,up tang pihole
  

• Update repo & restart a composition:

  git pull && ./comp pdu pihole
  

  Note how we specify verbs pull,down,up using just their first characters.

• Checking the status of compositions:

  ./comp status tang pihole
  

See comp#L319 for all supported supported flags and options.

Also see Structure & Conventions for optionally customizing compositions via overrides.

Arch. Support

╭──   Compositions     \     Supported Archs   ───				amd 64	386	arm v6	arm v7	arm 64	ppc 64le	risc v64
AirDC++ :443/airdcpp	A B	openvpn‑client airdcpp	2.6.11 2.12.2	✔️	✔️	✔️ ✖️	✔️	✔️	✔️ ✖️	✔️ ✖️
Certbot :80	A	certbot	3.0.0	✔️	✔️	✔️	✔️	✔️	✔️	✖️
docker_sock -	A	docker.sock‑proxy	2.8.10	✔️	✔️	✔️	✔️	✔️	✔️	✖️
Gitea :443/gitea	B	gitea	1.22.3-r...s	✔️	✖️	✖️	✖️	✔️	✖️	✖️
HAss :443/	B	hass	2024.10.4_2.0.1	✔️	✔️	✔️	✔️	✔️	✖️	✖️
Indexarr :443/jackett	A B B	openvpn‑client solvarr jackett	2.6.11 v3.3.21 0.22.898	✔️	✔️ ✔️ ✖️	✔️ ✖️ ✖️	✔️	✔️	✔️ ✖️ ✖️	✔️ ✖️ ✖️
InfluxDB :8086	B	influxdb	2.7.10-a...e	✔️	✖️	✖️	✖️	✔️	✖️	✖️
Kodi.DB :3306	B	mariadb	11.5.2	✔️	✖️	✖️	✖️	✔️	✔️	✖️
Monitarr :443/{lid,rad,son}arr	B B B	lidarr radarr sonarr	2.7.1.4417 5.14.0.9383 4.0.10.2544	✔️	✖️ ✖️ ✔️	✖️ ✖️ ✔️	✔️	✔️	✖️ ✖️ ✖️	✖️ ✖️ ✖️
Navidrome :443/navidrome	B	navidrome	0.53.3	✔️	✔️	✔️	✔️	✔️	✖️	✖️
Nextcloud :443/nextcloud	B B B C B	mariadb redis imaginary nextcloud cron	10.11.6 7.4.1 20241106...04 30.0.2-core 30.0.2-core	✔️	✖️ ✔️ ✖️ ✔️ ✔️	✖️ ✔️ ✖️ ✔️ ✔️	✖️ ✔️ ✖️ ✔️ ✔️	✔️	✔️ ✔️ ✖️ ✖️ ✖️	✖️ ✔️ ✖️ ✖️ ✖️
Pi‑hole :53, :9443/pihole	A B	dnscrypt‑proxy pihole	2.1.5 2024.07.0	✔️	✔️	✔️	✔️	✔️	✔️	✖️ ✔️
qBittorrent :5432, :443/qbittorrent	A A	openvpn‑client qbittorrent‑nox	2.6.11 5.0.1	✔️	✔️	✔️	✔️	✔️	✔️ ✖️	✔️ ✖️
Tang :9080/tang	A	tang	git.02...c8	✔️	✔️	✔️	✔️	✔️	✔️	✖️
Telegraf -	B	telegraf	1.32.2	✔️	✖️	✖️	✔️	✔️	✖️	✖️
Teslamate :1883, :9443/teslamate	B B B	db mqtt teslamate	17.0 2.0.20 1.31.1	✔️	✔️ ✔️ ✖️	✔️ ✔️ ✖️	✔️	✔️	✔️ ✔️ ✖️	✔️ ✖️ ✖️
Tiny HTTPD :443	A	tiny-httpd	2.29	✔️	✖️	✖️	✔️	✔️	✖️	✖️
Traefik :443, :8443, :9080, :9443	B	traefik	2.11.13	✔️	✖️	✔️	✖️	✔️	✔️	✔️
Unifi :3478, :8080, :8843, :10001	B X	mongodb unifi	4.4.18 8.6.9	✔️	✖️ ✖️	✖️ ✖️	✖️ ✔️	✔️	✖️ ✖️	✖️ ✖️

Legend

Below, we say that a container is a non-root container, if it allows running the target service as a non-root user, e.g. using --user with docker run.

Note that this is different from (less secure compared to) running the container with a rootless docker daemon.

A	Trustworthy: Non-root containers with binaries compiled during build, or from OS repos
B	Secure: Non-root containers with open-source binaries only
C	Open: Root-ed containers with open-source binaries only
X	Untrusted: Containers with closed-source binaries

Structure & Conventions

All specializations should be added to *.override.* files.

• at the repo root:

  • static.global.override.env may store global constants
    • e.g. ACME configs, ports to be open etc.
    • these are exposed as-is to docker compose
    • see static.global.env for the default
  • dynamic.global.override.env.sh may generate additional dynamic variables
    • e.g. as public IP, UID of calling user etc.
    • these are (re)computed just before running each a composition
    • see dynamic.global.env.sh for the default

• within each composition:

  • pre.override.reqs may name additional compositions as prerequisites
    • compositions listed in pre.reqs cannot be removed, however
    • see tang/pre.reqs for an example
  • options.override.env may contain overrides for default values of the {devices|labels|logging|ports} options
    • options provided on the command-line may still override options.override.conf
    • see .github/workflows/config/options.conf for an example
  • static.override.env may store additional service-specific constants
    • similar idea as its global counterpart static.global.env
    • see gitea/static.env for an example
  • dynamic.override.env.sh may generate additional service-specific evironment variables
    • similar idea as its global counterpart dynamic.global.env.sh
    • see pihole/dynamic.env.sh for an example
  • docker-compose.override.{yml|yaml} may contain overrides for docker compose
    • modular overrides may also be specified for individual YAML fragment files:
      docker-compose.{devices|labels|logging|ports}.override.{yml|yaml}
  • docker-compose.{pre,post}_hook.override.*.sh may define additional hooks to be run before docker compose
    • see nextcloud/docker-compose.pre_hook.sh for an example
  • docker-compose.{up,down,clean}.{pre,post}_hook.override.*.sh may define additional verb-specific hooks to be run
    • see monitarr/docker-compose.up.pre_hook.sh for an example

Subdirectories

• immediate subdirectories under config/, data/, extra/, and generated/ must match the service names within docker-compose.yml
• the directory structure at each of config/X/Y..., data/X/Y..., extra/X/Y..., and generated/X/Y... must match the root directory hierarchy /Y/... in the target container X

	Mount mode	Git commit	Comments
config/	:ro or -	✔️
data/	:rw	✖️
env/	-	✔️	May contain env_files
extra/	-	✔️	Only indirect access, e.g. via generated/
generated/	:ro	✖️