Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Handle Docker Compose configuration entirely with environment variables #282

Open
wants to merge 31 commits into
base: master
Choose a base branch
from
Open

Handle Docker Compose configuration entirely with environment variables #282

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
31 commits
Select commit Hold shift + click to select a range
229281a
Use env var SSD_DEVICE to pass SSD to container
aromanielloNTIA Mar 22, 2024
ee3b311
Add default SSD_DEVICE=/dev/nvme0n1
aromanielloNTIA Mar 22, 2024
645b034
Configure API container shared memory size with env var
aromanielloNTIA Mar 22, 2024
027b572
remove unnecessary quotes
aromanielloNTIA Mar 22, 2024
4814833
add SSD_DEVICE and API_SHM_SIZE to README
aromanielloNTIA Mar 22, 2024
d3c541b
remove testing changes
aromanielloNTIA Mar 22, 2024
c795c98
remove redundant environment variable setting
aromanielloNTIA Mar 22, 2024
d62ed2d
Merge branch 'master' into all-config-with-env
aromanielloNTIA May 10, 2024
340588c
Automatically get SSD_DEVICE
aromanielloNTIA May 10, 2024
5aa2503
Merge branch 'all-config-with-env' of https://github.com/ntia/scos-se…
aromanielloNTIA May 10, 2024
5875313
Fix SSD_DEVICE default for github actions runner
aromanielloNTIA May 10, 2024
d804d6d
Use env var SSD_DEVICE to pass SSD to container
aromanielloNTIA Mar 22, 2024
c01814a
Add default SSD_DEVICE=/dev/nvme0n1
aromanielloNTIA Mar 22, 2024
e4f343d
Configure API container shared memory size with env var
aromanielloNTIA Mar 22, 2024
80e4bd1
remove unnecessary quotes
aromanielloNTIA Mar 22, 2024
4e55d69
Automatically get SSD_DEVICE
aromanielloNTIA May 10, 2024
eb178f7
add SSD_DEVICE and API_SHM_SIZE to README
aromanielloNTIA Mar 22, 2024
b9c5bf9
remove testing changes
aromanielloNTIA Mar 22, 2024
e29e015
remove redundant environment variable setting
aromanielloNTIA Mar 22, 2024
e6fd715
Fix SSD_DEVICE default for github actions runner
aromanielloNTIA May 10, 2024
aca1017
merge upstream
aromanielloNTIA Jul 25, 2024
82bc5ed
Fix github action
aromanielloNTIA Jul 25, 2024
df2e1dc
Merge branch 'master' into all-config-with-env
aromanielloNTIA Oct 17, 2024
37cc54c
Use "docker compose" in CI
aromanielloNTIA Oct 17, 2024
2f26795
Merge branch 'master' into all-config-with-env
aromanielloNTIA Nov 19, 2024
58612f6
Merge branch 'master' of https://github.com/NTIA/scos-sensor into all…
jhazentia Dec 13, 2024
dab8607
Re-add mistakenly removed env variable
aromanielloNTIA Dec 13, 2024
cc553d0
Update scos-tekrsa branch for testing
aromanielloNTIA Dec 13, 2024
16b12dd
Do not dynamically generate SSD_DEVICE default
aromanielloNTIA Dec 13, 2024
1879b04
Update scos-tekrsa to 8.1.0
aromanielloNTIA Jan 3, 2025
ce9b28f
Merge branch 'master' into all-config-with-env
aromanielloNTIA Jan 3, 2025
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
1 change: 1 addition & 0 deletions .github/workflows/github-actions-test.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -76,6 +76,7 @@ jobs:
source ./env.template
export MOCK_SIGAN=1
export MOCK_SIGAN_RANDOM=1
export SSD_DEVICE=/dev/nvme0n1
docker compose build --no-cache
docker compose up -d

Expand Down
4 changes: 2 additions & 2 deletions compose.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -27,7 +27,7 @@ services:
interval: 6s
timeout: 3s
retries: 1
shm_size: '16gb'
shm_size: ${API_SHM_SIZE}
restart: always
depends_on:
db:
Expand Down Expand Up @@ -95,7 +95,7 @@ services:
- SYS_ADMIN # required for ^ with NVMe drives
devices:
- /dev/bus/usb:/dev/bus/usb:rw
- /dev/nvme0n1:/dev/nvme0n1:ro
- ${SSD_DEVICE}:/dev/nvme0n1:ro
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I wonder if this would work if the SSD_DEVICE is a SATA device which is then mapped to an nvme device in the container? Perhaps it would be better to use ${SSD_DEVICE} for both then pass the SSD_DEVICE environment variable to scos-actions to get the path?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This has worked when testing using my Linux laptop with a SATA SSD (/dev/sda on the host)

extra_hosts:
- "${MANAGER_FQDN}:${MANAGER_IP}"
logging:
Expand Down
21 changes: 15 additions & 6 deletions env.template
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -2,12 +2,11 @@

# Copy env.template to env and modify


# Mark all the following variables for export
set -o allexport

#Any names here will be added as additional users with the
#specified additional user password
# Any names here will be added as additional users with the
# specified additional user password
ADDITIONAL_USER_NAMES="" # comma separated
ADDITIONAL_USER_PASSWORD=""

Expand All @@ -18,15 +17,15 @@ ADMIN_PASSWORD=password

# set to CERT to enable scos-sensor certificate authentication
AUTHENTICATION=TOKEN
CALIBRATION_EXPIRATION_LIMIT=360

# Default callback api/results
# Set to CERT for certificate authentication
CALLBACK_AUTHENTICATION=TOKEN
# Set to false to disable SSL cert verification in the callback POST request
CALLBACK_SSL_VERIFICATION=false

#Set the number of seconds before timeout in postback when a scheduled
#action completes
# Set the number of seconds before timeout in postback when a scheduled
# action completes
CALLBACK_TIMEOUT=2

# SECURITY WARNING: don't run with debug turned on in production!
Expand Down Expand Up @@ -89,12 +88,22 @@ SSL_CA_PATH=scos_test_ca.crt
SSL_CERT_PATH=sensor01.pem
SSL_KEY_PATH=sensor01.pem

# SSD Device made available to SCOS Plugins, e.g. for getting SMART data
SSD_DEVICE=/dev/nvme0n1

# Size of shared memory for API container (/dev/shm), used for parallel processing
# See https://docs.docker.com/compose/compose-file/compose-file-v3/#shm_size
API_SHM_SIZE=16gb

# Calibration action selection
# The action specified here will be used to attempt an onboard
# sensor calibration on startup, if no onboard calibration data
# is available on startup. The specified action must be available.
STARTUP_CALIBRATION_ACTION=SEA_CBRS_Calibrate_Baseline

# How many seconds until calibration data is considered stale
CALIBRATION_EXPIRATION_LIMIT=3600
aromanielloNTIA marked this conversation as resolved.
Show resolved Hide resolved

# Debug dependant settings
if $DEBUG; then
GUNICORN_LOG_LEVEL=debug
Expand Down
4 changes: 2 additions & 2 deletions src/requirements-dev.txt
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -331,11 +331,11 @@ scipy==1.13.1
# via
# -r requirements.txt
# scos-actions
scos-actions @ git+https://github.com/NTIA/scos-actions@11.0.0
scos-actions @ git+https://github.com/NTIA/scos-actions@11.1.0
# via
# -r requirements.txt
# scos-tekrsa
scos-tekrsa @ git+https://github.com/NTIA/scos-tekrsa@8.0.0
scos-tekrsa @ git+https://github.com/NTIA/scos-tekrsa@8.1.0
# via -r requirements.txt
sigmf @ git+https://github.com/NTIA/SigMF@multi-recording-archive
# via
Expand Down
2 changes: 1 addition & 1 deletion src/requirements.in
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -12,7 +12,7 @@ psycopg2-binary>=2.0, <3.0
tzdata # https://code.djangoproject.com/ticket/33814
requests>=2.32.0
requests-mock>=1.0, <2.0
scos_tekrsa @ git+https://github.com/NTIA/scos-tekrsa@8.0.0
scos_tekrsa @ git+https://github.com/NTIA/scos-tekrsa@8.1.0

# The following are sub-dependencies for which SCOS Sensor enforces a
# higher minimum patch version than the dependencies which require them.
Expand Down
4 changes: 2 additions & 2 deletions src/requirements.txt
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -161,9 +161,9 @@ ruamel-yaml-clib==0.2.8
# via ruamel-yaml
scipy==1.13.1
# via scos-actions
scos-actions @ git+https://github.com/NTIA/scos-actions@11.0.0
scos-actions @ git+https://github.com/NTIA/scos-actions@11.1.0
# via scos-tekrsa
scos-tekrsa @ git+https://github.com/NTIA/scos-tekrsa@8.0.0
scos-tekrsa @ git+https://github.com/NTIA/scos-tekrsa@8.1.0
# via -r requirements.in
sigmf @ git+https://github.com/NTIA/SigMF@multi-recording-archive
# via scos-actions
Expand Down