Skip to content

Commit

Permalink
Test pipeline improvements (#88)
Browse files Browse the repository at this point in the history 
* -added branch name in run list; -added missisng dependency

* add name to manual run workflow

* refacted test setup script to parse as yaml, and programatically rename all services/volumes instead of specific SED commands

* edit etc/hosts script to match name in setup script

* update readme

* check commits uses new, more robust curl way of checking latest commits

* update badge to point to schedulaed vm workflow

* moved test setup, execution, teardown into own action; -updated a workflow to use new action and see if it works

* turn on workflow for on push to test

* turn on push to see if it works

* remove branch checkout to make sure actions work

* removed empty default string from action.yml, as that does not appear to compile

* added back default, as that wasn't the issue; removed empty output as I believe thta was causing the issue

* indenting error

* explcitly specify shell in actions

* fixed input context call

* use new python setup script in gmt-pytest action

* added renaming for the services/depends_on: key

* updated all workflows to use gmt-pytest action

* fix indenting error

* turn off conditional if just to confirm this is working as intended

* turned off debugging settings for vm workflow

* indenting error

* turn bare metal into vm for check step for debugging purposes

* turn on push for debug purproses

* removed workflow debug code

* Typos; Var renamings; __main__ py3 standard enforced; Using only keys to iterate over as pytonic alternative

* Copy before accessing keys, because we get the updated values in that case

* --remove-orphans added to stop the GMT containers if they are seen as identical and are still running

Co-authored-by: Dan M <dan@green-coding.org>
Co-authored-by: Arne Tarara <arne@datafuse.de>
  • Loading branch information
@dan-mm @ArneTR
3 people authored Dec 22, 2022
1 parent c9c2bf1 commit d4b6a25
Show file tree
Hide file tree
Showing 12 changed files with 278 additions and 212 deletions.
55 changes: 55 additions & 0 deletions .github/actions/gmt-pytest/action.yml
View file
Original file line number Diff line number Diff line change
@@ -0,0 +1,55 @@
name: 'GMT Pytest'
description: 'Run Pytest with setup and teardown'
inputs:
metrics-to-turn-off:
description: 'a list of metrics to turn off that is passed to the disable metrics script'
required: false
default: ''
runs:
using: 'composite'
steps:
- name: Install python dependencies
shell: bash
run: |
python -m pip install --upgrade pip
pip install fastapi pytest psycopg2 numpy pandas xgboost statsmodels scipy scikit-learn
- name: Run Prereqs and Install Script
shell: bash
run: |
sudo apt install lm-sensors libsensors-dev libglib2.0-0 libglib2.0-dev
./install.sh -p testpw
- name: Turn off metric providers which do not work in github vm
shell: bash
run: |
./disable-metric-providers.sh ${{ inputs.metrics-to-turn-off }}
- name: Setup Test environment
shell: bash
working-directory: ./test
run: |
python3 setup-test-env.py
- name: Start Test container
shell: bash
working-directory: ./test
run: |
docker compose -f ../docker/test-compose.yml up -d
- name: Sleep for 10 seconds
run: sleep 10s
shell: bash

- name: Run Tests
shell: bash
working-directory: ./test
run: |
pytest
- name: Stop Containers
shell: bash
if: always()
working-directory: ./test
run: |
docker compose -f ../docker/test-compose.yml down -v
100 changes: 42 additions & 58 deletions .github/workflows/tests-bare-metal.yml
View file
Original file line number Diff line number Diff line change
@@ -1,81 +1,65 @@
name: Daily Test Run - Bare Metal

run-name: Scheduled - ${{ github.ref_name }}
on:
schedule:
- cron: '0 0 * * *'
branches: [ "main", "dev" ]
# Allows you to run this workflow manually from the Actions tab
#workflow_dispatch:

#push:
# branch: test-pipeline-improvements
jobs:
#TODO: is there a way to do this that's even more light-weight?
check_date:
runs-on: self-hosted
name: Check latest commit
outputs:
should_run: ${{ steps.check_gmt_commits.outputs.should_run }}
recent_dev_commit: ${{ steps.check_dev_commits.outputs.recent_dev_commit }}
recent_main_commit: ${{ steps.check_main_commits.outputs.recent_main_commit }}
steps:
- uses: actions/checkout@v3
- name: print latest_commit
run: echo ${{ github.sha }}
- id: check_dev_commits
continue-on-error: true
name: check latest dev commit is less than a day
if: ${{ github.event_name == 'schedule' }}
run: |
dateofcommit=$(curl -s https://api.github.com/repos/green-coding-berlin/green-metrics-tool/commits/dev | jq -r '.commit.author.date')
commit_timestamp=$(date --date "$dateofcommit" +'%s')
one_day_ago=$(date --date '1 day ago' +'%s')
[ $commit_timestamp -gt $one_day_ago ] && echo "recent_dev_commit=true" >> $GITHUB_OUTPUT
#TODO: instead of 24 hours, last workflow run
- id: check_gmt_commits
- id: check_main_commits
continue-on-error: true
name: check latest commit is less than a day
name: check latest main commit is less than a day
if: ${{ github.event_name == 'schedule' }}
run: test -n $(git rev-list --after="24 hours" ${{ github.sha }}) && echo "should_run=true" >> $GITHUB_OUTPUT
run: |
dateofcommit=$(curl -s https://api.github.com/repos/green-coding-berlin/green-metrics-tool/commits/main | jq -r '.commit.author.date')
commit_timestamp=$(date --date "$dateofcommit" +'%s')
one_day_ago=$(date --date '1 day ago' +'%s')
[ $commit_timestamp -gt $one_day_ago ] && echo "recent_main_commit=true" >> $GITHUB_OUTPUT
run-tests:
run-tests-dev:
needs: check_date
if: ${{ needs.check_date.outputs.should_run == 'true' || github.event_name == 'workflow_dispatch' }}
if: ${{ needs.check_date.outputs.recent_dev_commit == 'true' }}
runs-on: self-hosted
strategy:
fail-fast: false
matrix:
branch: ['dev', 'main']
steps:
# Checks-out your repository under $GITHUB_WORKSPACE, so your job can access it
- uses: actions/checkout@v3
with:
ref: ${{ matrix.branch }}
ref: 'dev'
submodules: 'true'

- name: Install python dependencies
run: |
python -m pip install --upgrade pip
pip install fastapi pytest psycopg2 numpy pandas xgboost statsmodels scipy scikit-learn
- name: Run Prereqs and Install Script
run: |
sudo apt install lm-sensors libsensors-dev libglib2.0-dev
./install.sh -p testpw
- name: Turn off metric providers which do not work in github vm
run: |
./disable-metric-providers.sh System Sensors Debug
- name: 'Setup, Run, and Teardown Tests'
uses: ./.github/actions/gmt-pytest
with:
metrics-to-turn-off: 'System Sensors Debug'

- name: Setup Test environment
working-directory: ./test
run: |
./setup-test-env.sh -p testpw
- name: Start Test container
working-directory: ./test
run: |
docker compose -f ../docker/test-compose.yml up -d
- name: Sleep for 10 seconds
run: sleep 10s
shell: bash

- name: Run Tests
working-directory: ./test
run: |
pytest
- name: Stop Containers
if: always()
working-directory: ./test
run: |
docker compose -f ../docker/test-compose.yml down -v
run-tests-main:
needs: check_date
if: ${{ needs.check_date.outputs.recent_main_commit == 'true' }}
runs-on: self-hosted
steps:
- uses: actions/checkout@v3
with:
ref: 'main'
submodules: 'true'

- name: 'Setup, Run, and Teardown Tests'
uses: ./.github/actions/gmt-pytest
with:
metrics-to-turn-off: 'System Sensors Debug'
19 changes: 19 additions & 0 deletions .github/workflows/tests-manual.yml
View file
Original file line number Diff line number Diff line change
@@ -0,0 +1,19 @@
name: Manual Test Run - Virtual Machine
run-name: Manual - ${{ github.ref_name }}
on:
# Allows you to run this workflow manually from the Actions tab
workflow_dispatch:

jobs:
run-tests-manual:
runs-on: ubuntu-latest
steps:
# Checks-out your repository under $GITHUB_WORKSPACE, so your job can access it
- uses: actions/checkout@v3
with:
submodules: 'true'

- name: 'Setup, Run, and Teardown Tests'
uses: ./.github/actions/gmt-pytest
with:
metrics-to-turn-off: 'RAPL System Sensors Debug CGroupV2'
99 changes: 43 additions & 56 deletions .github/workflows/tests-vm.yml
View file
Original file line number Diff line number Diff line change
@@ -1,81 +1,68 @@
name: Daily Test Run - Virtual Machine

run-name: Scheduled - ${{ github.ref_name }}
on:
schedule:
- cron: '0 0 * * *'
branches: [ "main", "dev" ]
# Allows you to run this workflow manually from the Actions tab
#workflow_dispatch:
#push:
# branch: test-pipeline-improvements

jobs:
#TODO: is there a way to do this that's even more light-weight?
check_date:
runs-on: ubuntu-latest
name: Check latest commit
outputs:
should_run: ${{ steps.check_gmt_commits.outputs.should_run }}
recent_dev_commit: ${{ steps.check_dev_commits.outputs.recent_dev_commit }}
recent_main_commit: ${{ steps.check_main_commits.outputs.recent_main_commit }}
steps:
- uses: actions/checkout@v3
- name: print latest_commit
run: echo ${{ github.sha }}
- id: check_dev_commits
continue-on-error: true
name: check latest dev commit is less than a day
if: ${{ github.event_name == 'schedule' }}
run: |
dateofcommit=$(curl -s https://api.github.com/repos/green-coding-berlin/green-metrics-tool/commits/dev | jq -r '.commit.author.date')
commit_timestamp=$(date --date "$dateofcommit" +'%s')
one_day_ago=$(date --date '1 day ago' +'%s')
[ $commit_timestamp -gt $one_day_ago ] && echo "recent_dev_commit=true" >> $GITHUB_OUTPUT
#TODO: instead of 24 hours, last workflow run
- id: check_gmt_commits
- id: check_main_commits
continue-on-error: true
name: check latest commit is less than a day
name: check latest main commit is less than a day
if: ${{ github.event_name == 'schedule' }}
run: test -n $(git rev-list --after="24 hours" ${{ github.sha }}) && echo "should_run=true" >> $GITHUB_OUTPUT
run: |
dateofcommit=$(curl -s https://api.github.com/repos/green-coding-berlin/green-metrics-tool/commits/main | jq -r '.commit.author.date')
commit_timestamp=$(date --date "$dateofcommit" +'%s')
one_day_ago=$(date --date '1 day ago' +'%s')
[ $commit_timestamp -gt $one_day_ago ] && echo "recent_main_commit=true" >> $GITHUB_OUTPUT
run-tests:
run-tests-dev:
needs: check_date
if: ${{ needs.check_date.outputs.should_run == 'true' || github.event_name == 'workflow_dispatch' }}
if: ${{ needs.check_date.outputs.recent_dev_commit == 'true' }}
runs-on: ubuntu-latest
strategy:
fail-fast: false
matrix:
branch: ['dev', 'main']
steps:
# Checks-out your repository under $GITHUB_WORKSPACE, so your job can access it
- uses: actions/checkout@v3
with:
ref: ${{ matrix.branch }}
ref: 'dev'
submodules: 'true'

- name: Install python dependencies
run: |
python -m pip install --upgrade pip
pip install fastapi pytest psycopg2 numpy pandas xgboost statsmodels scipy scikit-learn
- name: Run Prereqs and Install Script
run: |
sudo apt install lm-sensors libsensors-dev libglib2.0-dev
./install.sh -p testpw
- name: Turn off metric providers which do not work in github vm
run: |
./disable-metric-providers.sh RAPL System Sensors Debug CGroupV2
- name: 'Setup, Run, and Teardown Tests'
uses: ./.github/actions/gmt-pytest
with:
metrics-to-turn-off: 'RAPL System Sensors Debug CGroupV2'

- name: Setup Test environment
working-directory: ./test
run: |
./setup-test-env.sh -p testpw
- name: Start Test container
working-directory: ./test
run: |
docker compose -f ../docker/test-compose.yml up -d
- name: Sleep for 10 seconds
run: sleep 10s
shell: bash

- name: Run Tests
working-directory: ./test
run: |
pytest
- name: Stop Containers
if: always()
working-directory: ./test
run: |
docker compose -f ../docker/test-compose.yml down -v
run-tests-main:
needs: check_date
if: ${{ needs.check_date.outputs.recent_main_commit == 'true' }}
runs-on: ubuntu-latest
steps:
# Checks-out your repository under $GITHUB_WORKSPACE, so your job can access it
- uses: actions/checkout@v3
with:
ref: 'main'
submodules: 'true'

- name: 'Setup, Run, and Teardown Tests'
uses: ./.github/actions/gmt-pytest
with:
metrics-to-turn-off: 'RAPL System Sensors Debug CGroupV2'
54 changes: 0 additions & 54 deletions .github/workflows/tests.yml
View file

This file was deleted.

2 changes: 1 addition & 1 deletion README.md
View file
Original file line number Diff line number Diff line change
Expand Up @@ -8,7 +8,7 @@ Currently no special setup is done to isolate the containers from the host OS an

Ensure that you run this only in an isolated VM / dedicated server.

![Tests Status](https://github.com/green-coding-berlin/green-metrics-tool/actions/workflows/tests.yml/badge.svg)
![Tests Status](https://github.com/green-coding-berlin/green-metrics-tool/actions/workflows/tests-vm.yml/badge.svg)


# Introduction
Expand Down
Loading

0 comments on commit d4b6a25

Please sign in to comment.