Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

ci: automatically run Larva tests as part of CI workflows #479

Merged
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
45 commits
Select commit Hold shift + click to select a range
00f4284
ci: add larva autorun workflow
DelanoWAF Dec 4, 2024
c2de17a
add temp workflow
DelanoWAF Dec 4, 2024
378a6c1
check ci change
DelanoWAF Dec 4, 2024
58bc77f
move ci in order
DelanoWAF Dec 4, 2024
2e55a66
docker instead of compose
DelanoWAF Dec 5, 2024
3cd3c11
Merge pull request #481 from wearefrank/master
DelanoWAF Dec 5, 2024
ed9d594
reuse docker compose from ci
DelanoWAF Dec 5, 2024
0f5ce7b
retry docker-compose
DelanoWAF Dec 5, 2024
093c533
increase timeout
DelanoWAF Dec 5, 2024
43c4756
refactor: set version to latest
DelanoWAF Dec 6, 2024
440cb4a
Merge pull request #482 from wearefrank/master
DelanoWAF Dec 6, 2024
41c6925
remove excess file
DelanoWAF Dec 6, 2024
dd09fff
change ci order
DelanoWAF Dec 6, 2024
287bf44
undo stage change
DelanoWAF Dec 6, 2024
f0623b4
attempt failed test trigger
DelanoWAF Dec 9, 2024
f312b8b
add more failed tests
DelanoWAF Dec 9, 2024
5d704d3
force logging
DelanoWAF Dec 9, 2024
a971abb
force more test failures
DelanoWAF Dec 9, 2024
4f1cd85
force another failure
DelanoWAF Dec 9, 2024
1ba5d19
change startup
DelanoWAF Dec 9, 2024
6f00db1
change order
DelanoWAF Dec 9, 2024
4805710
force log
DelanoWAF Dec 9, 2024
ce47fef
set proper command and version
DelanoWAF Dec 10, 2024
8effc61
rebuild
DelanoWAF Dec 10, 2024
c0462b4
keep only zaakbrug
DelanoWAF Dec 10, 2024
511ef99
add logging
DelanoWAF Dec 10, 2024
10f9797
undo logging
DelanoWAF Dec 10, 2024
969e838
reset to base state
DelanoWAF Dec 11, 2024
ef44852
Revert "force log"
DelanoWAF Dec 11, 2024
6b7032d
Revert "force another failure"
DelanoWAF Dec 12, 2024
17d93fa
Revert "force more test failures"
DelanoWAF Dec 12, 2024
875e638
Revert "add more failed tests"
DelanoWAF Dec 12, 2024
dfb0420
Revert "attempt failed test trigger"
DelanoWAF Dec 12, 2024
e3e2d35
use image-id instead for reliability
MLenterman Dec 19, 2024
7774789
increase timeout and fix container-id reference
MLenterman Dec 19, 2024
9b2afe3
fix container-id setter
MLenterman Dec 19, 2024
bda0bb3
generate larva result as github summary
MLenterman Dec 19, 2024
061eecd
filter results 1
MLenterman Dec 19, 2024
b430a71
filters results 2
MLenterman Dec 19, 2024
694c1a2
filter result 3
MLenterman Dec 19, 2024
e2f5bf4
filter results 4
MLenterman Dec 19, 2024
5637d23
give up on filtering result
MLenterman Dec 19, 2024
4ae55c3
run larva before soapui for fail fast
MLenterman Dec 19, 2024
623f6a4
add larva to release workflow too
MLenterman Dec 20, 2024
9331e4f
Merge branch 'master' into GT-1565-Automatically-run-Larva-tests-as-p…
MLenterman Dec 20, 2024
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
11 changes: 10 additions & 1 deletion .github/workflows/ci.yml
Original file line number Diff line number Diff line change
Expand Up @@ -42,11 +42,20 @@ jobs:
upload-sarif-to-security: false
run-frank-till-healthy-enabled: false

run-larva-scenarios:
uses: ./.github/workflows/run-larva-scenarios.yml
needs:
- version-next
- ci
with:
image-id: ${{ needs.ci.outputs.image-id }}

run-soapui-tests:
runs-on: ubuntu-latest
needs:
- version-next
- ci
- run-larva-scenarios
steps:
- uses: step-security/harden-runner@91182cccc01eb5e619899d80e4e971d6181294a7 # v2.10.1
with:
Expand Down Expand Up @@ -108,4 +117,4 @@ jobs:
if: always()
with:
name: reports-soapui-testreports
path: ./*/reports
path: ./*/reports
9 changes: 9 additions & 0 deletions .github/workflows/release.yml
Original file line number Diff line number Diff line change
Expand Up @@ -52,11 +52,20 @@ jobs:
upload-sarif-to-security: false
run-frank-till-healthy-enabled: false

run-larva-scenarios:
uses: ./.github/workflows/run-larva-scenarios.yml
needs:
- version-next
- ci
with:
image-id: ${{ needs.ci.outputs.image-id }}

run-soapui-tests:
runs-on: ubuntu-latest
needs:
- analyze-commits
- ci
- run-larva-scenarios
steps:
- uses: step-security/harden-runner@91182cccc01eb5e619899d80e4e971d6181294a7 # v2.10.1
with:
Expand Down
103 changes: 103 additions & 0 deletions .github/workflows/run-larva-scenarios.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,103 @@
name: Run Larva Scenarios

on:
workflow_call:
inputs:
image-id:
type: string
description: >
Docker image id of the loaded image to use.

Note that the Docker image needs to be available to the Docker runtime already.
required: false
docker-image-tag:
type: string
description: >
Docker image tag of the loaded image to use. For example: 'wearefrank/zaakbrug:1.2.0'.

Note that the Docker image needs to be available to the Docker runtime already.
required: false

jobs:
run-larva-scenarios:
runs-on: ubuntu-latest
steps:
- name: Checkout
uses: actions/checkout@692973e3d937129bcbf40652eb9f2f61becf3332 #4.1.7

- name: Download Docker tar
uses: actions/download-artifact@fa0a91b85d4f404e444e00e005971372dc801d16 #4.1.8
with:
name: build-docker-image

- name: Load Docker tar
shell: bash
run: |
docker load --input image.tar
docker image ls -a

- name: Start Application
id: docker-run
shell: bash
run: |
echo "container-id=$(docker run -d -p 8080:8080 -e dtap.stage=LOC ${{ inputs.image-id || inputs.docker-image-tag }})" >> "$GITHUB_OUTPUT"
timeout-minutes: 2

- name: Wait for Application to be Healthy
shell: bash
run: |
start_time=$(date +%s)
timeout=300
while true; do
if [ "$(docker inspect --format='{{json .State.Health.Status}}' ${{steps.docker-run.outputs.container-id}})" = '"healthy"' ]; then
echo "Application is healthy."
break
fi
current_time=$(date +%s)
elapsed_time=$((current_time - start_time))
if [ $elapsed_time -ge $timeout ]; then
echo "Application did not become healthy within $timeout seconds. Failing the job."
exit 1
fi
echo "Waiting for application to be healthy..."
sleep 5
done
timeout-minutes: 6

- name: Trigger Larva Tool
shell: bash
run: |
curl -X POST 'http://localhost:8080/iaf/larva/index.jsp' \
--header 'Content-Type: application/x-www-form-urlencoded' \
--data-urlencode 'scenariosrootdirectory=/opt/frank/testtool' \
--data-urlencode 'waitbeforecleanup=100' \
--data-urlencode 'loglevel=WRONG_PIPELINE_MESSAGES_PREPARED_FOR_DIFF' \
--data-urlencode 'execute=/opt/frank/testtool' \
--data-urlencode 'submit=start' > results.html

- name: Check if all scenarios passed
shell: bash
run: |
if ! grep -q 'All scenarios passed' results.html; then
echo "Not all scenarios passed. Failing the job."
exit 1
fi

- name: Generate Result in Markdown
uses: docker://pandoc/core:3.5
if: ${{ always() }}
with:
args: -s results.html --from html -o results.md --to gfm --standalone

- name: Add Larva Result To GitHub Actions Summary
shell: bash
if: ${{ always() }}
run: |
cat results.md >> $GITHUB_STEP_SUMMARY

- name: Cleanup
shell: bash
if: ${{ always() }}
run: |
docker stop ${{ steps.docker-run.outputs.container-id }}
docker rm --volumes ${{ steps.docker-run.outputs.container-id }}
Loading