Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

fix(ci): Skip GCP CI jobs on PRs from external contributors, let mergify test them after approval #7956

Merged
merged 17 commits into from
Nov 22, 2023
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
37 changes: 37 additions & 0 deletions .github/workflows/cd-deploy-nodes-gcp.patch-external.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,37 @@
# Workflow patches for skipping Google Cloud CD deployments on PRs from external repositories.
name: Deploy Nodes to GCP

# Run on PRs from external repositories, let them pass, and then Mergify will check them.
# GitHub doesn't support filtering workflows by source branch names, so we have to do it for each
# job.
on:
pull_request:

# IMPORTANT
#
# These job names must be kept in sync with the `.patch.yml` and `.yml` files.
jobs:
# We don't patch the testnet job, because testnet isn't required to merge (it's too unstable)
build:
name: Build CD Docker / Build images
# Only run on PRs from external repositories, skipping ZF branches and tags.
if: ${{ !contains(github.head_ref || github.ref, 'refs/heads/') && !contains(github.head_ref || github.ref, 'refs/tags/') }}
runs-on: ubuntu-latest
steps:
- run: 'echo "No build required"'

test-configuration-file:
name: Test CD default Docker config file / Test default-conf in Docker
# This dependency allows all these jobs to depend on a single condition, making it easier to
# change.
needs: build
runs-on: ubuntu-latest
steps:
- run: 'echo "No build required"'

test-zebra-conf-path:
name: Test CD custom Docker config file / Test custom-conf in Docker
needs: build
runs-on: ubuntu-latest
steps:
- run: 'echo "No build required"'
9 changes: 8 additions & 1 deletion .github/workflows/cd-deploy-nodes-gcp.patch.yml
Original file line number Diff line number Diff line change
@@ -1,5 +1,8 @@
# Workflow patches for skipping Google Cloud CD deployments, when Rust code or dependencies aren't
# modified in a PR.
name: Deploy Nodes to GCP

# Run on PRs with unmodified code and dependency files.
on:
pull_request:
paths-ignore:
Expand All @@ -19,7 +22,11 @@ on:
- '.github/workflows/cd-deploy-nodes-gcp.yml'
- '.github/workflows/sub-build-docker-image.yml'

# IMPORTANT
#
# These job names must be kept in sync with the `.patch-external.yml` and `.yml` files.
jobs:
# We don't patch the testnet job, because testnet isn't required to merge (it's too unstable)
build:
name: Build CD Docker / Build images
runs-on: ubuntu-latest
Expand All @@ -36,4 +43,4 @@ jobs:
name: Test CD custom Docker config file / Test custom-conf in Docker
runs-on: ubuntu-latest
steps:
- run: 'echo "No build required"'
- run: 'echo "No build required"'
11 changes: 11 additions & 0 deletions .github/workflows/cd-deploy-nodes-gcp.yml
Original file line number Diff line number Diff line change
@@ -1,3 +1,6 @@
# Google Cloud node deployments and tests that run when Rust code or dependencies are modified,
# but only on PRs from the ZcashFoundation/zebra repository.
# (External PRs are tested/deployed by mergify.)
name: Deploy Nodes to GCP

# Ensures that only one workflow task will run at a time. Previous deployments, if
Expand Down Expand Up @@ -31,6 +34,7 @@ on:

# TODO: Temporarily disabled to reduce network load, see #6894.
#push:
# # Skip main branch updates where Rust code and dependencies aren't modified.
# branches:
# - main
# paths:
Expand All @@ -52,6 +56,7 @@ on:

# Only runs the Docker image tests, doesn't deploy any instances
pull_request:
# Skip PRs where Rust code and dependencies aren't modified.
paths:
# code and tests
- '**/*.rs'
Expand All @@ -73,6 +78,9 @@ on:
types:
- published

# IMPORTANT
#
# These job names must be kept in sync with the `.patch.yml` and `.patch-external.yml` files.
jobs:
# If a release was made we want to extract the first part of the semver from the
# tag_name
Expand Down Expand Up @@ -107,6 +115,9 @@ jobs:
# The image will be commonly named `zebrad:<short-hash | github-ref | semver>`
build:
name: Build CD Docker
# Skip PRs from external repositories, let them pass, and then Mergify will check them.
# Since this workflow also runs on release tags, we need to check for them as well.
if: ${{ contains(github.head_ref || github.ref, 'refs/heads/') || contains(github.head_ref || github.ref, 'refs/tags/') }}
uses: ./.github/workflows/sub-build-docker-image.yml
with:
dockerfile_path: ./docker/Dockerfile
Expand Down
101 changes: 101 additions & 0 deletions .github/workflows/ci-integration-tests-gcp.patch-external.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,101 @@
# Workflow patches for skipping Google Cloud unit test CI on PRs from external repositories.
name: Integration Tests on GCP

# Run on PRs from external repositories, let them pass, and then Mergify will check them.
# GitHub doesn't support filtering workflows by source branch names, so we have to do it for each
# job.
on:
pull_request:

# IMPORTANT
#
# These job names must be kept in sync with the `.patch.yml` and `.yml` files.
jobs:
# We don't patch the testnet job, because testnet isn't required to merge (it's too unstable)
get-available-disks:
name: Check if cached state disks exist for Mainnet / Check if cached state disks exist
# Only run on PRs from external repositories.
# (github.ref is always a local branch, so this check will skip non-PRs as well.)
if: ${{ !contains(github.head_ref || github.ref, 'refs/heads/') }}
runs-on: ubuntu-latest
steps:
- run: 'echo "No build required"'

build:
name: Build CI Docker / Build images
# This dependency allows all these jobs to depend on a single condition, making it easier to
# change.
needs: get-available-disks
runs-on: ubuntu-latest
steps:
- run: 'echo "No build required"'

test-stateful-sync:
name: Zebra checkpoint update / Run sync-past-checkpoint test
needs: get-available-disks
runs-on: ubuntu-latest
steps:
- run: 'echo "No build required"'

test-update-sync:
name: Zebra tip update / Run update-to-tip test
needs: get-available-disks
runs-on: ubuntu-latest
steps:
- run: 'echo "No build required"'

checkpoints-mainnet:
name: Generate checkpoints mainnet / Run checkpoints-mainnet test
needs: get-available-disks
runs-on: ubuntu-latest
steps:
- run: 'echo "No build required"'

lightwalletd-rpc-test:
name: Zebra tip JSON-RPC / Run fully-synced-rpc test
needs: get-available-disks
runs-on: ubuntu-latest
steps:
- run: 'echo "No build required"'

lightwalletd-transactions-test:
name: lightwalletd tip send / Run lwd-send-transactions test
needs: get-available-disks
runs-on: ubuntu-latest
steps:
- run: 'echo "No build required"'

get-block-template-test:
name: get block template / Run get-block-template test
needs: get-available-disks
runs-on: ubuntu-latest
steps:
- run: 'echo "No build required"'

submit-block-test:
name: submit block / Run submit-block test
needs: get-available-disks
runs-on: ubuntu-latest
steps:
- run: 'echo "No build required"'

lightwalletd-full-sync:
name: lightwalletd tip / Run lwd-full-sync test
needs: get-available-disks
runs-on: ubuntu-latest
steps:
- run: 'echo "No build required"'

lightwalletd-update-sync:
name: lightwalletd tip update / Run lwd-update-sync test
needs: get-available-disks
runs-on: ubuntu-latest
steps:
- run: 'echo "No build required"'

lightwalletd-grpc-test:
name: lightwalletd GRPC tests / Run lwd-grpc-wallet test
needs: get-available-disks
runs-on: ubuntu-latest
steps:
- run: 'echo "No build required"'
8 changes: 6 additions & 2 deletions .github/workflows/ci-integration-tests-gcp.patch.yml
Original file line number Diff line number Diff line change
@@ -1,7 +1,8 @@
# Workflow patches for skipping Google Cloud integration test CI when Rust code or dependencies
# aren't modified in a PR.
name: Integration Tests on GCP

# These jobs *don't* depend on cached Google Cloud state disks,
# so they can be skipped when the modified files make the actual workflow run.
# Run on PRs with unmodified code and dependency files.
on:
pull_request:
paths-ignore:
Expand All @@ -25,6 +26,9 @@ on:
- '.github/workflows/sub-find-cached-disks.yml'
- '.github/workflows/sub-build-docker-image.yml'

# IMPORTANT
#
# These job names must be kept in sync with the `.patch-external.yml` and `.yml` files.
jobs:
# We don't patch the testnet job, because testnet isn't required to merge (it's too unstable)
get-available-disks:
Expand Down
11 changes: 11 additions & 0 deletions .github/workflows/ci-integration-tests-gcp.yml
Original file line number Diff line number Diff line change
@@ -1,3 +1,5 @@
# Google Cloud integration tests that run when Rust code or dependencies are modified,
# but only on PRs from the ZcashFoundation/zebra repository. (External PRs are tested by mergify.)
name: Integration Tests on GCP

# Ensures that only one workflow task will run at a time. Previous builds, if
Expand Down Expand Up @@ -47,6 +49,7 @@ on:
default: false

pull_request:
# Skip PRs where Rust code and dependencies aren't modified.
paths:
# code and tests
- '**/*.rs'
Expand All @@ -68,6 +71,7 @@ on:
- '.github/workflows/sub-find-cached-disks.yml'

push:
# Skip main branch updates where Rust code and dependencies aren't modified.
branches:
- main
paths:
Expand All @@ -91,6 +95,9 @@ on:
- '.github/workflows/sub-find-cached-disks.yml'
- '.github/workflows/sub-build-docker-image.yml'

# IMPORTANT
#
# These job names must be kept in sync with the `.patch.yml` and `.patch-external.yml` files.
jobs:
# to also run a job on Mergify head branches,
# add `|| (github.event_name == 'push' && startsWith(github.head_ref, 'mergify/merge-queue/'))`:
Expand All @@ -104,6 +111,8 @@ jobs:
# The outputs for this job have the same names as the workflow outputs in sub-find-cached-disks.yml
get-available-disks:
name: Check if cached state disks exist for ${{ inputs.network || vars.ZCASH_NETWORK }}
# Skip PRs from external repositories, let them pass, and then Mergify will check them
if: ${{ contains(github.head_ref || github.ref, 'refs/heads/') }}
uses: ./.github/workflows/sub-find-cached-disks.yml
with:
network: ${{ inputs.network || vars.ZCASH_NETWORK }}
Expand All @@ -114,6 +123,7 @@ jobs:
# Some outputs are ignored, because we don't run those jobs on testnet.
get-available-disks-testnet:
name: Check if cached state disks exist for testnet
if: ${{ contains(github.head_ref || github.ref, 'refs/heads/') }}
uses: ./.github/workflows/sub-find-cached-disks.yml
with:
network: 'Testnet'
Expand All @@ -125,6 +135,7 @@ jobs:
# testnet when running the image.
build:
name: Build CI Docker
if: ${{ contains(github.head_ref || github.ref, 'refs/heads/') }}
uses: ./.github/workflows/sub-build-docker-image.yml
with:
dockerfile_path: ./docker/Dockerfile
Expand Down
4 changes: 4 additions & 0 deletions .github/workflows/ci-lint.yml
Original file line number Diff line number Diff line change
Expand Up @@ -148,7 +148,11 @@ jobs:
with:
level: warning
fail_on_error: false
# This gives an error when run on PRs from external repositories, so we skip it.
- name: validate-dependabot
# If this is a PR, check that the PR source is from a local branch.
# (github.ref is always a local branch, so this check always passes for non-PRs.)
if: ${{ contains(github.head_ref || github.ref, 'refs/heads/') }}
uses: marocchino/validate-dependabot@v2.1.0

codespell:
Expand Down
65 changes: 65 additions & 0 deletions .github/workflows/ci-unit-tests-docker.patch-external.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,65 @@
# Workflow patches for skipping Google Cloud unit test CI on PRs from external repositories.
name: Docker Unit Tests

# Run on PRs from external repositories, let them pass, and then Mergify will check them.
# GitHub doesn't support filtering workflows by source branch names, so we have to do it for each
# job.
on:
pull_request:

# IMPORTANT
#
# These job names must be kept in sync with the `.patch.yml` and `.yml` files.
jobs:
build:
name: Build CI Docker / Build images
# Only run on PRs from external repositories.
# (github.ref is always a local branch, so this check will skip non-PRs as well.)
if: ${{ !contains(github.head_ref || github.ref, 'refs/heads/') }}
runs-on: ubuntu-latest
steps:
- run: 'echo "No build required"'

test-all:
name: Test all
# This dependency allows all these jobs to depend on a single condition, making it easier to
# change.
needs: build
runs-on: ubuntu-latest
steps:
- run: 'echo "No build required"'

test-fake-activation-heights:
name: Test with fake activation heights
needs: build
runs-on: ubuntu-latest
steps:
- run: 'echo "No build required"'

test-empty-sync:
name: Test checkpoint sync from empty state
needs: build
runs-on: ubuntu-latest
steps:
- run: 'echo "No build required"'

test-lightwalletd-integration:
name: Test integration with lightwalletd
needs: build
runs-on: ubuntu-latest
steps:
- run: 'echo "No build required"'

test-configuration-file:
name: Test CI default Docker config file / Test default-conf in Docker
needs: build
runs-on: ubuntu-latest
steps:
- run: 'echo "No build required"'

test-zebra-conf-path:
name: Test CI custom Docker config file / Test custom-conf in Docker
needs: build
runs-on: ubuntu-latest
steps:
- run: 'echo "No build required"'
7 changes: 5 additions & 2 deletions .github/workflows/ci-unit-tests-docker.patch.yml
Original file line number Diff line number Diff line change
@@ -1,7 +1,7 @@
# Workflow patches for skipping unit test CI when Rust code or dependencies aren't modified in a PR.
name: Docker Unit Tests

# These jobs *don't* depend on cached Google Cloud state disks,
# so they can be skipped when the modified files make the actual workflow run.
# Run on PRs with unmodified code and dependency files.
on:
pull_request:
paths-ignore:
Expand All @@ -25,6 +25,9 @@ on:
- '.github/workflows/sub-find-cached-disks.yml'
- '.github/workflows/sub-build-docker-image.yml'

# IMPORTANT
#
# These job names must be kept in sync with the `.patch-external.yml` and `.yml` files.
jobs:
build:
name: Build CI Docker / Build images
Expand Down
Loading
Loading