test: add lightweight integration tests #102
Merged
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
kormide
force-pushed
the
e2e
branch
14 times, most recently
from
1efaa87 to
d117fc0
Compare
kormide
changed the title
kormide
force-pushed
the
e2e
branch
3 times, most recently
from
279392b to
0861b4f
Compare
cgrindel
reviewed
Jan 1, 2024
| with: | ||
| version: ">= 363.0.0" | ||
| - run: yarn install --frozen-lockfile | ||
| - run: yarn e2e |
There was a problem hiding this comment.
If you are interested, I have a GitHub action that joins workflow jobs so that the branch protection rule can be configured to track a single check (usage example). Otherwise, you need to update the branch protection rules whenever a job is added/removed from the CI workflow.
cgrindel
approved these changes
Jan 1, 2024
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Until now I've been doing manual regression testing before each deployment which is a pain.
In this integration test setup, some services are real, some are faked: the webhook runs locally using Functions Framework, the same runtime used by Cloud Functions. Emails are sent to a real email service for testing called Ethereal. Google Secrets manager is stubbed, and so is the GitHub API itself.
Attempting to use the real GitHub feels too burdensome as it would require the SIG to maintain a set of orgs, repos, and users for testing which need to be kept in a pristine state across test runs. The API is well defined and easy enough to stub. By faking GitHub, the tests can be run locally or on CI and don't conflict with one another. All of the test "repos" are just local fixtures defined in this repo that get set up as fake "remote" repos on disk.
Some of the e2e tests are snapshot tests. Snapshot tests take a snapshot of all the BCR entry files created in a pull request and perform a diff test. This will make it easier to spot any regressions when changes cause a diff to fail.
Closes #84.