-
Notifications
You must be signed in to change notification settings - Fork 4.9k
Commit
This commit does not belong to any branch on this repository, and may belong to a fork outside of the repository.
feat: Create a quality gate for unit test coverage (#27123)
<!-- Please submit this PR as a draft initially. Do not mark it as "Ready for review" until the template has been completely filled out, and PR status checks have passed at least once. --> ## **Description** <!-- Write a short description of the changes included in this pull request, also include relevant motivation and context. Have in mind the following questions: 1. What is the reason for the change? 2. What is the improvement/solution? --> [![Open in GitHub Codespaces](https://github.com/codespaces/badge.svg)](https://codespaces.new/MetaMask/metamask-extension/pull/27123?quickstart=1) This PR: 1. Refactors the current testing pipeline in one workflow (run-tests) 2. Removes codecov 3. Adds SonarCloud ## **Related issues** Fixes: MetaMask/MetaMask-planning#2962 ## **Manual testing steps** 1. Quality gate works for test coverage ## **Screenshots/Recordings** Not applicable ## **Pre-merge author checklist** - [x] I've followed [MetaMask Contributor Docs](https://github.com/MetaMask/contributor-docs) and [MetaMask Extension Coding Standards](https://github.com/MetaMask/metamask-extension/blob/develop/.github/guidelines/CODING_GUIDELINES.md). - [x] I've completed the PR template to the best of my ability - [x] I’ve included tests if applicable - [x] I’ve documented my code using [JSDoc](https://jsdoc.app/) format if applicable - [x] I’ve applied the right labels on the PR (see [labeling guidelines](https://github.com/MetaMask/metamask-extension/blob/develop/.github/guidelines/LABELING_GUIDELINES.md)). Not required for external contributors. ## **Pre-merge reviewer checklist** - [ ] I've manually tested the PR (e.g. pull and build branch, run the app, test code being changed). - [ ] I confirm that this PR addresses all acceptance criteria described in the ticket it closes and includes the necessary testing evidence such as recordings and or screenshots.
- Loading branch information
1 parent
832e6fc
commit 107f8da
Showing
7 changed files
with
150 additions
and
121 deletions.
There are no files selected for viewing
This file was deleted.
Oops, something went wrong.
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,142 @@ | ||
name: Run tests | ||
|
||
on: | ||
push: | ||
branches: | ||
- develop | ||
- master | ||
pull_request: | ||
types: | ||
- opened | ||
- reopened | ||
- synchronize | ||
|
||
jobs: | ||
test-unit: | ||
name: Unit tests | ||
runs-on: ubuntu-latest | ||
strategy: | ||
matrix: | ||
shard: [1, 2, 3, 4, 5, 6] | ||
steps: | ||
- name: Checkout repository | ||
uses: actions/checkout@v4 | ||
|
||
- name: Setup environment | ||
uses: metamask/github-tools/.github/actions/setup-environment@main | ||
|
||
- name: test:unit:coverage | ||
run: yarn test:unit:coverage --shard=${{ matrix.shard }}/${{ strategy.job-total }} | ||
|
||
- name: Rename coverage | ||
run: mv coverage/unit/coverage-final.json coverage/unit/coverage-unit-${{matrix.shard}}.json | ||
|
||
- uses: actions/upload-artifact@v4 | ||
with: | ||
name: coverage-unit-${{matrix.shard}} | ||
path: coverage/unit/coverage-unit-${{matrix.shard}}.json | ||
|
||
test-webpack: | ||
name: Webpack tests | ||
runs-on: ubuntu-latest | ||
steps: | ||
- name: Checkout repository | ||
uses: actions/checkout@v4 | ||
|
||
- name: Setup environment | ||
uses: metamask/github-tools/.github/actions/setup-environment@main | ||
|
||
- name: test:unit:webpack:coverage | ||
run: yarn test:unit:webpack:coverage | ||
|
||
- name: Rename coverage | ||
run: mv coverage/webpack/coverage-final.json coverage/webpack/coverage-webpack.json | ||
|
||
- uses: actions/upload-artifact@v4 | ||
with: | ||
name: coverage-webpack | ||
path: coverage/webpack/coverage-webpack.json | ||
|
||
test-integration: | ||
name: Integration tests | ||
runs-on: ubuntu-latest | ||
steps: | ||
- name: Checkout repository | ||
uses: actions/checkout@v4 | ||
|
||
- name: Setup environment | ||
uses: metamask/github-tools/.github/actions/setup-environment@main | ||
|
||
- name: test:integration:coverage | ||
run: yarn test:integration:coverage | ||
|
||
- name: Rename coverage | ||
run: mv coverage/integration/coverage-final.json coverage/integration/coverage-integration.json | ||
|
||
- uses: actions/upload-artifact@v4 | ||
with: | ||
name: coverage-integration | ||
path: coverage/integration/coverage-integration.json | ||
|
||
sonarcloud: | ||
name: SonarCloud | ||
runs-on: ubuntu-latest | ||
needs: | ||
- test-unit | ||
- test-webpack | ||
- test-integration | ||
steps: | ||
- name: Checkout repository | ||
uses: actions/checkout@v4 | ||
with: | ||
fetch-depth: 0 # Shallow clones should be disabled for better relevancy of analysis | ||
|
||
- name: Setup environment | ||
uses: metamask/github-tools/.github/actions/setup-environment@main | ||
|
||
- name: Download artifacts | ||
uses: actions/download-artifact@v4 | ||
with: | ||
path: coverage | ||
merge-multiple: true | ||
|
||
- name: Merge coverage reports | ||
run: yarn nyc merge coverage .nyc_output/coverage-final.json && yarn nyc report --reporter lcov | ||
|
||
- uses: actions/upload-artifact@v4 | ||
with: | ||
name: lcov.info | ||
path: coverage/lcov.info | ||
|
||
- name: Get Sonar coverage | ||
id: get-sonar-coverage | ||
env: | ||
SONAR_TOKEN: ${{ secrets.SONAR_TOKEN }} | ||
run: | | ||
projectKey=$(grep 'sonar.projectKey=' sonar-project.properties | cut -d'=' -f2) | ||
sonar_coverage=$(curl --silent --header "Authorization: Bearer $SONAR_TOKEN" "https://sonarcloud.io/api/measures/component?component=$projectKey&metricKeys=coverage" | jq -r '.component.measures[0].value // 0') | ||
echo "The Sonar coverage of $projectKey is $sonar_coverage%." | ||
echo 'SONAR_COVERAGE='"$sonar_coverage" >> "$GITHUB_OUTPUT" | ||
- name: Validate test coverage | ||
env: | ||
SONAR_COVERAGE: ${{ steps.get-sonar-coverage.outputs.SONAR_COVERAGE }} | ||
run: | | ||
coverage=$(yarn nyc report --reporter=text-summary | grep 'Lines' | awk '{gsub(/%/, ""); print $3}') | ||
if [ -z "$coverage" ]; then | ||
echo "::error::Could not retrieve test coverage." | ||
exit 1 | ||
fi | ||
if (( $(echo "$coverage < $SONAR_COVERAGE" | bc -l) )); then | ||
echo "::error::Quality gate failed for test coverage. Current test coverage is $coverage%, please increase coverage to at least $SONAR_COVERAGE%." | ||
exit 1 | ||
else | ||
echo "Test coverage is $coverage%. Quality gate passed." | ||
fi | ||
- name: SonarCloud Scan | ||
# This is SonarSource/sonarcloud-github-action@v2.0.0 | ||
uses: SonarSource/sonarcloud-github-action@4b4d7634dab97dcee0b75763a54a6dc92a9e6bc1 | ||
env: | ||
GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }} | ||
SONAR_TOKEN: ${{ secrets.SONAR_TOKEN }} |
This file was deleted.
Oops, something went wrong.
This file was deleted.
Oops, something went wrong.
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters