JaCoCo is used for measuring test code coverage in the build, in order to obtain metrics on the current state of EDC testing as well as its evolution over time.
The Codecov platform is used for visualizing code coverage statistics on PRs. This will raise developer awareness on an increase/decrease of coverage introduced by PRs. Codecov provides a detailed report including a dashboard with additional metrics like code complexity.
Test code coverage is a measure of the source code that executed when a test suite is run. A program with high test coverage has a lower chance of containing bugs.
At the time of writing, code coverage in the solution is under 50%. Increasing code coverage can best be achieved over time by providing feedback on coverage impact on each PR. This requires a more advanced tool than JaCoCo on its own can provide, and is well achieved by Codecov.
We evaluated the following options:
- JaCoCo without or with aggregation
- JaCoCo with Codecov
- JaCoCo with Codacy
- JaCoCo with SonarQube
- JaCoCo with GitHub Action
| Tool | Project coverage report | Coverage on PR in GitHub | Additional comments |
|---|---|---|---|
| JaCoCo with Codecov | ✅ Detailed report on Codecov dashboard | ✅ GitHub bot messages on every PR (coverage after the PR is merged, total project coverage, code complexity) | ✅ Reports on code complexity ✅ Easy configuration |
| JaCoCo with GitHub Actions | ✅ Basic report (percentage) | ✅ GitHub bot messages on every PR (coverage on changed files and total project coverage) | |
| JaCoCo with Codacy | ✅ Report available on Codacy dashboard |