Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

test(ci): coverpkg=gno.land/... for txtar tests #3088

Merged
merged 3 commits into from
Nov 8, 2024

Conversation

moul
Copy link
Member

@moul moul commented Nov 7, 2024

Note: I'm uncertain about what will happen after the merge.

Fixes #3085
Addresses #3003

@moul moul self-assigned this Nov 7, 2024
@github-actions github-actions bot added 🧾 package/realm Tag used for new Realms or Packages. 📦 ⛰️ gno.land Issues or PRs gno.land package related labels Nov 7, 2024
Signed-off-by: moul <94029+moul@users.noreply.github.com>
@moul moul changed the title dev/moul/fix 3085 test(ci): coverpkg=gno.land/... for txtar tests Nov 7, 2024
@moul moul marked this pull request as ready for review November 7, 2024 23:05
@gfanton
Copy link
Member

gfanton commented Nov 7, 2024

I tried something more generic in 9adfc59 (#3086); unfortunately, it seems to makes the gnovm tests timeout :(

There is a chance that this PR could work since it only applies to gno.land, but having this kind of different rules per module/package will make potential similar issues, like the one you had in #3003, harder to debug in the futur.


side note: why is this PR not triggering test jobs?

Signed-off-by: moul <94029+moul@users.noreply.github.com>
@moul
Copy link
Member Author

moul commented Nov 8, 2024

side note: why is this PR not triggering test jobs?

Probably because of 629fef9 :)

Copy link

codecov bot commented Nov 8, 2024

Codecov Report

All modified and coverable lines are covered by tests ✅

Project coverage is 63.74%. Comparing base (d73b6c6) to head (1d282a5).
Report is 1 commits behind head on master.

Additional details and impacted files
@@            Coverage Diff             @@
##           master    #3088      +/-   ##
==========================================
+ Coverage   63.33%   63.74%   +0.40%     
==========================================
  Files         548      548              
  Lines       78680    78680              
==========================================
+ Hits        49835    50153     +318     
+ Misses      25481    25143     -338     
- Partials     3364     3384      +20     
Flag Coverage Δ
contribs/gnodev 60.54% <ø> (-0.63%) ⬇️
contribs/gnofaucet 14.82% <ø> (ø)
gno.land 73.62% <ø> (+6.48%) ⬆️
gnovm 67.90% <ø> (ø)
misc/genstd 79.72% <ø> (ø)
tm2 62.34% <ø> (-0.09%) ⬇️

Flags with carried forward coverage won't be shown. Click here to find out more.

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

@moul
Copy link
Member Author

moul commented Nov 8, 2024

Alternative option: #3090

@thehowl
Copy link
Member

thehowl commented Nov 8, 2024

Reporting my 2c from signal:

I personally think it's fine if txtar tests aren't counted for coverage. Two reasons:

  1. The set up to make coverage work with txtar is ugly (maybe not with gnoland, but IIRC gnokey builds a separate binary and then runs it, so this setup is required there). I've seen what Guillhem had set up and it requires joining coverage files and creating related reports; there's no way to integrate this directly into go test -cover.
  2. txtar should be integration tests in addition to other unit tests. They're good for verifying specific conditions and testing out features in "real life" scenarios, but they are slow. Testing for normal features should go in unit tests, so should the verification and coverage of all code paths.

I'd prefer if we kept txtar tests to something similar to #2159, ie. integreation-testing common user flows, and then regression tests for complex bugs and behaviours which are hard to test.

@thehowl
Copy link
Member

thehowl commented Nov 8, 2024

#3090 (comment)

@thehowl thehowl merged commit da79c84 into gnolang:master Nov 8, 2024
135 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
📦 ⛰️ gno.land Issues or PRs gno.land package related 🧾 package/realm Tag used for new Realms or Packages.
Projects
Status: Done
Status: ✅ Done
Development

Successfully merging this pull request may close these issues.

Make txtar and Codecov play nice (upload coverage)
3 participants