[ci] Update cppyy testing job #96
Conversation
.github/workflows/ci.yml
Outdated
| echo ::endgroup:: | ||
|
|
||
| echo ::group::Run complete test suite | ||
| python -m pytest || true |
There was a problem hiding this comment.
This seems wrong. We should not force the test suite to succeed. That’s our metric for what works and we want to know when it regresses.
There was a problem hiding this comment.
Ok, I think I understand now. This command runs also the xfailed. If an xfailed test crashes then we abort. That's what happens with the osx builds. To make them work we should mark the crashing xfailed as crashing.
The other reason for the remaining failures is the valgrind complaints on the tests where we call a void function. We should fix that...
There was a problem hiding this comment.
Yes but currently python -m pytest skips the crashing xfails since we have marked them as run = False so it is already taken care of. The current failures are in the groups - cranshing logs, xfail logs and passing test logs (because of valgrind) since they seem to bring up the errors.
I will update the CI to handle the same and we should be able to drop || true when we run the whole test suite with python -m pytest
.github/workflows/ci.yml
Outdated
| @@ -750,7 +755,7 @@ jobs: | |||
| declare -i RETCODE=0 | |||
|
|
|||
| set -o pipefail | |||
| valgrind --error-exitcode=1 --suppressions=../etc/valgrind-cppyy-cling.supp python -m pytest -m "not xfail" -v | |||
| valgrind --error-exitcode=1 --suppressions=../etc/valgrind-cppyy-cling.supp python -m pytest -m "not xfail" -v || true | |||
|
@vgvassilev understood, I have kept the true condition only for the crashing test runs and removed it from both the runs you addressed. |
Yes, and that's the trouble - we have regressed. We need to fix this before moving forward... |
|
@vgvassilev this PR now works as intended, could you take a look at the jobs? On linux only clang-16 reports a failure due to valgrind. The OS X jobs are currently red because of the 33 failures due to libc++ not handling std library types like libstdc++ |
We should open an issue for these and mark them as xfail on osx. |
|
@vgvassilev This has been updated to the same as compiler-research/CppInterOp#239, we can go ahead with the merge |
No description provided.