DOC: document pytest-mpl update workflow #4418
Conversation
doc/source/developing/testing.rst
Outdated
| There are 3 situations where updating baseline images may be necessary | ||
|
|
||
| - bugfixes | ||
| - intentional change of style in yt | ||
| - old baseline fails with a new version of matplotlib, but changes are not noticeable to the human eye |
There was a problem hiding this comment.
I'm blanking this morning and can't remember... do you also do all this when adding a new answer test? or is this only when modifying existing answer tests?
There was a problem hiding this comment.
The process is intentionally very similar indeed. Adding new answers or updating existing one is basically the same workflow.
There was a problem hiding this comment.
Oh -- my initial question was vague. What I meant to ask was whether you need to submit the PR to the answer repo when adding a new image test (since this section is about "updating" images). And the answer is obviously yes now that I've done that :) I assume that'll be mentioned up above in the TODO section? Or maybe this section should change from "Updating Image Baselines" to "Updating or Adding New Image Baselines"?
There was a problem hiding this comment.
yes this is what I intended to write where the TODO placeholder is, but I think I could never decide what to do about the generic_image documentation that we have now, because I never used it and I generally would like to stay away from answer tests as I've had bad experiences trying to debug some.
Anyway, it's a shame that I could never finish this PR as it's clearly needed. If you'd like to take over, be my guest. At a minimum I would need some feedback about my TODO (@matthewturk ?).
doc/source/developing/testing.rst
Outdated
| @@ -287,6 +287,8 @@ Answer test examples can be found in ``yt/frontends/enzo/tests/test_outputs.py`` | |||
| How to Write Image Comparison Tests | |||
| ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ | |||
|
|
|||
| TODO(#4415): update this section (maybe keep the old content but add a note for why it's out of date) | |||
There was a problem hiding this comment.
I think that would be great. Even just a warning callout with a link to the section below would work.
There was a problem hiding this comment.
I just realised that generic_image is actually not used at all anymore. I think we should just deprecate it, so I'll write the warning accordingly.
neutrinoceros
force-pushed
the
doc_pytest_mpl
branch
from
68788d7 to
1d49684
Compare
neutrinoceros
force-pushed
the
doc_pytest_mpl
branch
from
1d49684 to
2bc9dda
Compare
|
@yt-fido test this please |
|
@matthewturk and @chrishavlin, I hope you're up to review this |
neutrinoceros
force-pushed
the
doc_pytest_mpl
branch
from
2bc9dda to
40b96d0
Compare
neutrinoceros
force-pushed
the
doc_pytest_mpl
branch
from
40b96d0 to
0377d3a
Compare
|
@matthewturk , you wanna take another look at this one? IMO it's ready to be merged. |
neutrinoceros
dismissed stale reviews from matthewturk and chrishavlin
via
31c43a4
PR Summary
Complements #4122, partially adresses #4415
note to reviewers: this workflow may seem very cumbersome (it is), but it's not new.
In fact it is extremely close to what we were doing even before #4122, it just happens that this process was never properly documented.
Opening as a draft for now because I need to come back and update the
How to Write Image Comparison Testssection too.