You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
For artifact evaluations, we should provide more features for reproducibility:
run a suite based on suite_design in a results folder rather than on the design
store the commit hash or something like this next to the results (how to deal with local changes? maybe have a "reproducible" flag that enforces that you don't have changes?)
provide a way to run a meaningful subset of the results?
store info on which cloud the experiment was executed (introduce check that when calling last, that the cloud still matches)
The text was updated successfully, but these errors were encountered:
For artifact evaluations, we should provide more features for reproducibility:
suite_design
in a results folder rather than on the designThe text was updated successfully, but these errors were encountered: