Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add bisection interactive mode for debugging. #2148

Closed
wants to merge 6 commits into from
Closed

Conversation

xuzhao9
Copy link
Contributor

@xuzhao9 xuzhao9 commented Feb 2, 2024

In https://github.com/pytorch/benchmark/actions/runs/7745905863, the bisection returns empty result where it should find the root cause commit of hf_longformer bugfix.

  1. Fix a bug where bisection returns empty result: the regression detector will change the commit metrics, so we need to pass in a copy of the commit digest.
  2. Add an interactive mode for easier debugging.

Test Plan:

To use the interactive mode, run (pytorch repo sources are still needed):

python bisection.py --work-dir ~/work/bisection/test --torch-repos-path ~/tmp/torch_srcs --config ~/work/bisection/test/regression-2024-02-01.yaml --torchbench-repo-path $PWD --output ~/work/bisection/test/output.json --interactive

Run the nightly bisection workflow on 2024-02-01:

https://github.com/pytorch/benchmark/actions/runs/7793771518

The workflow finds the right commit: #2151

@facebook-github-bot
Copy link
Contributor

@xuzhao9 has imported this pull request. If you are a Meta employee, you can view this diff on Phabricator.

Copy link
Member

@aaronenyeshi aaronenyeshi left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is a great addition!

@facebook-github-bot
Copy link
Contributor

@xuzhao9 merged this pull request in ab07f16.

@xuzhao9 xuzhao9 deleted the xz9/fix-bisection branch February 6, 2024 17:04
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants