Fix a100 bisection workflow

[xuzhao9]

Test plan:

Test workflow: https://github.com/pytorch/benchmark/actions/runs/6985353191

Automatic bisection of the test_bench userbenchmark on the accuracy issue of llama_v2_7b_16h:

Start commit hash (2023-11-15): a5a404865c01f86881f6b3ab0cd9a562d0b420de

End commit hash (2023-11-16): 690c805c8b539501aad5fbf18914ac92afb65d5a

Userbenchmark name: test_bench

Userbenchmark arguments: llama_v2_7b_16h -d cuda -t eval --accuracy

Known root cause commit: 12b2dd16b050e6495910fc564517fbb51dde1f20

Result:

{
  "target_repo": "pytorch",
  "start": "6f4409073f9fad32016f18b3b083f959d7600a2b",
  "end": "a3b859fc673515f74d103494e61b432c9cc9009f",
  "result": [
    {
      "commit1": "cc11c0d11bf",
      "commit1_time": "2023-11-15 19:55:47 +0000",
      "commit1_digest": {
        "name": "test_bench",
        "environ": {
          "pytorch_git_version": "cc11c0d11bfdcc1f1804cedfcd634a5e79715638",
          "pytorch_version": "2.2.0a0+gitcc11c0d",
          "device": "NVIDIA A100-SXM4-40GB",
          "git_commit_hash": "cc11c0d11bfdcc1f1804cedfcd634a5e79715638"
        },
        "metrics": {
          "model=llama_v2_7b_16h, test=eval, device=cuda, bs=None, extra_args=['--accuracy'], metric=accuracy": "pass"
        }
      },
      "commit2": "12b2dd16b05",
      "commit2_time": "2023-11-15 20:26:13 +0000",
      "commit2_digest": {
        "name": "test_bench",
        "environ": {
          "pytorch_git_version": "12b2dd16b050e6495910fc564517fbb51dde1f20",
          "pytorch_version": "2.2.0a0+git12b2dd1",
          "device": "NVIDIA A100-SXM4-40GB",
          "git_commit_hash": "12b2dd16b050e6495910fc564517fbb51dde1f20"
        },
        "metrics": {
          "model=llama_v2_7b_16h, test=eval, device=cuda, bs=None, extra_args=['--accuracy'], metric=accuracy": "eager_1st_run_fail"
        }
      }
    }
  ]
}

The bisection is successful.

[facebook-github-bot]

@xuzhao9 has imported this pull request. If you are a Meta employee, you can view this diff on Phabricator.

[aaronenyeshi]

LGTM!

[facebook-github-bot]

@xuzhao9 merged this pull request in 9dea281.