Skip to content

Commit

Permalink
the accuracy test always returns none for the output
Browse files Browse the repository at this point in the history
Summary:
As titled, we should fix it to have the actual output return and compared.

Before the fix, test will always pass because it is comparing None with None

After the fix, it can compare the actual outputs.

we detected output mismatch in cmf_10x, cmf_10x_batch_fusion and ads_dhen_5x and alexnet now (see v2), which we will disable and need investigation ore replacing with up-to-date models.

Reviewed By: xuzhao9, jackiexu1992

Differential Revision: D49077949

fbshipit-source-id: 2952bf007e522bf8cacc1415b92349a82d1fab9b
  • Loading branch information
Lu Fang authored and facebook-github-bot committed Sep 8, 2023
1 parent 0a64c55 commit 1118332
Showing 1 changed file with 2 additions and 2 deletions.
4 changes: 2 additions & 2 deletions torchbenchmark/util/env_check.py
Original file line number Diff line number Diff line change
Expand Up @@ -367,9 +367,9 @@ def forward_and_backward_pass(mod, inputs, contexts, optimizer, collect_outputs=
def run_n_iterations(mod, inputs, contexts, optimizer=None, is_training=False, iterations=STABLENESS_CHECK_ROUNDS):
def _model_iter_fn(mod, inputs, contexts, optimizer, collect_outputs):
if is_training:
forward_and_backward_pass(mod, inputs, contexts, optimizer, collect_outputs)
return forward_and_backward_pass(mod, inputs, contexts, optimizer, collect_outputs)
else:
forward_pass(mod, inputs, contexts, collect_outputs)
return forward_pass(mod, inputs, contexts, collect_outputs)
for _ in range(iterations - 1):
_model_iter_fn(mod, inputs, contexts, optimizer, collect_outputs=False)
return _model_iter_fn(mod, inputs, contexts, optimizer, collect_outputs=True)
Expand Down

0 comments on commit 1118332

Please sign in to comment.