Skip to content

Commit

Permalink
use iter to compare with gwd_start_iter
Browse files Browse the repository at this point in the history
Summary:
X-link: facebookresearch/FBGEMM#292

`self.step` is used for logging purposes only and is not properly checkpointed/reloaded. Instead, `iter` should be used as it's reloaded from external https://fburl.com/code/48caflz4

Reviewed By: spcyppt

Differential Revision: D63616621

fbshipit-source-id: fb7aacf0ad57088595b8b0d39bee3076b308d394
  • Loading branch information
Wang Zhou authored and facebook-github-bot committed Sep 30, 2024
1 parent 00f2fd5 commit 93dcc07
Showing 1 changed file with 3 additions and 2 deletions.
Original file line number Diff line number Diff line change
Expand Up @@ -1910,16 +1910,17 @@ def forward( # noqa: C901
),
)
elif self._used_rowwise_adagrad_with_global_weight_decay:
iter_ = int(self.iter.item())
apply_global_weight_decay = (
self.step >= self.gwd_start_iter and self.training
iter_ >= self.gwd_start_iter and self.training
)
return self._report_io_size_count(
"fwd_output",
invokers.lookup_rowwise_adagrad.invoke(
common_args,
self.optimizer_args,
momentum1,
iter=int(self.iter.item()),
iter=iter_,
apply_global_weight_decay=apply_global_weight_decay,
prev_iter_dev=self.prev_iter_dev,
gwd_lower_bound=self.gwd_lower_bound,
Expand Down

0 comments on commit 93dcc07

Please sign in to comment.