Skip to content

Commit

Permalink
Merge pull request #140 from huggingface/xrsrke/fix_print_data_stages…
Browse files Browse the repository at this point in the history
…_break_in_brrr

[Bug] Remove printing of HF dataset in data stages
  • Loading branch information
xrsrke committed Apr 19, 2024
2 parents 450fb67 + 3c872f7 commit 96ac464
Showing 1 changed file with 1 addition and 1 deletion.
2 changes: 1 addition & 1 deletion src/nanotron/trainer.py
Original file line number Diff line number Diff line change
Expand Up @@ -335,7 +335,7 @@ def clear_dataloader_from_memory(dataloader: DataLoader, stage_name: str):
clear_dataloader_from_memory(prev_dataloader, stage_name=stage.name)

log_rank(
f"[Training Stage: {stage.name}] Switching to a new dataset {stage.data.dataset.hf_dataset_or_datasets}",
f"[Training Stage: {stage.name}] Switching to a new dataset",
logger=logger,
level=logging.INFO,
rank=0,
Expand Down

0 comments on commit 96ac464

Please sign in to comment.