Skip to content

Commit

Permalink
recommend jemalloc for gpt-neox-20b 8x
Browse files Browse the repository at this point in the history
modifications to gpt-neox-20b documentation recommendation to use jemalloc instead of temalloc for this model execution
  • Loading branch information
hsubramony committed Sep 23, 2024
1 parent fc2e671 commit 373a334
Showing 1 changed file with 2 additions and 0 deletions.
2 changes: 2 additions & 0 deletions examples/language-modeling/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -137,6 +137,8 @@ The following command triggers the fine-tuning of [GPT-NeoX-20B](https://hugging
Fine-tuning on 16 HPU cards (2 Gaudi2 nodes) takes around 9 minutes with a batch size of 32 (2 per device).
It reaches a perplexity of 10.469.

**Note:** For GPT-NeoX-20B model, please switch to jemalloc in case of host OOM issues using ``` export LD_PRELOAD=<path>/libjemalloc.so.2 ```

> Please refer to [this page](https://github.com/huggingface/optimum-habana/tree/main/examples/multi-node-training) for performing multi-node training properly.
```bash
Expand Down

0 comments on commit 373a334

Please sign in to comment.