.

README.md

@@ -63,6 +63,20 @@ torchrun --nproc_per_node=1 run_generate.py --ckpt-path checkpoints/10/ --tp 1 -
 # We could set a larger TP for faster generation, and a larger PP in case of very large models.
 ```
 
+### Custom examples
+You can find more examples in the [`/examples`](/examples) directory:
+<!-- Make a table of the examples we support -->
+| Example | Description |
+| --- | --- |
+| `custom-dataloader` | Plug a custom dataloader to nanotron |
+| `datatrove` | Use the datatrove library to load data |
+| `doremi` | Use DoReMi to speed up training |
+| `mamba` | Train an example Mamba model |
+| `moe` | Train an example Mixture-of-Experts (MoE) model |
+| `mup` | Use spectral µTransfer to scale up your model |
+We're working on adding more examples soon! Feel free to add a PR to add your own example. 🚀
+
+
 ## Features
 We currently support the following features:
 - [x] 3D parallelism (DP+TP+PP)