Releases: AI-Hypercomputer/maxtext
Releases · AI-Hypercomputer/maxtext
MoE v1.0.0
MoE v1.0.0 supports:
- Megablox with Fully Sharded Data Parallelism (FSDP) and Token Parallelism (TP)
- Dropping strategies with FSDP, TP, and Expert Parallelism (EP)
MoE v1.0.0 supports: