ROCm-Apex v0.2
ROCm Apex v0.2 Release Notes:
- BFloat16 Mixed Precision Training support with single GPU.
- Introduced new Optimization levels - O4 and O5 for BFloat16 training.
- BFloat16 support for FusedOptimizers
- BFloat16 support for FusedLayerNorm.
- Performance Improvements.
- Test Infrastructure and CI setup on ROCm.