ROCm-Apex v0.3.2

Critical bug fixes for ROCm4.0

ROCm-Apex v0.3.1

This is a bugfix release that addresses the following issue:

• Fix LayerNorm op on ROCm (#36)

ROCm-Apex v0.3

The following are the new extensions that are supported in this release.

• SyncBatchnorm extension.
• MLP CUDA extension
• Apex Contrib library support - Deprecated FusedLAMB and Deprecated FusedAdam.
• Apex Contrib library support - Xentropy extension.

ROCm-Apex v0.2

ROCm Apex v0.2 Release Notes:

• BFloat16 Mixed Precision Training support with single GPU.
  • Introduced new Optimization levels - O4 and O5 for BFloat16 training.
  • BFloat16 support for FusedOptimizers
  • BFloat16 support for FusedLayerNorm.
• Performance Improvements.
• Test Infrastructure and CI setup on ROCm.

Initial support of Apex on ROCm.

The following features are currently supported for Apex on ROCm:

• Apex AMP - FP16
• Extensions - Multi Tensor Apply, FusedLayerNorm, FusedOptimizers,
• apex.DDP
• Sparse Tensor support for Loss Scaling (Dynamic) [ New Feature].