v0.9.0

Burn v0.9.0 sees the addition of the Burn Book, a new model repository, and many new operations and optimizations.

Burn Book

The Burn Book is available at https://burn-rs.github.io/book/

• Burn Book setup and plan @nathanielsimard @wdoppenberg @antimora
• Motivation & Getting started @louisfd @nathanielsimard
• Basic Workflow: from training to inference @nathanielsimard @louisfd
• Building blocks @nathanielsimard
• ONNX models @antimora
• Advanced sections @nathanielsimard

Model repository

The Model repository is available at https://github.com/burn-rs/models

• Setup @nathanielsimard
• Add SqueezeNet @antimora
• Multiple models made with Burn @Gadersd
  • Llama 2
  • Whisper
  • Stable Diffusion v1.4

Changes to Burn

Neural networks

• Three new optimizers
  • AdamW @wdoppenberg
  • AdaGrad @CohenAriel
  • RMSProp @AuruTus
• Custom initializer for transformer-related modules @wbrickner
• Cross Entropy with label smoothing and weights @ArvidHammarlund

Tensors

• Many new operators
  • cast @trfdeer @nathanielsimard
  • clamp, clamp_min, clamp_max @antimora
  • abs @mmalczak
  • max_pool1d, max_pool with dilation @caiopiccirillo
  • adaptive_avg_pool 1d and 2d @nathanielsimard
  • conv_transpose 1d and 2d, with backward @nathanielsimard
  • Not operator @louisfd
  • Dim iterator @ArvidHammarlund
• More tests for basic tensor ops @louisfd

Training

• New training metrics @Elazrod56
  • CPU temperature and use
  • GPU temperature
  • Memory use
• Custom training and validation metric loggers @nathanielsimard
• Migration from log4rs to tracing, better integration in a GUI app @dae
• Training interruption @dae
• New custom optimize method @nathanielsimard

Backends

• WGPU backend
  • Autotune @louisfd @nathanielsimard
    • Cache optimization @agelas
  • Pseudo-random number generator @louisfd
  • Fix configs @nathanielsimard
  • Matmul optimization @louisfd
• ndarray backend
  • Optimization of argmin/argmax @DrChat
  • Optimization of conv2d @DrChat
• Candle backend @louisfd
  • Support for all basic operations
  • Work in progress

Dataset

• Option for with or without replacement in dataset sampler @nathanielsimard

Import & ONNX

• Refactor, performance, tests and fixes @antimora @Luni-4 @nathanielsimard, @Gadersd
• New operators @Luni-4 @antimora @AuruTus
  • Reshape
  • Transpose
  • Binary operators
  • Concat
  • Dropout
  • Avg pool
  • Softmax
  • Conv1d, Conv2d
  • Scalar and constants
  • tanh
  • clip

Fix

• Hugging Face downloader Windows support @Macil
• Fix grad replace and autodiff backward broadcast @nathanielsimard
• Fix processed count at learning completion @dae
• Adjust some flaky tests @dae
• Ability to disable experiment logging @dae

Configuration

• Rewrite publish and checks scripts in Rust, with cargo-xtask @Luni-4 @DrChat
• Add Typos verification to checks @caiopiccirillo @antimora
• Checks for Python and venv environment @mashirooooo
• Feature flags for crates in different scenarios @dae

Documentation

• Configuration doc for vscode environment setup @caiopiccirillo
• Jupyter notebook examples @antimora
• Readme updated @louisfd

Thanks

Thanks to all aforemetioned contributors and to our sponsors @smallstepman and @premAI-io.