Code

Implemented Mamba Model#32

Code

Merged

amrit110 merged 21 commits intomainVectorInstitute/odyssey:mainfrom feature/mambaVectorInstitute/odyssey:feature/mambaCopy head branch name to clipboardMay 1, 2024

+4,300-116

Commits

Commits on Apr 18, 2024

• Setup the code infrastructure for Mamba.

  Adibvafa

  committed
  f96877b

  Copy full SHA for f96877b

Commits on Apr 19, 2024

• Fixed the issue with duplicate task tokens in tokenizer.

  Adibvafa

  committed
  f79233e

  Copy full SHA for f79233e

• Minor improvements to the pipeline.

  Adibvafa

  committed
  3c0afd9

  Copy full SHA for 3c0afd9

• Updated the config file for finetuning. Setup primary model script for Mamba.

  Adibvafa

  committed
  26793eb

  Copy full SHA for 26793eb

Commits on Apr 30, 2024

• Implemented Mamba pretrain model.

  Adibvafa

  committed
  d6d5dec

  Copy full SHA for d6d5dec

• Updated pretrain and finetune scripts to use with Mamba.

  Adibvafa

  committed
  8887ad3

  Copy full SHA for 8887ad3

• Update data and tokenizer modules to work with Mamba.

  Adibvafa

  committed
  bee6963

  Copy full SHA for bee6963

• Minor improvements to interp and evals packages.

  Adibvafa

  committed
  148e2a6

  Copy full SHA for 148e2a6

• Minor updates to bigbird config.

  Adibvafa

  committed
  fcc7b76

  Copy full SHA for fcc7b76

• Cleared the cell outputs of TestAnalysis.ipynb

  Adibvafa

  committed
  561e490

  Copy full SHA for 561e490

• Fixed bugs with the Mamba pretraining pipeline.

  Adibvafa

  committed
  4b0eb78

  Copy full SHA for 4b0eb78

• Updated Mamba training loop with maximum possible batch size.

  Adibvafa

  committed
  c0102d4

  Copy full SHA for c0102d4

• Minor style improvements.

  Adibvafa

  committed
  a6ab852

  Copy full SHA for a6ab852

• [pre-commit.ci] Add auto fixes from pre-commit.com hooks

  Show description for 618854b
  pre-commit-ci[bot]

  committed
  618854b

  Copy full SHA for 618854b

Commits on May 1, 2024

• Renamed the notebook used for debugging Mamba.

  Adibvafa

  committed
  e6d946c

  Copy full SHA for e6d946c

• Fixed merge conflicts.

  Adibvafa

  committed
  cef4afe

  Copy full SHA for cef4afe

• [pre-commit.ci] Add auto fixes from pre-commit.com hooks

  Show description for 08af4fb
  pre-commit-ci[bot]

  committed
  08af4fb

  Copy full SHA for 08af4fb

• Implemented a decoder dataset for training Mamba.

  Adibvafa

  committed
  17ebdb7

  Copy full SHA for 17ebdb7

• Implemented Mamba related HuggingFace utility functions and the MambaForSequenceClassification model for finetuning.

  Adibvafa

  committed
  4dab179

  Copy full SHA for 4dab179

• Fixed style merge conflicts.

  Adibvafa

  committed
  bd28037

  Copy full SHA for bd28037

• [pre-commit.ci] Add auto fixes from pre-commit.com hooks

  Show description for 7e46abf
  pre-commit-ci[bot]

  committed
  7e46abf

  Copy full SHA for 7e46abf