README.md

---

This is a fork of version 1.0.0a0+e3fafbd (Sept/Oct 2021) of Fairseq that has been adapted for the methods and experiments in the CipherDAug paper.

Requirements and Installation

• PyTorch version >= 1.5.0
• Python version >= 3.6
• For training new models, you'll also need an NVIDIA GPU and NCCL
• To install fairseq and develop locally:

git clone https://github.com/pytorch/fairseq
cd fairseq
pip install --editable ./

# on MacOS:
# CFLAGS="-stdlib=libc++" pip install --editable ./

# to install the latest stable release (0.10.x)
# pip install fairseq

• For faster training install NVIDIA's apex library:

git clone https://github.com/NVIDIA/apex
cd apex
pip install -v --no-cache-dir --global-option="--cpp_ext" --global-option="--cuda_ext" \
  --global-option="--deprecated_fused_adam" --global-option="--xentropy" \
  --global-option="--fast_multihead_attn" ./

Changes/Additions for CipherDAug

These are the main changes to Fairseq

• tasks - supports switchout family

  • multilingual translation with evaluation
  • multilingual translation with cipherdaug extends the task above ⬆️

• data - the tasks depend on these

  • language triple dataset -- very much like language pair dataset for the classic translation task, but supports multisource [source1 + source2]:arrow_right:target
  • improved general purpose multilingual data manager that supports evaluation and language pair/triple datasets necessary for multisource

• criterions - loss functions

  • symmetric KL loss can be easily changed to jensen-shannon divergence as well

There might be a few tiny modifications here and there that are not listed here but can be easily traced through a code walkthrough.

Join the fairseq community

• Twitter: https://twitter.com/fairseq
• Facebook page: https://www.facebook.com/groups/fairseq.users
• Google group: https://groups.google.com/forum/#!forum/fairseq-users

License

fairseq(-py) is MIT-licensed. The license applies to the pre-trained models as well.

Citation

Please cite as:

@inproceedings{ott2019fairseq,
  title = {fairseq: A Fast, Extensible Toolkit for Sequence Modeling},
  author = {Myle Ott and Sergey Edunov and Alexei Baevski and Angela Fan and Sam Gross and Nathan Ng and David Grangier and Michael Auli},
  booktitle = {Proceedings of NAACL-HLT 2019: Demonstrations},
  year = {2019},
}