Bytenet - A Pytorch Implementation

Introduction

This is a PyTorch version of Bytenet, a fully convolutional encoder-decoder network for machine translation. Notable features of this network include the stacked nature of the encoder and the decoder, the dilated (atrous) convolutions of exponentially expanding size in the encoder and decoder, and the masked (casual) convolutions in the decoder.

Requirements

• Python 3.6 (may work with other versions, but I used 3.6)
• PyTorch 0.3 or greater
• sacrebleu - BLEU scoring [not implemented]
• ctcdecode - CTC Beam Search [not implemented]
• warpctc - CTC Loss Function [not implemented]

Datasets

• WMT Europarl and News Translation '13, '14, and '15 de-en - Translation task
• Hutter Prize Wikipedia - used to test decoder net only

How to use

• Install PyTorch. I suggest using Anaconda
• Clone this repo

git clone https://github.com/dhpollack/bytenet.pytorch.git --recursive

• Download and unzip datasets
• Edit, config.json to point to the location of your datasets
• Train full network

python train_bytenet_wmt.py

• Predict training samples
• Score predictions with sacrebleu