Transformers Notebooks

This repository contains my notes from Hugging Face's O'Reilly book Natural Language Processing with Transformers:

Getting started

I ran notebooks on Google Colab and commented throughout if I changed anything. Scroll to end of notebook for other insights :)

• Chapter 01. This chapter was very straightforward. My only advice being to use the Colab notebook (vs. setting up everything locally)
• Chapter 02. Scroll to end for Case Study on "Tesla". Using chapter 02, I started extending my "Laughbot" (classifying humor) here https://github.com/katepark/laughbot-transformer
• Chapter 03. Scroll to end for implementions of Decoder Layer (ie. GPT) and Encoder-Decoder Layer (ie. T5) from scratch
• (WIP) Chapter 04 At end, I include my takeaways and the steps I would take to compare XLM-R to Mad-X.

Citations

Citing the text book with BibTeX entry:

@book{tunstall2022natural,
  title={Natural Language Processing with Transformers: Building Language Applications with Hugging Face},
  author={Tunstall, Lewis and von Werra, Leandro and Wolf, Thomas},
  isbn={1098103246},
  url={https://books.google.ch/books?id=7hhyzgEACAAJ},
  year={2022},
  publisher={O'Reilly Media, Incorporated}
}