Slides: ./lecture_llm_tricks.pdf
Videos (russian): lecture and seminar
Videos (english):
- EMNLP tutorial on PEFT by Jonas Pfeifer - https://www.youtube.com/watch?v=KoOlcX3XLd4
- If you don't have 3.5 hours, here's a short version from MunichNLP - https://www.youtube.com/watch?v=StdrAJZsmw4
Practice assignment: ./practice.ipynb ,
Extra materials (model architecture):
- "Building ML models like we build open-source software" by Colin Raffel - https://www.youtube.com/watch?v=0oGxT_i7nk8
- Rotary position embeddings explanation from EleutherAI - https://blog.eleuther.ai/rotary-embeddings/
- Group query attention to reduce the memory usage for inference - https://arxiv.org/abs/2305.13245v2
- Gated activations improve transformer (apparently due to divine benevolence) - https://arxiv.org/abs/2002.05202
- as usual, there are dozens of links in the lecture slides (top of this readme)