This repository contains my detailed implementations of Andrej Karpathy's Neural Networks: Zero to Hero youtube lecture series & exercises in Jupyter Notebook. The notebooks go into extreme details to ensure a proper, robust fundamental understanding of the deep learning concepts being covered. This lecture series covers neural networks and how to build them from scratch in code. The course starts with the basics of backpropagation, then proceeds to multi-layer perceptrons (MLPs), convolutional neural networks (CNNs), and finally builds up to modern deep neural networks like generative pre-trained transformers (GPTs). The course also introduces and covers diagnostic tools for understanding neural networks dynamics and performance. Finally, this course covers the basics of tokenization, byte-pair encoding (BPE), and how to build a tokenizer from scratch. The field of focus in this course is language modeling (LM) because not only are language models a superb place to learn deep learning but also most of the skills learned here are immediately transferable to other fields of deep learning such as computer vision (CV).
[NN] ---> [MLP] ---> [CNN] ---> [GPT] ---> [BPE]| Notebook | Exercise | |
|---|---|---|
| Micrograd | Micrograd | Micrograd Exercises |
| Makemore 1 | Bigrams | Bigrams Exercises |
| Makemore 2 | MLP | MLP Exercises |
| Makemore 3 | BatchNorm | BatchNorm Exercises |
| Makemore 4 | BackProp | --- |
| Makemore 5 | Wavenet | Wavenet Exercises |
| GPT | GPT | 1, 2a, 2b, 3 |
| minBPE | minBPE | --- |
Still Ongoing...
Thank you Andrej Karpathy for creating and sharing the "Neural Networks: Zero to Hero" tutorial lecture series on Youtube & Github. Your dedication and expertise has made this resource possible. Find more of Andrej's work on his GitHub, Twitter, Website and AI native school, Eureka Labs.
This repository is licensed under the MIT License. Feel free to use the provided materials for educational purposes, but please attribute the original content to Andrej Karpathy.