Learn deep learning from zero
Curated by Claude · 11 videos · 11 hr 18 min
A study plan for programmers who want to understand modern language models properly. It starts with visual intuition, moves to building backprop and a convnet by hand, covers why transformers replaced RNNs, and finishes with building and using GPT-style models. Each video assumes you have watched the ones before it. Plan for about 15 hours, and the from-scratch lectures are worth coding along with.
Play all on YouTube- 118:40
But what is a neural network? | Deep learning chapter 1
25.1M views • 9 years ago3Blue1Brown3Blue1Brown's first chapter on deep learning explains what neurons and layers are and the math underlying a basic neural network, using handwritten digit recognition with visual animations.HN's default first stop, with 16 recommenders: neurons, layers and digit recognition as animation, no code yet.
- 23:07
Neural Networks Demystified [Part 1: Data and Architecture]
838K views • 12 years agoWelch LabsThe first of a short series by Welch Labs on building a complete neural network in Python. It introduces the data and the architecture of a simple network before later parts cover forward propagation, gradient descent, and backpropagation.The first of Welch Labs' very short videos that turn the picture into a working Python network, which an HN commenter called one of the clearest walkthroughs online.
- 32:25:52
The spelled-out intro to neural networks and backpropagation: building micrograd
4.4M views • 4 years agoAndrej KarpathyAndrej Karpathy's first Neural Networks: Zero to Hero lecture, which builds micrograd, a tiny autograd engine, from scratch in Python. It explains backpropagation and neural network training step by step, assuming only basic Python and some high-school calculus.The step where backprop stops being magic, because you build a tiny autograd engine yourself, and HN's top-scored video in machine learning.
- 42:35:53
Tensorflow and deep learning - without a PhD by Martin Görner
610K views • 9 years agoDevoxxMartin Gorner's Devoxx talk introduces deep learning with TensorFlow, walking through neural networks for MNIST, how to choose an architecture and how to diagnose networks that fail to train.Two practical hours on MNIST and convnets that cover what textbooks skip: why networks fail to train and how to fix them.
- 5
28:48LSTM is dead. Long Live Transformers!
540K views • 6 years agoSeattle Applied Deep LearningLeo Dirac explains how transformer models have replaced LSTMs in NLP. He gives background from bag of words through RNNs and LSTMs, then a technical deep dive into multi-headed self-attention and positional encoding, with sample code.Leo Dirac covers bag of words, RNNs and LSTMs, then attention, so you know what problem transformers solved before you meet them.
- 6
27:07Attention Is All You Need
814K views • 8 years agoYannic KilcherYannic Kilcher walks through the Attention Is All You Need paper, which introduced the Transformer, a sequence model based solely on attention without recurrence or convolutions. He reviews the architecture and the paper's translation results.Yannic Kilcher reads the original transformer paper with you, a good point to see the source before the polished explanations.
- 7
27:14Transformers, the tech behind LLMs | Deep Learning Chapter 5
11.6M views • 2 years ago3Blue1Brown3Blue1Brown's visual explanation of how transformers and GPT-style language models work, tracing data through tokens, embeddings and attention.The visual version of the paper, which many HN commenters say is where embeddings and attention finally clicked.
- 8
1:56:20Let's build GPT: from scratch, in code, spelled out.
8.2M views • 3 years agoAndrej KarpathyAndrej Karpathy builds a GPT from scratch in code, following Attention Is All You Need and GPT-2/3, in about two hours using PyTorch. He connects it to ChatGPT and assumes familiarity with his earlier makemore videos.The capstone build: a GPT in PyTorch in two hours, which 14 HN commenters call the best video on the topic.
- 9
24:07AI can't cross this line and we don't know why.
2.5M views • 2 years agoWelch LabsA Welch Labs explainer on neural scaling laws and the compute-efficient frontier, asking why model loss seems to hit a line it cannot cross. It draws an analogy to an ideal gas law for AI.After building one model, step back to scaling laws and the compute-efficient frontier that every model seems to run up against.
- 10
39:43Do we need Attention? - Linear RNNs and State Space Models (SSMs) for NLP
35K views • 3 years agoSasha RushSasha Rush surveys linear RNNs and state space models as replacements for attention in transformer-style models, in a talk for MLSys. It covers the foundational preliminaries and predates Mamba.Sasha Rush on the main alternative to attention, so you finish knowing where the architecture debate is heading.
- 11
1:31:13A Hackers' Guide to Language Models
580K views • 3 years agoJeremy HowardJeremy Howard's 90-minute practical tour of language models. It covers how they work, GPT-4's capabilities and limits, using the OpenAI API, code writing, data analysis and fine-tuning.The practical close: Jeremy Howard on using, prompting and fine-tuning the models you now understand from the inside.