Let's build GPT: from scratch, in code, spelled out.
8.2M views • 3 years ago
Andrej Karpathy
Andrej Karpathy builds a GPT from scratch in code, following Attention Is All You Need and GPT-2/3, in about two hours using PyTorch. He connects it to ChatGPT and assumes familiarity with his earlier makemore videos.
