llm training

Andrej Karpathy's three-hour-plus general-audience deep dive into how ChatGPT-style LLMs are built, covering the full training stack from pretraining to post-training. It also gives mental models for their quirks, like hallucination, and how to use them well.

Nicholas Carlini of Anthropic talks at [un]prompted 2026 about how adversaries could misuse current LLMs for attacks. He shows recent models finding zero-day vulnerabilities in heavily tested software, including a SQL injection in Ghost.

Lex Fridman's long podcast interview with Yann LeCun on Meta AI, open source, the limits of LLMs, AGI and the future of AI.

A short clip from a lecture Q&A in which Richard Stallman appears to pick something off his foot and eat it.

A short compilation of Steve Ballmer energetically shouting and dancing on stage at Microsoft events, including the famous developers chant.

A short clip of Steve Ballmer at a Microsoft event chanting "developers" over and over. It became a lasting internet meme.

A short explainer on the history and mechanics of public key cryptography and the Diffie-Hellman key exchange. It uses a mixing-paint-colors analogy to show how two parties can agree on a shared secret in public.

Doug McGuff presents the biochemistry behind high-intensity strength training and the paleo diet, including how the body stores and burns fat. It is a roughly hour-and-a-half full-length lecture.

A short extended clip of Steve Ballmer sweating and chanting "developers" on stage at a .NET presentation around 1999 or 2000.

A 2012 lecture by Richard Stallman at Reykjavik University arguing that copyright, built for the printing press, fits poorly with computer networks. He criticizes draconian enforcement and corporate lobbying.

Andrej Karpathy gives a one-hour general-audience introduction to large language models, covering what they are, how they are trained, comparisons to operating systems, and security challenges such as jailbreaks.

Andrej Karpathy builds the GPT tokenizer from scratch, implementing byte pair encoding along with encode and decode. He shows how many odd LLM behaviors trace back to tokenization, and covers Unicode byte encodings along the way.

An episode of Gwynne Dyer's 1983 documentary series War, about how military basic training turns recruits into soldiers. It covers the psychological side of conditioning.

Andrej Karpathy walks through how GPT assistants are trained, from tokenization and pretraining to supervised finetuning and RLHF, and gives practical prompting strategies and a view of the tooling ecosystem.

Richard Stallman's TEDxGeneva 2014 talk describing how nonfree programs give companies control over users and what users can do to regain control. It introduces the free software movement and the four freedoms.

Lex Fridman interviews MIT psycholinguist Edward Gibson on human language, syntax, grammar and how large language models relate to them. The conversation runs nearly three hours.

A short Art of the Problem clip explaining Diffie-Hellman key exchange, using a paint mixing analogy for one-way functions and touching on modular arithmetic and discrete logarithms.

Footage of FAA altitude chamber hypoxia training for pilots in which a participant passes out from oxygen deprivation. It shows how quickly and subtly cognition fails at altitude.

A short clip of Richard Stallman singing the free software song, recorded during a visit to Ecuador.

A Ray Summit 2022 keynote fireside chat between OpenAI's Greg Brockman and Anyscale's Robert Nishihara. They discuss how OpenAI's training infrastructure evolved and its use of Ray.

A short clip from MIX08 in which Microsoft CEO Steve Ballmer grabs Guy Kawasaki's MacBook Air from its envelope during a Q&A and mocks it.

A two-minute clip of Steve Ballmer laughing at the $500 price of the newly announced iPhone and dismissing its lack of a keyboard. It captures a famously wrong early reaction from an industry executive.

A one-minute clip of Steve Ballmer laughing at the original iPhone and dismissing its price and appeal.

A one-minute 1986 promo of Microsoft Windows 1.0 presented by Steve Ballmer, with a hyperbolic price pitch ending in the tag line about Nebraska.

3Blue1Brown's visual explanation of how transformers and GPT-style language models work, tracing data through tokens, embeddings and attention.

Yann LeCun on the Big Technology Podcast discusses why current AI models cannot make discoveries despite their knowledge. He argues systems need abstract world models and also covers limits of AI research, investor expectations, and open source after DeepSeek.

A No Priors podcast conversation with Andrej Karpathy covering self-driving (Tesla versus Waymo), the Optimus robot, bottlenecks in AI development, and his education project Eureka Labs.

Dwarkesh Patel interviews Richard Sutton, reinforcement learning pioneer, Turing Award winner and author of The Bitter Lesson. Sutton argues LLMs are a dead end because they cannot learn on the job, and that continual learning from experience will require a new architecture.

Rick Hartley gives a roughly two-hour live training on proper grounding in PCB design. He explains how energy actually flows in the space between conductors, and how that governs noise, EMI, return paths and signal reflections.

Zach Tellman's Clojure West talk covers the basics of queueing theory and its practical use in systems, including backpressure, how queues break down under load, and pitfalls of naive core.async use.

A 1973 AT&T training film made for new employees at Bell Labs' Holmdel Computing Center, when Unix was in its third version and running at 16 computer centers in the Bell System. It shows how users submitted jobs and used the center's machines.

Yosh trains a reinforcement learning agent to drive Trackmania until he can no longer beat it. The video explains how the training works and the design choices involved.

A clip in which Eben Moglen tells how he grew up when software was freer and how his path crossed Richard Stallman's. He describes how they came to work together to make software free again.

Tom Gally, a former professional translator and University of Tokyo professor, discusses in March 2023 whether GPT-4 and similar models could translate literary and popular fiction. He gives his assessment of Japanese-to-English quality.

A demo of a Skyrim mod that uses Inworld AI to let players hold free-form conversations with NPCs. It previews LLM-driven characters in role-playing games.

A South Park clip in which Butters' dad takes a job at an Amazon fulfillment center to make ends meet. It satirizes warehouse work conditions.

A 1953 US Navy training film on the basic mechanisms in fire control computers: shafts, gears, cams and differentials. It shows how these mechanical parts compute ship and target firing solutions.

Jeremy Howard's 90-minute practical tour of language models. It covers how they work, GPT-4's capabilities and limits, using the OpenAI API, code writing, data analysis and fine-tuning.

A 1930s Chevrolet training film explaining the principles and development of the differential gear. It builds the idea step by step from first principles to the finished mechanism.

Rick Beato argues that adults cannot develop true perfect pitch, which he ties to critical-period brain plasticity and language acquisition. He says it can only be developed in young children, and discusses the research behind this.