model 3

Sasha Rush surveys linear RNNs and state space models as replacements for attention in transformer-style models, in a talk for MLSys. It covers the foundational preliminaries and predates Mamba.

The first lecture of Andy Pavlo's CMU 15-445 Intro to Database Systems (Fall 2019), covering course introduction and the relational model.

Andrej Karpathy's three-hour-plus general-audience deep dive into how ChatGPT-style LLMs are built, covering the full training stack from pretraining to post-training. It also gives mental models for their quirks, like hallucination, and how to use them well.

Geoffrey Hinton's 2025 Royal Institution Discourse on what AI has learned from biological intelligence and how it might surpass it. He explains neural networks and argues that human language use resembles what backpropagation-trained networks do.

Peter Thiel's lecture from the How to Start a Startup course on business strategy and monopoly theory, arguing that competition destroys profits and that startups should aim to build monopolies. It covers the ideas of Zero to One.

Clayton Christensen explains disruptive innovation using the steel industry, showing how mini-mills beat incumbents from the low end. He argues disruption is not about stupid or bad management.

Jaron Lanier delivers the 2018 Willy Brandt Lecture in Berlin on who civilization is for, covering data, AI, tech platforms and the economics of who benefits, followed by a Q&A with Germany's data protection commissioner Ulrich Kelber.

Clayton Christensen explains his theory of disruptive innovation at Oxford's Said Business School, using examples from steel, Toyota, Sony, Walmart and Godrej. He applies the theory to why some economies stagnate and others grow.

A short clip of Milton Friedman explaining his free-market view of monopoly, namely that monopolies persist mainly through government help. It is excerpted from the Milton Friedman Speaks series.

Leo Dirac explains how transformer models have replaced LSTMs in NLP. He gives background from bag of words through RNNs and LSTMs, then a technical deep dive into multi-headed self-attention and positional encoding, with sample code.

A lecture from MIT's Missing Semester course covering version control with git. It builds up git from its data model rather than from commands.

David Tong, a Cambridge theoretical physicist, explains at the Royal Institution that the fundamental building blocks of nature are quantum fields rather than particles. He covers what we know about these fields and how they fit into the Standard Model.

Peter Norvig's UBC distinguished lecture on how massive amounts of simple data, such as for language modeling, can yield understanding that hand-built models could not.

Microsoft's Brian Beckman gives an hour-long, whiteboard-style walk through monads, building from function composition and monoids using school-level math. It is aimed at practicing programmers rather than category theorists.

Database pioneer Michael Stonebraker argues that 'big data' is really at least four different problems: big volumes with small analytics, big analytics, velocity, and variety. He comments on where each market is heading.

Andrej Karpathy builds the GPT tokenizer from scratch, implementing byte pair encoding along with encode and decode. He shows how many odd LLM behaviors trace back to tokenization, and covers Unicode byte encodings along the way.

The opening lecture of Steve Brunton's Control Bootcamp, covering the kinds of control and why closed-loop feedback matters. It starts a series that follows the control chapter of the Brunton and Kutz data-driven science book.

Sebastien Bubeck presents the Sparks of AGI paper at MIT, showing early experiments with GPT-4 across many domains and asking whether artificial intelligence has arrived. It includes the TikZ unicorn example.

Turing Award winner Michael Stonebraker gives the 2018 Donald B. Gillies Memorial Lecture at Illinois. He discusses the gap between academic research and industry problems, publishing incentives, and the influence of large industry users on research.

The first lecture of MIT 6.042J Mathematics for Computer Science, taught by Tom Leighton, introducing mathematical proofs. It covers propositions, implication and basic proof methods.

Vladimir Voevodsky's IAS lecture starts from Godel's second incompleteness theorem to explore whether first-order arithmetic could be inconsistent. It motivates his vision of univalent foundations and computer-checked proofs.

A university lecture from CS480/680 by Pascal Poupart on attention mechanisms and transformer networks, worked through on a chalkboard.

The first lecture of an MIT course on Douglas Hofstadter's Godel, Escher, Bach, covering themes like self-reference, recursion and incompleteness. The full lecture series is on YouTube.

The introductory lecture of Stephen Boyd's Stanford Convex Optimization I course (EE 364A). It outlines convex sets, functions and problems, and the course's coverage of least-squares, linear and semidefinite programming, duality and interior-point methods.

Dan Britt lectures on the history of climate, covering orbital cycles, ice ages and conditions millions of years ago. It frames current climate debate with deep-time context.

A guest lecture Jon Gjengset gave at Two Sigma on using Rust for large, high-performance systems. It covers why Rust suits such projects, the Noria research database, a concurrent data structure used in it, and practical experience.

Andrej Bauer's IAS talk introduces constructive mathematics and what it feels like to do mathematics without excluded middle. It is framed as a tour for classical mathematicians.

Scott Aaronson's Harvard CMSA talk traces what mathematical truths are knowable, from Godel and Turing to the Busy Beaver function. He covers recent work on Busy Beaver values.

Part 1 of Jean-Marc Jancovici's 2019 course at Ecole des Mines de Paris, on energy and its central role in the economy and climate. It is in French with English subtitles available and runs about two and a half hours.

Thomas Garrity discusses mathematical maturity, a concept mathematicians use for professional development, and how it can inform teaching, mentoring and careers. It was given at the IAS/PCMI Park City Mathematics Institute.

An RSA Animate drawn from a Sir Ken Robinson talk arguing that current education systems are modeled on the industrial age. It discusses divergent thinking, standardization, and how schools disengage kids.

A roughly three-hour San Francisco workshop by Marshall Rosenberg on Nonviolent Communication, with full English subtitles added by a fan. He teaches the model with his jackal and giraffe language examples.

Economist Richard Wolff's Talks at Google lecture on his book Democracy at Work. He critiques capitalism and argues for worker-run cooperatives as a path to a more democratic economy.

Stanford Health Care hosts Christopher Gardner and Robert Lustig on whether all calories are equal, with a focus on processed food and sugar. The talk questions the idea that calories in versus calories out fully explains weight and health.

Dr. Bruce Bugbee walks through the calculations for crop yield potential when light is the limiting factor, from photosynthetic quantum yield through respiration efficiency and harvest index, using the Energy Cascade Model.

Andres Gomez Emilsson of the Qualia Research Institute presents a model of DMT experiences using hyperbolic geometry and dynamical systems theory. He describes how different intoxication levels correspond to changing curvature and information content of experience.

Psychologist Irving Kirsch gives a distinguished lecture on antidepressant trials and the placebo effect. He argues drug-placebo differences are small outside severe depression.

Neuroscientist Karl Friston gives a 15-minute talk on the free energy principle, covering Markov blankets, Bayesian model evidence and global brain theories.

A 1990 lecture by James Gosling on NeWS, Sun's networked window system in which clients send PostScript programs to the server for execution. It covers the high-level imaging model and compares NeWS with similar systems.

Nathan Kutz presents the spatio-temporal dynamics of rotating detonation rocket engines, including combustion fronts that behave like solitons. He shows a bifurcation analysis of a multiscale model that captures the main features.