causal models

A four-minute minutephysics video arguing that causal models and causal networks let statistics infer causation from correlation. It touches on Reichenbach's principle, common causes, feedback and quantum causal discovery.

Rob Miles explains on Computerphile how language models generate plausible text and how transformers improved on recurrent networks, with a look at what comes next.

PBS Space Time explains how spacetime geometry and light cones determine the order of events and what can cause what. It visualizes causality in special relativity.

Sasha Rush surveys linear RNNs and state space models as replacements for attention in transformer-style models, in a talk for MLSys. It covers the foundational preliminaries and predates Mamba.

Jeremy Howard's 90-minute practical tour of language models. It covers how they work, GPT-4's capabilities and limits, using the OpenAI API, code writing, data analysis and fine-tuning.

Andrej Karpathy gives a one-hour general-audience introduction to large language models, covering what they are, how they are trained, comparisons to operating systems, and security challenges such as jailbreaks.

A short cartoon explaining how language models like ChatGPT work, from autocomplete and conditional probabilities to neural networks, gradient descent and backpropagation. It shows how networks act as universal function approximators.

Emery Berger explains why performance is hard to measure and deliver on modern hardware, showing how code layout and alignment can swamp optimization effects. He presents Stabilizer for statistically sound evaluation and Coz, a causal profiler.

A silent PLOS ONE video of New Caledonian crows solving Aesop's Fable puzzles by dropping objects to raise water and reach food. It shows example trials from experiments testing causal understanding of water displacement.

Yann LeCun on the Big Technology Podcast discusses why current AI models cannot make discoveries despite their knowledge. He argues systems need abstract world models and also covers limits of AI research, investor expectations, and open source after DeepSeek.

Sabine Hossenfelder explains climate sensitivity and why recent observations suggest climate models may have underestimated warming. She says this changed her view on how worried to be.

Rob Miles discusses GPT-3 on Computerphile, covering how scaling parameters kept improving results over GPT-2 and how the model nearly learned arithmetic without anyone knowing exactly how.

3Blue1Brown's visual explanation of how transformers and GPT-style language models work, tracing data through tokens, embeddings and attention.

Andrej Karpathy's three-hour-plus general-audience deep dive into how ChatGPT-style LLMs are built, covering the full training stack from pretraining to post-training. It also gives mental models for their quirks, like hallucination, and how to use them well.

PBS Space Time argues that the speed of light is better understood as the speed of causality, the universal limit on how fast cause and effect propagate. Light travels at that speed because it is massless, but the limit itself is a property of spacetime.

Audio of Charlie Munger's 1995 Harvard speech on the psychology of human misjudgment, laying out a framework of cognitive biases and mental models for decision making. It covers factors like incentives, social proof and denial.

Martin Fowler's GOTO 2012 introduction to NoSQL databases: where they came from, their data models, how consistency thinking differs, and when to use them. He argues they won't make relational databases obsolete and introduces polyglot persistence.

PBS Space Time explains the quantum eraser experiment, in which erasing which-path information appears to reverse causality in a double-slit setup. It discusses what this means for quantum mechanics.

Sabine Hossenfelder explains the delayed choice quantum eraser and argues it does not rewrite the past, as popular accounts claim. She walks through what the experiment actually shows.

Sentdex demonstrates GAN Theft Auto, a GameGAN-based neural network that recreates GTA V so the network itself is the playable environment. He shows the results and releases the code.

A panel with Tony Hoare, Joe Armstrong and Carl Hewitt, hosted by Francesco Cesarini, on the history and future of concurrency. The three creators of CSP, Erlang and the actor model discuss their differing approaches.

A Technology Connections video essay on how analog and digital clocks shape different mental models of time, with the minute hand acting as a progress indicator.

A PBS Space Time episode hosted by Gabe Perez-Giz on the nature of spacetime and whether space and time are fundamental. It builds up reference frames and causality in a short, dense format.

Peter Norvig's UBC distinguished lecture on how massive amounts of simple data, such as for language modeling, can yield understanding that hand-built models could not.

A 2015 interview in which physicist Freeman Dyson, then 91, argues that global warming concerns are exaggerated, that CO2 is beneficial and that climate models oversimplify complex systems. It is posted on a channel that opposes climate consensus.

A VPRO documentary on quantitative analysts who designed the mathematical models and high-speed trading systems behind modern finance. It features figures such as Paul Wilmott and asks what risks come from treating markets as a machine.

UC Berkeley EECS colloquium by OpenAI's John Schulman on reinforcement learning from human feedback. He covers why RL helps with truthfulness, hallucination, and the limits of behavior cloning.

A guest video by Welch Labs on 3Blue1Brown explaining how text-to-image and video models work. It covers CLIP, shared embedding spaces, diffusion models (DDPM, DDIM), conditioning, guidance and negative prompts.

A one-minute clip from The Simpsons in which Lisa demonstrates the Anti-Tiger Rock to Homer, showing that the rock keeps away tigers because no tigers are around. It is a short comedic scene about faulty causal reasoning.

Andrej Karpathy builds a GPT from scratch in code, following Attention Is All You Need and GPT-2/3, in about two hours using PyTorch. He connects it to ChatGPT and assumes familiarity with his earlier makemore videos.

The first lecture in Martin Kleppmann's 8-lecture distributed systems series from the University of Cambridge undergraduate course. It introduces the subject, and accompanying lecture notes are available.

David MacKay's 2007 Google Tech Talk presents Dasher, a text-entry system driven by continuous gestures and a language model, reaching about 35 words per minute with one finger and 25 hands-free. It explains the information-theory reasoning behind the design.

Geoffrey Hinton's 2025 Royal Institution Discourse on what AI has learned from biological intelligence and how it might surpass it. He explains neural networks and argues that human language use resembles what backpropagation-trained networks do.

Yannic Kilcher explains how he trained a language model on three years of 4chan's /pol/ board and let it post on the site. He evaluates how good the model was and discusses the ethics and fallout.

Huygens Optics explores the idea that space is a non-linear elastic medium, and uses simulations to show how wave energy might be confined locally into particle-like objects. It touches on inertia, the Schwinger limit and the vacuum as a medium.

Alex O'Connor shows that ChatGPT cannot draw a wine glass filled to the brim, and ties the failure to David Hume's missing shade of blue. It asks what the failure says about how image models learn.

Yannic Kilcher interviews Emad Mostaque, founder of Stability AI, about the company's funding, its mission, and the release of Stable Diffusion. Topics include access to compute, handling harmful outputs, and avoiding the corrupting influence of profit.

Emery Berger's CppCon 2020 plenary on why performance is hard on modern hardware after the end of Dennard scaling. It covers benchmarking pitfalls and introduces new profiling tools for finding real bottlenecks.

Computerphile interviews AI safety researcher Rob Miles about ChatGPT: how it was trained with human feedback, why it tends to produce confident nonsense, and where it may or may not be dangerous.

Scott Jenson, a veteran UX designer formerly at Apple and Google, argues that the 'UX/UI' split pits programmers against designers and narrows ambition in open source. He makes the case that desktop UX has stagnated and that open source projects could lead experimentation beyond current interaction models.