language models

Rob Miles explains on Computerphile how language models generate plausible text and how transformers improved on recurrent networks, with a look at what comes next.

A short cartoon explaining how language models like ChatGPT work, from autocomplete and conditional probabilities to neural networks, gradient descent and backpropagation. It shows how networks act as universal function approximators.

Rob Miles discusses GPT-3 on Computerphile, covering how scaling parameters kept improving results over GPT-2 and how the model nearly learned arithmetic without anyone knowing exactly how.

3Blue1Brown's visual explanation of how transformers and GPT-style language models work, tracing data through tokens, embeddings and attention.

Computerphile video in which Mike Pound explains a University of Maryland paper on watermarking LLM output. The method biases token choices so generated text can later be detected statistically.

Tom Scott, with linguists Gretchen McCulloch and Molly Ruhl, examines whether the language you speak changes how you think. The short answer is mostly no, with caveats about the weak version of the Sapir-Whorf hypothesis.

A guest video by Welch Labs on 3Blue1Brown explaining how text-to-image and video models work. It covers CLIP, shared embedding spaces, diffusion models (DDPM, DDIM), conditioning, guidance and negative prompts.

Alex O'Connor shows that ChatGPT cannot draw a wine glass filled to the brim, and ties the failure to David Hume's missing shade of blue. It asks what the failure says about how image models learn.

Sabine Hossenfelder explains climate sensitivity and why recent observations suggest climate models may have underestimated warming. She says this changed her view on how worried to be.

A Welch Labs explainer on neural scaling laws and the compute-efficient frontier, asking why model loss seems to hit a line it cannot cross. It draws an analogy to an ideal gas law for AI.

Rob Miles explains "glitch" tokens, odd strings in large language model vocabularies that make the model produce gibberish. He covers how they likely arise from tokenizer training data, such as Reddit usernames, that never appeared in the model's own training.

Vox explains the Berlin and Kay research showing that languages around the world tend to invent color words in the same order. It walks through the hierarchy and the debate around it.

A Technology Connections video essay on how analog and digital clocks shape different mental models of time, with the minute hand acting as a progress indicator.

A four-minute minutephysics video arguing that causal models and causal networks let statistics infer causation from correlation. It touches on Reichenbach's principle, common causes, feedback and quantum causal discovery.

Tom Scott makes a short descriptivist argument that language keeps changing and that casual internet writing conventions, like lowercase typing and dropped full stops, are legitimate. It is part of his Language Files series, written with Gretchen McCulloch.

Yuta asks whether native Japanese speakers can actually write kanji by hand, showing that many struggle to recall characters from memory.

3Blue1Brown's visual walkthrough of the attention mechanism inside transformers, covering self-attention, multiple heads and cross-attention. It is chapter 6 of his deep learning series.

Yannic Kilcher walks through the Attention Is All You Need paper, which introduced the Transformer, a sequence model based solely on attention without recurrence or convolutions. He reviews the architecture and the paper's translation results.

Kurzgesagt examines how AI-generated content is flooding the internet, from video and music to news and books. It looks at how this makes verifying what is real harder.

Musician Benn Jordan explains and demonstrates adversarial noise that poison-pills music files against AI training and scraping.

Chubbyemu presents a case study of a man who followed AI health advice, replaced salt with sodium bromide and developed bromism with severe neurological effects. It is told as a dramatized medical case by a doctor.

Musician Benn Jordan uses AI to detect AI-generated music and digs into music industry data, including streaming payouts and sample library copyright problems. He covers ways to tell whether a track is human-made.

The Map Men explain why British place names are so hard to pronounce, tracing it to layers of Viking, Anglo-Saxon, Celtic and Norman influence, with a pronunciation guide.

Aaron Francis walks through how PHP has changed from 2012 to 2023, covering traits, array destructuring, generators, typed properties and more. It is a quick chaptered tour of modern PHP features.

Huygens Optics explores the idea that space is a non-linear elastic medium, and uses simulations to show how wave energy might be confined locally into particle-like objects. It touches on inertia, the Schwinger limit and the vacuum as a medium.

A Veritasium video on how a feud between Russian mathematicians led to Markov chains and the modern prediction algorithms built on them. It covers the history and the math behind them.

An animated introduction to geometric algebra, presented as a language that can describe most of physics. It was made with manim and covers multivectors and how they generalize familiar tools like cross products and quaternions.

3Blue1Brown builds toy SIR models to simulate how an epidemic spreads and tests scenarios like social distancing, travel restrictions, central gathering places and quarantine. It was made during the early COVID-19 outbreak.

A 1936 Chevrolet-sponsored film that explains how a car transmission works by treating gears as spinning levers. It uses physical models to make the mechanism clear.

Linguist Simon Roper reconstructs how a London accent would have sounded from the 14th to the 21st centuries, reading passages in period pronunciation. It draws on the Cambridge History of the English Language.

Rick Beato argues that adults cannot develop true perfect pitch, which he ties to critical-period brain plasticity and language acquisition. He says it can only be developed in young children, and discusses the research behind this.

A five-minute animated explainer of liquid democracy, in which voters can delegate their vote to trusted people. It starts from the origins of democracy and contrasts direct and representative models.

A short explainer from Nat and Friends on where Google Earth's 3D imagery comes from, how it is stitched together, and how often it is updated. It describes aircraft carrying camera rigs and photogrammetry that turns the photos into 3D models.

PBS Space Time examines Roger Penrose's idea that consciousness arises from quantum processes in the brain, and reviews new evidence about microtubules. It ends with an estimate of what replicating consciousness would demand.

Yuta, a Japanese YouTuber, explains whether Japanese speakers can read Chinese text. He shows how shared kanji let people guess meanings while the two languages remain completely different.

A PBS Space Time episode on fuzzballs, the string-theory proposal for black holes that replaces the singularity with a tangle of strings. It explains why black holes seem to both have to exist and break known physics.

Geekerwan's Chinese-language review of Intel's Lunar Lake lightweight laptop chip, covering performance, perf-per-watt curves and battery life benchmarks. It tests whether x86 can compete on efficiency.

Tom Scott explains why subtitles often differ from dubbed dialogue, covering how translation, timing and reading speed shape each. It features translators and voice talent working in several languages.