transformers

Rob Miles explains on Computerphile how language models generate plausible text and how transformers improved on recurrent networks, with a look at what comes next.

Leo Dirac explains how transformer models have replaced LSTMs in NLP. He gives background from bag of words through RNNs and LSTMs, then a technical deep dive into multi-headed self-attention and positional encoding, with sample code.

3Blue1Brown's visual explanation of how transformers and GPT-style language models work, tracing data through tokens, embeddings and attention.

3Blue1Brown's visual walkthrough of the attention mechanism inside transformers, covering self-attention, multiple heads and cross-attention. It is chapter 6 of his deep learning series.

Yannic Kilcher walks through the Attention Is All You Need paper, which introduced the Transformer, a sequence model based solely on attention without recurrence or convolutions. He reviews the architecture and the paper's translation results.

Ilya Sutskever gives a Simons Institute talk on why neural networks generalize, framing learning as finding short descriptions, like compression. It was part of the Large Language Models and Transformers program.

Andrej Karpathy gives a one-hour general-audience introduction to large language models, covering what they are, how they are trained, comparisons to operating systems, and security challenges such as jailbreaks.

Sebastien Bubeck presents the Sparks of AGI paper at MIT, showing early experiments with GPT-4 across many domains and asking whether artificial intelligence has arrived. It includes the TikZ unicorn example.

Geoffrey Hinton's 2025 Royal Institution Discourse on what AI has learned from biological intelligence and how it might surpass it. He explains neural networks and argues that human language use resembles what backpropagation-trained networks do.

Andrej Karpathy walks through how GPT assistants are trained, from tokenization and pretraining to supervised finetuning and RLHF, and gives practical prompting strategies and a view of the tooling ecosystem.

Sasha Rush surveys linear RNNs and state space models as replacements for attention in transformer-style models, in a talk for MLSys. It covers the foundational preliminaries and predates Mamba.

Nicholas Carlini of Anthropic talks at [un]prompted 2026 about how adversaries could misuse current LLMs for attacks. He shows recent models finding zero-day vulnerabilities in heavily tested software, including a SQL injection in Ghost.

OpenAI's 2021 live demo of Codex translating natural language into code, from Hello World to building a game and a Microsoft Word plugin.

Alex O'Connor shows that ChatGPT cannot draw a wine glass filled to the brim, and ties the failure to David Hume's missing shade of blue. It asks what the failure says about how image models learn.

A Welch Labs explainer on neural scaling laws and the compute-efficient frontier, asking why model loss seems to hit a line it cannot cross. It draws an analogy to an ideal gas law for AI.

Yannic Kilcher interviews Emad Mostaque, founder of Stability AI, about the company's funding, its mission, and the release of Stable Diffusion. Topics include access to compute, handling harmful outputs, and avoiding the corrupting influence of profit.

A 42-minute CBS interview with Geoffrey Hinton at the Vector Institute in 2023, covering the history of neural networks and the present and future of AI. He discusses AGI timelines and whether language models truly understand.

Yann LeCun on the Big Technology Podcast discusses why current AI models cannot make discoveries despite their knowledge. He argues systems need abstract world models and also covers limits of AI research, investor expectations, and open source after DeepSeek.

Computerphile interviews AI safety researcher Rob Miles about ChatGPT: how it was trained with human feedback, why it tends to produce confident nonsense, and where it may or may not be dangerous.

Musician Benn Jordan explains and demonstrates adversarial noise that poison-pills music files against AI training and scraping.

Marques Brownlee reviews the Humane AI Pin and calls it the worst product he has reviewed, arguing almost no one should buy it. He explains why smartphones still beat it as a daily carry device.

WSJ's Joanna Stern interviews Apple's Craig Federighi and Greg Joswiak at WWDC 2025 about Siri's delays, AI strategy, the Liquid Glass redesign in iOS 26, iPad versus Mac, and tariffs.

Daniel Stenberg's keynote at FrOSCon 2025 on AI slop attacks on the curl project, with examples of bogus AI-generated security reports. He discusses the toll on maintainers and why various countermeasures don't fit the project.

Andrej Karpathy's keynote at AI Startup School laying out software 1.0, 2.0 and 3.0, with LLMs seen as utilities, fabs and operating systems. It covers LLM quirks, partial-autonomy apps and keeping a human in the generate-and-verify loop.

Physicist Angela Collier critiques 'vibe physics', the idea that non-experts or billionaires can make scientific discoveries by chatting with AI chatbots, drawing a parallel with vibe coding.

Steve Yegge shows Efrit, a tool that gives Claude full control of Emacs, on the Vibe Coding with Steve and Gene show. Claude opens info pages, runs shells and even draws a Mandelbrot fractal in a buffer.

Corridor Crew explain how they made an anime short by filming themselves and running the footage through custom-trained Stable Diffusion models. The video walks through the technique and its open-source foundations.

A guest video by Welch Labs on 3Blue1Brown explaining how text-to-image and video models work. It covers CLIP, shared embedding spaces, diffusion models (DDPM, DDIM), conditioning, guidance and negative prompts.

OpenAI's livestream demo of GPT-4 with Greg Brockman, showing its capabilities and limitations. Highlights include writing code, turning a hand-drawn sketch into a website, and a tax-question example.

The keynote launch video for the Rabbit r1 handheld AI device, presenting its assistant and a teachable mode for agents. It is a product announcement with live-style demos.

Tom Gally, a former professional translator and University of Tokyo professor, discusses in April 2023 how GPT-4 and similar models may affect translators' careers. He draws on Japanese-English translation experience.

OpenAI cofounder Greg Brockman explains the design principles behind ChatGPT and demos unreleased plug-ins at TED 2023. In the follow-up, TED head Chris Anderson presses him on the timeline of development and the risks of releasing such a tool.

A Y Combinator video on vibe coding, the term Andrej Karpathy coined for letting LLMs write code. It reports what YC founders say about debugging, models and how much of their code is AI-written.

Dwarkesh Patel interviews OpenAI chief scientist Ilya Sutskever on time to AGI, what comes after generative models, data limits, alignment, and competing with Google. The title centers on whether next-token prediction could surpass human intelligence.

Rob Miles discusses GPT-3 on Computerphile, covering how scaling parameters kept improving results over GPT-2 and how the model nearly learned arithmetic without anyone knowing exactly how.

A demo of a Skyrim mod that uses Inworld AI to let players hold free-form conversations with NPCs. It previews LLM-driven characters in role-playing games.

Part two of a StrictlyVC interview with OpenAI CEO Sam Altman, focused on AI. He discusses regulation, commodification of AI, Google's reluctance to release its own models, and best- and worst-case futures.

Internet of Bugs analyzes Cognition's Devin demo frame by frame, arguing the video did not show Devin completing paid Upwork jobs as claimed, and showing what the task required versus what Devin produced.