next-token prediction

Dwarkesh Patel interviews OpenAI chief scientist Ilya Sutskever on time to AGI, what comes after generative models, data limits, alignment, and competing with Google. The title centers on whether next-token prediction could surpass human intelligence.

A Computer History Museum oral history with Avie Tevanian covering his math talent, work on the Mach microkernel at CMU, joining NeXT, leading NEXTSTEP development and the 1996 Apple acquisition.

Andrej Karpathy builds the GPT tokenizer from scratch, implementing byte pair encoding along with encode and decode. He shows how many odd LLM behaviors trace back to tokenization, and covers Unicode byte encodings along the way.

Rob Miles explains "glitch" tokens, odd strings in large language model vocabularies that make the model produce gibberish. He covers how they likely arise from tokenizer training data, such as Reddit usernames, that never appeared in the model's own training.

A 1992 MIT Sloan talk and Q&A by Steve Jobs, then CEO of NeXT, covering NeXT's strategy, co-creating Apple, his ouster, and the value of direct experience. It ranges across technology, business and career advice.

John Oliver's Last Week Tonight segment examines prediction markets such as Polymarket: the legal and moral questions, weak regulation, and insider trading and manipulation concerns.

A 35-minute demonstration of NeXTSTEP Release 3 by Steve Jobs. It shows the object-oriented development environment and features such as dragging a live chart between apps.

An internal 1991 NeXT video in which Steve Jobs explains the company's product and marketing strategy. It shows NeXT aiming at the Unix workstation market.

Steve Jobs demos NeXTSTEP Release 3, showing the object-oriented development environment and UI that later became the basis of Mac OS X. Midway through he builds a database-backed GUI app with Interface Builder and Enterprise Objects Framework with almost no code.

Fernando Rojo explains how BeatGig reached $10M in sales with one frontend engineer by sharing code between React Native and Next.js, covering navigation, animation, theming and performance.

A 1990 Lotus sales demo of Improv, a spreadsheet built for the NeXT computer that rethought the grid with named categories and multiple views. The uploader, who tested it at Price Waterhouse, presents it as an early, innovative vision of the spreadsheet.

David Nolen presents Om Next, the redesign of the ClojureScript Om library, drawing on Relay, Netflix's JSONGraph and Datomic. He covers the tradeoffs in handling the client-server boundary over immutable data.

A 1991 NeXT promotional video that stages a head-to-head programming contest between a NeXT computer and a Sun workstation. It shows NeXT's interface builder and object-oriented tooling producing an application far faster than traditional UNIX development.

Andrej Karpathy builds a bigram character-level language model from scratch, introducing torch.Tensor and the framework of training, sampling and loss evaluation. It is the first step toward a GPT-style Transformer in his makemore series.

Jeff Bell's Blender Conference 2018 talk on how Netflix's animated film Next Gen was produced using Blender. It covers the studio's pipeline and tooling.

A panel from the Lang NEXT 2014 conference on systems programming in 2014 and beyond. Language designers including Bjarne Stroustrup and Rob Pike discuss the topic, including C++ header compile times.

A NeXT, Inc. promotional video showing its highly automated factory and the manufacturing of the NeXT Cube, including automated circuit board assembly.

The day 2 keynote at Google Cloud Next '17, announcing new GCP and G Suite products with demos and customer stories from Google leaders.

Paul Graham's keynote at PyCon 2012, in which he talks about startup ideas and what kinds of companies are likely to matter in the future. He discusses ideas such as a better email client and a search engine for hackers, and argues for looking back at the present from the future.

A long Portal podcast episode in which Bret Weinstein tells Eric Weinstein the story of his lab mice telomere work, the prediction behind it, and what it means for science and aging.

Moxie Marlinspike speaks at Next Generation Threats 2014 about the challenges of encrypted communication and how Signal addresses them. He also draws on American action movies as narrative for secure communications.

Bill Gates's 2015 TED talk, given after the 2014 Ebola outbreak, argues the world is unprepared for the next fast-spreading epidemic. He calls for scenario planning, vaccine research, and trained health workers, in the way countries prepare for war.

Scott Arciszewski's DEF CON 26 Crypto and Privacy Village talk critiquing the JOSE standards (JWT, JWS, JWE) and showing how to design cryptographic tokens more safely.

A video essay arguing that the next major real-time strategy game will fail unless it gets the fundamentals right. It covers the playerbase and demographics, what older RTS games did well, and the roles of the engine, spectacle, development tools and player-generated content.

The day one keynote of Google Cloud's 2016 GCP NEXT conference, with Sundar Pichai, Diane Greene, Eric Schmidt, Urs Holzle and customers. It includes a Cloud ML demo by Jeff Dean.

Robert Moss's JuliaCon 2015 talk describes using Julia as a specification language for ACAS X, the next-generation airborne collision avoidance system that succeeds TCAS.

Jordan Hubbard, FreeBSD co-founder, discusses where FreeBSD might go over the next decade at MeetBSD California 2014. He draws on his Apple Unix experience, including bringing launchd-style init ideas to FreeBSD.

A 1979 BBC Nationwide report asking whether word processors and networked microprocessors could end office commuting. It shows early word processing and a woman working from home, with surprisingly prophetic predictions.

An AWS re:Invent 2018 session diving into the Nitro system, a set of hardware offload and security building blocks behind next-generation EC2 instances. It covers the design and architecture and how Nitro enables new instance types.

Tom Lehrer performs his satirical song Who's Next, about the spread of nuclear weapons, with a spoken introduction. It is a short musical comedy piece from the Cold War era.

Douglas Crockford's code::dive 2022 talk on what the next programming language should look like. He presents ideas behind his Misty language, including message-passing actors.

Computer History Museum oral history of Steve Naroff, an engineer at Stepstone, NeXT and Apple who worked on Objective-C. This first session covers his childhood, music background and early career.

David Nolen reviews the first five years of ClojureScript, documenting its landmarks and looking ahead to the next five. It is a history-and-roadmap talk from the Clojure community.

A one-minute clip of Steve Ballmer laughing at the original iPhone and dismissing its price and appeal.

Blizzard's Timothy Ford explains in this 2017 GDC session how Overwatch uses an Entity Component System architecture for layered gameplay. He also covers its server-authoritative netcode with client-side prediction and how they keep the game responsive and fair.

Rick Reed of WhatsApp explains at Erlang Factory 2014 how the service scaled to billions of messages with a very small engineering team. He covers the FreeBSD and Erlang stack, Mnesia usage, server counts and bottleneck work.

Seeker's Focal Point short documentary on ASML's extreme ultraviolet lithography machines, which make the most advanced chips. It shows how plasma-generated 13.5 nm light is used to push Moore's Law forward.

A short animated TED-Ed lesson in which two competing ice cream vendors on a beach illustrate why rival shops cluster together. It introduces game theory and the Nash equilibrium through the Hotelling model.

Chandler Carruth shows how to squeeze more performance out of already-optimized C++ code, covering how the CPU executes it. The talk goes into pipelining, branch prediction and speculative execution using benchmarks.

Hillel Wayne's PyCon 2018 talk on testing beyond pytest and mypy. It introduces property-based testing with the Hypothesis library and design-by-contract with dpcontracts, and shows how to combine them with an existing test suite.