learning

Valve's John McDonald presents at GDC 2018 on how Valve uses deep learning (VACnet) to detect cheaters in Counter-Strike: Global Offensive. He covers training on human-reviewed replays and how model output is confirmed by manual review.

Max Tegmark's MIT CBMM talk on why deep learning works so well. He argues that the laws of physics and the symmetries in physical data make it compressible into small neural networks, tying physics to network structure.

Dwarkesh Patel interviews Andy Matuschak on how he learns, why memorization matters for understanding, and why most learning tools fail. The conversation also covers handling confusion and the role of tutoring.

Ali Rahimi accepts the NIPS 2017 Test of Time Award and argues that machine learning has become alchemy, with practice running ahead of understanding.

Geoff Hinton's 2010 Google Tech Talk on deep belief networks, learning layers of representation one at a time from unlabeled data, and extending them with multiplicative interactions.

Bill Dally of NVIDIA gives a Hot Chips 2023 keynote on the challenges of building hardware for deep learning. He traces how GPUs have been adapted for machine learning, especially training.

Bill Dally of NVIDIA gives a University of Washington lecture on trends in deep learning hardware. It covers numeric formats, specialized instructions, energy per operation and co-design of hardware and software.

Chris Lattner and Brennan Saeta present Swift for TensorFlow at the TensorFlow Dev Summit 2019, covering language-integrated automatic differentiation and tooling for machine learning in Swift.

The final lecture of Richard Hamming's Learning to Learn course, from 1995, based on his well-known talk 'You and Your Research'. It covers how to do important work and build a career.

UC Berkeley EECS colloquium by OpenAI's John Schulman on reinforcement learning from human feedback. He covers why RL helps with truthfulness, hallucination, and the limits of behavior cloning.

A five-minute SuperMemo demo of incremental reading: importing a Wikipedia article on nutrition, extracting passages, making cloze deletions and finishing the item. It lists the keyboard shortcuts used.

Media Molecule's Alex Evans recounts the journey, including the failures, of finding the Dreams engine, based on his Siggraph 2015 talk. It covers the signed-distance-field and splat-based rendering approach and the artistic goals behind it.

Aaron Hsu explains why programmers often find APL harder to learn than people with no programming background. He identifies eight patterns that contrast traditional software engineering practice with APL practice, starting from Ken Iverson's language design principles.

Ali Rahimi's NIPS 2017 Test-of-Time award talk, in which he argues that modern machine learning has become alchemy, working without being rigorously understood. He calls for more rigor and understanding in the field.

A short vintage film clip of Seymour Papert, creator of Logo, on using computers to spark children's imagination and learning.

Toyota Research Institute presents an approach for teaching robots new dexterous behaviors from human demonstration. It explains why this capability matters and what advances made it possible.

Simon Knowles, co-founder and CTO of Graphcore, talks about "intelligence processors", chips designed specifically for machine learning. The talk was given at the RAAIS 2017 summit and covers Graphcore's architecture.

A 1970 demonstration of W. Timothy Gallwey, author of The Inner Game of Tennis, teaching a complete beginner to play tennis in about ten minutes. He chats with her, feeds balls and gives positive feedback instead of instruction.

Alan Kay's TED talk on teaching ideas, in which he argues the world is not what it seems and shows children learning through programming and building their own lessons with Etoys/Squeak.

Jose Valim introduces Nx, the project bringing numerical computing to Elixir, covering multi-dimensional tensors, automatic differentiation and JIT compilation to CPU and GPU.

A one-minute clip from a 2016 Toronto conference in which Geoff Hinton comments on deep learning and radiology. He suggests radiologists should stop being trained because machines would soon read scans better.

A full upload of Alan Kay's 1987 talk, with demos of several Smalltalk-era software systems and ideas about visual, kinesthetic and symbolic reasoning. It includes a Smalltalk program written by a 12-year-old and footage of the Rand tablet.

Peter Norvig's 2007 Google talk on theorizing from data, covering what huge datasets can do for language understanding, translation, information extraction and inference. It argues that more data often beats cleverer models.

Peter Norvig's UBC distinguished lecture on how massive amounts of simple data, such as for language modeling, can yield understanding that hand-built models could not.

Alan Kay argues for rethinking how computer science is taught in schools, drawing on his work on object-oriented programming and GUIs. He discusses how poorly people understand complex systems and what education could do about it.

Peter Norvig, then Director of Research at Google, talks at Startup School 2008 about leveraging large amounts of data and about technical challenges in Google projects such as image search. A central idea is that data can beat hand-written programming.

Chris Stucchio presents the ethical principles behind algorithmic decision making and shows with examples from lending, criminology and education that they cannot all be satisfied at once. The talk frames AI fairness as a set of quantitative tradeoffs.

A short clip from a 1973 NOVA episode in which Richard Feynman recalls his father teaching him that knowing the name of a bird in many languages tells you nothing about the bird.

Andrej Karpathy's Databricks talk on Software 2.0, the shift from human-written code to code produced by neural network training. It covers how developer work moves toward curating datasets, the advantages and challenges, and opportunities for new tooling.

An eight-minute excerpt of a 1983 video featuring Seymour Papert, creator of Logo, on children learning programming and abstract ideas through embodied play. It shows kids acting out the Logo turtle.

John Gustafson presents posits, a proposed drop-in replacement for IEEE 754 floats that is simpler in hardware and more flexible in precision. The seminar includes a live demo and machine learning results.

Jim Keller's talk at the 2020 Scaled Machine Learning conference on Moore's Law in the age of AI chips. He compares scalar, vector and GPU-style parallel approaches to computation.

Dragan Djuric presents Clojure libraries for GPU and numerical computing, including ClojureCL, ClojureCUDA and Neanderthal. He shows how a dynamic REPL workflow can be combined with low-level hardware acceleration for machine learning and data science.

Felienne Hermans's Strange Loop keynote on the history and pedagogy of teaching programming. She questions exploration-only approaches like Logo and discusses research supporting explicit instruction and rote practice.

Conal Elliott presents his paper on automatic differentiation, deriving a simple, generalized AD algorithm from a natural specification. Varying the representation of derivatives yields forward and reverse mode algorithms that avoid the complexity and statefulness of backpropagation.

Bret Victor's 2013 MIT Media Lab talk argues that the media we think with determine which thoughts are possible. He demos new representations for scientific papers, circuits, filters and signal processing.

Sandy Pentland of MIT presents his book on social physics, which uses big-data studies of how people interact to explain how ideas spread. He describes findings on collective intelligence and social influence.

Alan Kay visits Carnegie Mellon's School of Computer Science in 2007 for an informal conversation with faculty and students. He discusses the state of CS education and criticizes mainstream computing.

Sasha Rush surveys linear RNNs and state space models as replacements for attention in transformer-style models, in a talk for MLSys. It covers the foundational preliminaries and predates Mamba.

Andrej Karpathy, then Tesla's head of AI, gives a CVPR 2021 workshop keynote on how Autopilot's neural networks work. A large part covers the move to a vision-only stack, including why radar was dropped and how vision produces depth estimates.