machine learning

Ali Rahimi accepts the NIPS 2017 Test of Time Award and argues that machine learning has become alchemy, with practice running ahead of understanding.

Chris Lattner and Brennan Saeta present Swift for TensorFlow at the TensorFlow Dev Summit 2019, covering language-integrated automatic differentiation and tooling for machine learning in Swift.

Andrew Ng presents unsupervised feature learning and deep learning, showing how algorithms can learn representations from unlabeled data instead of hand-designed features. He gives examples in vision and audio, including visualizations of features learned by successive layers.

Tom Scott-style absurdist research by tom7 for SIGBOVIK 2021, using machine learning to discover letterforms that are 'uppercaser' than uppercase A and lowercaser than lowercase. It is a comedic, technically inventive deep dive with fonts and a paper.

Ali Rahimi's NIPS 2017 Test-of-Time award talk, in which he argues that modern machine learning has become alchemy, working without being rigorously understood. He calls for more rigor and understanding in the field.

Max Tegmark's MIT CBMM talk on why deep learning works so well. He argues that the laws of physics and the symmetries in physical data make it compressible into small neural networks, tying physics to network structure.

3Blue1Brown's second neural network chapter explains cost functions and gradient descent, showing how a network learns by minimizing error during training.

Yosh trains a reinforcement learning agent to drive Trackmania until he can no longer beat it. The video explains how the training works and the design choices involved.

James Mickens's humorous USENIX Security keynote argues that improving security and trusting opaque technology like machine learning is far harder than the industry admits.

3Blue1Brown's visual walkthrough of the attention mechanism inside transformers, covering self-attention, multiple heads and cross-attention. It is chapter 6 of his deep learning series.

Peter Norvig's 2007 Google talk on theorizing from data, covering what huge datasets can do for language understanding, translation, information extraction and inference. It argues that more data often beats cleverer models.

Jose Valim introduces Nx, the project bringing numerical computing to Elixir, covering multi-dimensional tensors, automatic differentiation and JIT compilation to CPU and GPU.

3Blue1Brown's visual explanation of how transformers and GPT-style language models work, tracing data through tokens, embeddings and attention.

An episode of Tim Hunkin's The Secret Life of Machines that explains how a washing machine works, using models, cut-aways and animations. It was first aired in 1988.

The introductory lecture of Richard Hamming's capstone course, The Art of Doing Science and Engineering: Learning to Learn. It focuses on style of thinking and on educating rather than training, using technical material only as illustration.

George Hotz of Comma AI gives a 2017 conference talk on lessons from building a self-driving system. He argues about how driving should be defined and why the common approach to autonomous vehicles is flawed.

Simon Knowles, co-founder and CTO of Graphcore, talks about "intelligence processors", chips designed specifically for machine learning. The talk was given at the RAAIS 2017 summit and covers Graphcore's architecture.

A Veritasium video on AlphaFold, DeepMind's protein structure prediction system, and the protein design work it enabled. It features interviews with John Jumper and with David Baker's protein design group and explains why predicting protein folding matters.

Peter Norvig, then Director of Research at Google, talks at Startup School 2008 about leveraging large amounts of data and about technical challenges in Google projects such as image search. A central idea is that data can beat hand-written programming.

Chris Stucchio presents the ethical principles behind algorithmic decision making and shows with examples from lending, criminology and education that they cannot all be satisfied at once. The talk frames AI fairness as a set of quantitative tradeoffs.

A Google Tech Talk by Laurens van der Maaten, co-creator of t-SNE, on visualizing high-dimensional data by mapping it to two dimensions. It explains how the technique works and how it compares with other visualization approaches.

The final lecture of Richard Hamming's Learning to Learn course, from 1995, based on his well-known talk 'You and Your Research'. It covers how to do important work and build a career.

Johnny Harris investigates why McDonald's ice cream machines are so often broken, covering the Taylor machines, the repair restrictions on franchisees and the Kytch workaround device.

The first lecture of Stanford's CS231n course on convolutional neural networks for visual recognition. It surveys the history and key challenges of computer vision, from the Cambrian explosion and Hubel and Wiesel to ImageNet.

Technology Connections takes apart the step-by-step electromechanical logic of a 1970s pinball machine. He shows how relays, steppers and switches track state and score without electronics.

The first lecture of Gilbert Strang's MIT 18.06 Linear Algebra course, covering the geometry of linear equations. It opens a full 35-lecture university course.

Andrej Karpathy, then Tesla's head of AI, presents at ScaledML 2020 on how Tesla builds full self-driving using vision-based neural nets and fleet data collection. He covers training custom detectors and neural nets gradually replacing traditional code.

Technology Connections takes a long, detailed look at how a 1970s electromechanical pinball machine works, from relays to stepper motors and mechanical logic. It is the first part of a series.

Yannic Kilcher walks through the Attention Is All You Need paper, which introduced the Transformer, a sequence model based solely on attention without recurrence or convolutions. He reviews the architecture and the paper's translation results.

Tim Hunkin's 1987 TV series episode on how vacuum cleaners and related household machines work, remastered and upscaled in 2021. It is part of the six-episode 'Secret Life of Machines' series with hands-on demonstrations of everyday technology.

A Welch Labs explainer on neural scaling laws and the compute-efficient frontier, asking why model loss seems to hit a line it cannot cross. It draws an analogy to an ideal gas law for AI.

A 42-minute CBS interview with Geoffrey Hinton at the Vector Institute in 2023, covering the history of neural networks and the present and future of AI. He discusses AGI timelines and whether language models truly understand.

Jim Keller's talk at the 2020 Scaled Machine Learning conference on Moore's Law in the age of AI chips. He compares scalar, vector and GPU-style parallel approaches to computation.

Dragan Djuric presents Clojure libraries for GPU and numerical computing, including ClojureCL, ClojureCUDA and Neanderthal. He shows how a dynamic REPL workflow can be combined with low-level hardware acceleration for machine learning and data science.

David Ditzel of Esperanto Technologies presents their massively parallel, energy-efficient RISC-V processors at the 7th RISC-V Workshop in 2017. It covers the high-performance many-core design aimed at machine learning.

Ilya Sutskever gives a Simons Institute talk on why neural networks generalize, framing learning as finding short descriptions, like compression. It was part of the Large Language Models and Transformers program.

Leo Dirac explains how transformer models have replaced LSTMs in NLP. He gives background from bag of words through RNNs and LSTMs, then a technical deep dive into multi-headed self-attention and positional encoding, with sample code.

An illustrated 23-minute summary of David Friedman's talk based on The Machinery of Freedom. It describes how law, police and courts could work as market services in a stateless society, covering private law, polycentric law and objections.

Andrej Karpathy gives a one-hour general-audience introduction to large language models, covering what they are, how they are trained, comparisons to operating systems, and security challenges such as jailbreaks.

Veritasium explains why analog computing may suit AI workloads, covering old analog computers, The Analog Thing and Mythic's analog chips that use flash memory for neural network inference. It argues digital hardware is hitting limits.