ml

Yaron Minsky of Jane Street gave this talk to Harvard's OCaml-based CS51 class in 2010. It collects advice on being an effective ML programmer in an industrial setting.

Sasha Rush surveys linear RNNs and state space models as replacements for attention in transformer-style models, in a talk for MLSys. It covers the foundational preliminaries and predates Mamba.

Andrej Karpathy's first Neural Networks: Zero to Hero lecture, which builds micrograd, a tiny autograd engine, from scratch in Python. It explains backpropagation and neural network training step by step, assuming only basic Python and some high-school calculus.

The first lecture of MIT's 6.001 (SICP) with Abelson and Sussman, giving an overview of the course and an introduction to Lisp. It argues that computer science is about managing complexity and abstraction, not computers.

Douglas Crockford's first Yahoo lecture puts JavaScript in historical context. He traces the language's structure and quirks back through the early decades of computer science and programming language history.

The first lecture of MIT's 1986 6.001 Structure and Interpretation of Computer Programs, given by Harold Abelson, introducing Lisp and the idea of computer science as the study of process.

Microsoft's Brian Beckman gives an hour-long, whiteboard-style walk through monads, building from function composition and monoids using school-level math. It is aimed at practicing programmers rather than category theorists.

Andrej Karpathy's three-hour-plus general-audience deep dive into how ChatGPT-style LLMs are built, covering the full training stack from pretraining to post-training. It also gives mental models for their quirks, like hallucination, and how to use them well.

The first lecture in Bartosz Milewski's category theory series for programmers. It covers the motivation and philosophy behind category theory and why composition matters for handling complexity.

Gabriel Lebec's Fullstack Academy talk introduces lambda calculus syntax, the history of mathematical logic, and the essence of functional programming using JavaScript. It also covers combinators, currying and Church encodings.

Simon Peyton Jones gives Churchill College's annual computer science lecture on Haskell's birth and evolution. He covers the most influential ideas, from laziness to monads and type system evolution, plus current developments and guesses about the future.

Andrej Karpathy builds the GPT tokenizer from scratch, implementing byte pair encoding along with encode and decode. He shows how many odd LLM behaviors trace back to tokenization, and covers Unicode byte encodings along the way.

Leo Dirac explains how transformer models have replaced LSTMs in NLP. He gives background from bag of words through RNNs and LSTMs, then a technical deep dive into multi-headed self-attention and positional encoding, with sample code.

Geoffrey Hinton's 2025 Royal Institution Discourse on what AI has learned from biological intelligence and how it might surpass it. He explains neural networks and argues that human language use resembles what backpropagation-trained networks do.

The first lecture of Stanford's Natural Language Processing with Deep Learning course, taught by Chris Manning and Richard Socher. It introduces NLP and word vectors, including SVD, skip-gram, CBOW, negative sampling and word2vec.

Peter Norvig's UBC distinguished lecture on how massive amounts of simple data, such as for language modeling, can yield understanding that hand-built models could not.

The first lecture of David MacKay's Cambridge course on information theory, pattern recognition and neural networks. It introduces the core ideas of his book Information Theory, Inference, and Learning Algorithms.

A university lecture from CS480/680 by Pascal Poupart on attention mechanisms and transformer networks, worked through on a chalkboard.

John Gustafson presents posits, a proposed drop-in replacement for IEEE 754 floats that is simpler in hardware and more flexible in precision. The seminar includes a live demo and machine learning results.

A Stanford EE380 seminar by Michael P. Frank of Sandia National Laboratories on generalized reversible computing. It surveys unconventional computing approaches as transistor scaling nears its limits and energy efficiency stops improving.

Sacha Arnoud, Waymo's Director of Engineering, gives a lecture in MIT's 2018 deep learning for self-driving cars course on the rise of machine learning in autonomous vehicles. It covers how Waymo uses deep learning and how it handles tricky driving situations.

Bill Dally of NVIDIA gives a University of Washington lecture on trends in deep learning hardware. It covers numeric formats, specialized instructions, energy per operation and co-design of hardware and software.

Geoffrey Hinton gives an MIT talk on what is wrong with convolutional neural networks and introduces his capsules project. The description points to the capsule network papers.

The first lecture of Stanford's CS231n course on convolutional neural networks for visual recognition. It surveys the history and key challenges of computer vision, from the Cambrian explosion and Hubel and Wiesel to ImageNet.

Lecture 7A from MIT 6.001 (SICP) by Abelson and Sussman introduces the metacircular evaluator, a Scheme interpreter written in Scheme. It shows how evaluating a program is itself a program.

Lecture 1B of MIT's 6.001 Structure and Interpretation of Computer Programs (1986) with Gerald Sussman, covering procedures, processes and the substitution model. It includes the line about the power of naming things.

An excerpt from Hal Abelson's opening SICP lecture introducing computer science and contrasting declarative and imperative knowledge. He argues that computer science is not really a science and is more like magic or engineering of ideas.

Lecture 7A of the 1986 MIT 6.001 SICP course, with Gerald Sussman presenting the metacircular evaluator, part 1. It shows how a Lisp evaluator can be written in Lisp itself.

Simon Peyton Jones's first lecture on types in Haskell from the Oregon Programming Languages Summer School 2013. It covers type classes and how they relate to OOP, subtyping and polymorphism.

Michael Betancourt's lecture at MLSS Iceland 2014 on efficient Bayesian inference with Hamiltonian Monte Carlo. It covers MCMC concepts and motivation and the problems of sampling in high-dimensional distributions.

Physicist Al Bartlett's well-known lecture on the arithmetic of exponential growth, applied to population, energy consumption and resource depletion. He argues that steady percentage growth cannot be sustained on a finite planet and that most public discussion ignores this.

Lecture 1 of Harvard's Statistics 110 with Joe Blitzstein, introducing sample spaces, the naive definition of probability, counting, binomial coefficients and the sampling table.