supervised learning

The first lecture of Andrew Ng's Stanford CS229 Machine Learning course, giving an overview of the course. It introduces supervised and unsupervised learning, learning theory and reinforcement learning.

The first lecture of Caltech's CS 156 Machine Learning course by Yaser Abu-Mostafa, introducing the learning problem and supervised, unsupervised and reinforcement learning. It is one of 18 lectures in the Learning from Data course.

Machine Learning Street Talk interviews Paul Lessard about Categorical Deep Learning, which uses category theory and monads to make neural network architectures more interpretable and composable. They also discuss neural networks' limits in reasoning and unbounded computation.

Valve's John McDonald presents at GDC 2018 on how Valve uses deep learning (VACnet) to detect cheaters in Counter-Strike: Global Offensive. He covers training on human-reviewed replays and how model output is confirmed by manual review.

A GDC 2018 talk by Valve's John McDonald on VACNet, a deep learning system that analyzes CS:GO gameplay to detect cheaters. It covers how the model is trained on human-reviewed Overwatch cases and how its output feeds human moderation.

Geoff Hinton's 2010 Google Tech Talk on deep belief networks, learning layers of representation one at a time from unlabeled data, and extending them with multiplicative interactions.

Dwarkesh Patel interviews Andy Matuschak on how he learns, why memorization matters for understanding, and why most learning tools fail. The conversation also covers handling confusion and the role of tutoring.

The first lecture of David Silver's reinforcement learning course, introducing the field and its basic concepts. The course follows the Sutton and Barto textbook.

Martin Gorner's Devoxx talk introduces deep learning with TensorFlow, walking through neural networks for MNIST, how to choose an architecture and how to diagnose networks that fail to train.

Veritasium's Derek Muller gives a Perimeter Institute livestream on how AI might change teaching and learning. He draws on past technology shifts and cognitive science to argue that learning requires effortful repetition.

The first lecture of Stanford's Natural Language Processing with Deep Learning course, taught by Chris Manning and Richard Socher. It introduces NLP and word vectors, including SVD, skip-gram, CBOW, negative sampling and word2vec.

Lesson 1 of fast.ai's Practical Deep Learning for Coders, taught by Jeremy Howard. It takes coders from running code in a Jupyter notebook straight into training a real image model.

Geoffrey Hinton's Google Tech Talk linking machine learning to biology, explaining dropout as a form of model averaging and why sexual reproduction and stochastic spiking neurons may prevent overfitting.

Bill Dally of NVIDIA gives a Hot Chips 2023 keynote on the challenges of building hardware for deep learning. He traces how GPUs have been adapted for machine learning, especially training.

Bill Dally of NVIDIA gives a University of Washington lecture on trends in deep learning hardware. It covers numeric formats, specialized instructions, energy per operation and co-design of hardware and software.

Andrew Huberman introduces how the nervous system produces sensation, emotion and behavior, and how neuroplasticity lets us change it. It covers neurons, synapses, brain chemicals and the rhythms governing focus, learning and sleep.

Sacha Arnoud, Waymo's Director of Engineering, gives a lecture in MIT's 2018 deep learning for self-driving cars course on the rise of machine learning in autonomous vehicles. It covers how Waymo uses deep learning and how it handles tricky driving situations.

UC Berkeley EECS colloquium by OpenAI's John Schulman on reinforcement learning from human feedback. He covers why RL helps with truthfulness, hallucination, and the limits of behavior cloning.

Lex Fridman interviews George Hotz, founder of Comma.ai, in a three-hour conversation. Topics include self-driving, Tesla and Waymo, crypto, GPT-3, immortality and book recommendations.

The first lecture of David MacKay's Cambridge course on information theory, pattern recognition and neural networks. It introduces the core ideas of his book Information Theory, Inference, and Learning Algorithms.

A 2023 University of Toronto conversation between Geoffrey Hinton and Fei-Fei Li on responsible AI development. They look back at the history of deep learning and ImageNet and discuss the ethics and risks of AI.

Jonathan Blow gives a roughly 90-minute talk, followed by Q&A, on how video games could actually improve education, in contrast to what is currently being tried.

The first lecture of Stanford's CS231n Winter 2016 course on convolutional neural networks for visual recognition, introducing the course and the historical context of computer vision.

Dwarkesh Patel interviews Richard Sutton, reinforcement learning pioneer, Turing Award winner and author of The Bitter Lesson. Sutton argues LLMs are a dead end because they cannot learn on the job, and that continual learning from experience will require a new architecture.

Dwarkesh Patel interviews Andrej Karpathy on why reinforcement learning is flawed, why AGI may blend into steady GDP growth, why self-driving took so long, and the future of education. The conversation runs about two and a half hours.

A long Machine Learning Street Talk interview with Kenneth Stanley, creator of NEAT and novelty search, about his book on why greatness cannot be planned. They discuss open-endedness, the tyranny of objectives, serendipity and the evolution of ideas.

Feature-length documentary following DeepMind's AlphaGo program as it prepares for and plays the 2016 match against Go champion Lee Sedol. It covers the team, the games, and the emotional reaction of the Go world to an AI beating a top human.

Harvard physicist Eric Mazur explains how he realized his students were memorizing rather than understanding, and how he changed his teaching. He presents peer instruction and data showing improved student performance.

A full upload of Alan Kay's 1987 talk, with demos of several Smalltalk-era software systems and ideas about visual, kinesthetic and symbolic reasoning. It includes a Smalltalk program written by a 12-year-old and footage of the Rand tablet.

Raymond Hettinger's PyBay 2019 talk on the mental habits behind effective programming, drawn from coaching, teaching and StackOverflow. He covers problem decomposition, chunking, managing working memory and getting unstuck.

Peter Norvig's UBC distinguished lecture on how massive amounts of simple data, such as for language modeling, can yield understanding that hand-built models could not.

A university lecture from CS480/680 by Pascal Poupart on attention mechanisms and transformer networks, worked through on a chalkboard.

Alan Kay argues for rethinking how computer science is taught in schools, drawing on his work on object-oriented programming and GUIs. He discusses how poorly people understand complex systems and what education could do about it.

Geoffrey Hinton gives an MIT talk on what is wrong with convolutional neural networks and introduces his capsules project. The description points to the capsule network papers.

Physicist and YouTuber Angela Collier argues in this hour-long video that AI as commonly described does not really exist. She covers the limits of the tools, biased datasets, and why they should not be used to make decisions.

Mike Acton, in a HandmadeCon 2015 interview with Casey Muratori, discusses game engine development, hiring and learning on your own.

Douglas Crockford's 2009 Google tech talk on the good and bad parts of JavaScript, based on his book. He argues for using a reliable subset of the language and avoiding its flawed features.

Andrej Karpathy builds a GPT from scratch in code, following Attention Is All You Need and GPT-2/3, in about two hours using PyTorch. He connects it to ChatGPT and assumes familiarity with his earlier makemore videos.

Michael Steil and Christian Hessmann's 34C3 talk explaining the Apollo Guidance Computer end to end: hardware design, operating system, and mission software.

The first lecture of Jordan Peterson's 2017 Maps of Meaning course, setting out the Cold War context for his theory of belief and culture. He introduces the narrative structure of culture and archetypal characters for the individual, the known and the unknown.