deep learning

Max Tegmark's MIT CBMM talk on why deep learning works so well. He argues that the laws of physics and the symmetries in physical data make it compressible into small neural networks, tying physics to network structure.

Machine Learning Street Talk interviews Paul Lessard about Categorical Deep Learning, which uses category theory and monads to make neural network architectures more interpretable and composable. They also discuss neural networks' limits in reasoning and unbounded computation.

Andrew Ng presents unsupervised feature learning and deep learning, showing how algorithms can learn representations from unlabeled data instead of hand-designed features. He gives examples in vision and audio, including visualizations of features learned by successive layers.

Martin Gorner's Devoxx talk introduces deep learning with TensorFlow, walking through neural networks for MNIST, how to choose an architecture and how to diagnose networks that fail to train.

Geoff Hinton's 2010 Google Tech Talk on deep belief networks, learning layers of representation one at a time from unlabeled data, and extending them with multiplicative interactions.

Valve's John McDonald presents at GDC 2018 on how Valve uses deep learning (VACnet) to detect cheaters in Counter-Strike: Global Offensive. He covers training on human-reviewed replays and how model output is confirmed by manual review.

A GDC 2018 talk by Valve's John McDonald on VACNet, a deep learning system that analyzes CS:GO gameplay to detect cheaters. It covers how the model is trained on human-reviewed Overwatch cases and how its output feeds human moderation.

The first lecture of Stanford's Natural Language Processing with Deep Learning course, taught by Chris Manning and Richard Socher. It introduces NLP and word vectors, including SVD, skip-gram, CBOW, negative sampling and word2vec.

Lesson 1 of fast.ai's Practical Deep Learning for Coders, taught by Jeremy Howard. It takes coders from running code in a Jupyter notebook straight into training a real image model.

3Blue1Brown's first chapter on deep learning explains what neurons and layers are and the math underlying a basic neural network, using handwritten digit recognition with visual animations.

3Blue1Brown's visual walkthrough of the attention mechanism inside transformers, covering self-attention, multiple heads and cross-attention. It is chapter 6 of his deep learning series.

3Blue1Brown's third deep learning chapter gives a visual, intuitive account of backpropagation and what happens to a neural network as it learns. A follow-up video covers the underlying calculus.

Bill Dally of NVIDIA gives a Hot Chips 2023 keynote on the challenges of building hardware for deep learning. He traces how GPUs have been adapted for machine learning, especially training.

Bill Dally of NVIDIA gives a University of Washington lecture on trends in deep learning hardware. It covers numeric formats, specialized instructions, energy per operation and co-design of hardware and software.

3Blue1Brown's second neural network chapter explains cost functions and gradient descent, showing how a network learns by minimizing error during training.

3Blue1Brown's visual explanation of how transformers and GPT-style language models work, tracing data through tokens, embeddings and attention.

3Blue1Brown's chapter 4 on deep learning gives a more formal, symbol-based account of backpropagation calculus, connecting the intuition from the previous chapter to the math found in textbooks and code.

A short smartphone-recorded clip from a 2014 event in Paris where DeepMind CEO Demis Hassabis presents an AI learning to play old Atari games. The system learns from only the video signal and the score. Audio quality is poor.

A two-minute Insider Tech clip on DeepMind's simulated agents that learned to walk, run, jump and climb without guidance, with the goofy resulting gaits.

Ali Rahimi accepts the NIPS 2017 Test of Time Award and argues that machine learning has become alchemy, with practice running ahead of understanding.

The first lecture of David Silver's reinforcement learning course, introducing the field and its basic concepts. The course follows the Sutton and Barto textbook.

The first lecture of Andrew Ng's Stanford CS229 Machine Learning course, giving an overview of the course. It introduces supervised and unsupervised learning, learning theory and reinforcement learning.

The first episode of Google's Machine Learning Recipes series. It introduces machine learning and builds a first supervised-learning classifier in about six lines of Python using scikit-learn.

Ali Rahimi's NIPS 2017 Test-of-Time award talk, in which he argues that modern machine learning has become alchemy, working without being rigorously understood. He calls for more rigor and understanding in the field.

Mark Rober and Jabril build an app that uses machine learning to decode baseball sign sequences. The video also explains neural networks in a simple, accessible way.

A six-minute journey through the layers of a neural network using Deep Dream, with each frame fed back into the network starting from random noise and the targeted layer changing every four seconds. It is set to downtempo music.

Geoffrey Hinton's Google Tech Talk linking machine learning to biology, explaining dropout as a form of model averaging and why sexual reproduction and stochastic spiking neurons may prevent overfitting.

The first lecture of Caltech's CS 156 Machine Learning course by Yaser Abu-Mostafa, introducing the learning problem and supervised, unsupervised and reinforcement learning. It is one of 18 lectures in the Learning from Data course.

Sacha Arnoud, Waymo's Director of Engineering, gives a lecture in MIT's 2018 deep learning for self-driving cars course on the rise of machine learning in autonomous vehicles. It covers how Waymo uses deep learning and how it handles tricky driving situations.

Dwarkesh Patel interviews Andy Matuschak on how he learns, why memorization matters for understanding, and why most learning tools fail. The conversation also covers handling confusion and the role of tutoring.

Tom Scott-style absurdist research by tom7 for SIGBOVIK 2021, using machine learning to discover letterforms that are 'uppercaser' than uppercase A and lowercaser than lowercase. It is a comedic, technically inventive deep dive with fonts and a paper.

3kliksphilip summarizes Valve's GDC talk on VACNet, a deep-learning system that flags CS:GO cheaters and feeds the Overwatch review queue. It covers Trust Factor, VAC, and how the data is collected and analyzed.

The introductory lecture of Richard Hamming's capstone course, The Art of Doing Science and Engineering: Learning to Learn. It focuses on style of thinking and on educating rather than training, using technical material only as illustration.

Feature-length documentary following DeepMind's AlphaGo program as it prepares for and plays the 2016 match against Go champion Lee Sedol. It covers the team, the games, and the emotional reaction of the Go world to an AI beating a top human.

A 2023 University of Toronto conversation between Geoffrey Hinton and Fei-Fei Li on responsible AI development. They look back at the history of deep learning and ImageNet and discuss the ethics and risks of AI.

ArtosisTV's commentary on Game 1 of DeepMind's AlphaStar (Protoss) against top Zerg player Serral at Blizzcon 2019. Artosis analyzes the match with beginner-friendly commentary.

The first lecture of Stanford's CS231n course on convolutional neural networks for visual recognition. It surveys the history and key challenges of computer vision, from the Cambrian explosion and Hubel and Wiesel to ImageNet.

Yosh trains a reinforcement learning agent to drive Trackmania until he can no longer beat it. The video explains how the training works and the design choices involved.

The first lecture of Stanford's CS231n Winter 2016 course on convolutional neural networks for visual recognition, introducing the course and the historical context of computer vision.

Chris Lattner and Brennan Saeta present Swift for TensorFlow at the TensorFlow Dev Summit 2019, covering language-integrated automatic differentiation and tooling for machine learning in Swift.