neural networks

Andrej Karpathy's first Neural Networks: Zero to Hero lecture, which builds micrograd, a tiny autograd engine, from scratch in Python. It explains backpropagation and neural network training step by step, assuming only basic Python and some high-school calculus.

The first of a short series by Welch Labs on building a complete neural network in Python. It introduces the data and the architecture of a simple network before later parts cover forward propagation, gradient descent, and backpropagation.

The first lecture of Stanford's CS231n course on convolutional neural networks for visual recognition. It surveys the history and key challenges of computer vision, from the Cambrian explosion and Hubel and Wiesel to ImageNet.

3Blue1Brown's second neural network chapter explains cost functions and gradient descent, showing how a network learns by minimizing error during training.

Geoffrey Hinton's 2025 Royal Institution Discourse on what AI has learned from biological intelligence and how it might surpass it. He explains neural networks and argues that human language use resembles what backpropagation-trained networks do.

Geoffrey Hinton gives an MIT talk on what is wrong with convolutional neural networks and introduces his capsules project. The description points to the capsule network papers.

3Blue1Brown's first chapter on deep learning explains what neurons and layers are and the math underlying a basic neural network, using handwritten digit recognition with visual animations.

Two Minute Papers covers a paper on speeding up fluid and smoke simulation with convolutional networks. It contrasts the approach with earlier regression forest methods.

Sentdex demonstrates GAN Theft Auto, a GameGAN-based neural network that recreates GTA V so the network itself is the playable environment. He shows the results and releases the code.

Tom 7 (suckerpinch) delivers a long, comedic digression on gradient descent and floating-point imprecision. He shows how rounding behavior can supply the nonlinearity needed to build a neural network.

SethBling shows MarI/O, a program using neural networks and the NEAT genetic algorithm to learn to play Super Mario World. The description links the source code, the paper and background reading.

Martin Gorner's Devoxx talk introduces deep learning with TensorFlow, walking through neural networks for MNIST, how to choose an architecture and how to diagnose networks that fail to train.

Veritasium explains why analog computing may suit AI workloads, covering old analog computers, The Analog Thing and Mythic's analog chips that use flash memory for neural network inference. It argues digital hardware is hitting limits.

Max Tegmark's MIT CBMM talk on why deep learning works so well. He argues that the laws of physics and the symmetries in physical data make it compressible into small neural networks, tying physics to network structure.

Mark Rober and Jabril build an app that uses machine learning to decode baseball sign sequences. The video also explains neural networks in a simple, accessible way.

Andrej Karpathy, then Tesla's head of AI, gives a CVPR 2021 workshop keynote on how Autopilot's neural networks work. A large part covers the move to a vision-only stack, including why radar was dropped and how vision produces depth estimates.

A short cartoon explaining how language models like ChatGPT work, from autocomplete and conditional probabilities to neural networks, gradient descent and backpropagation. It shows how networks act as universal function approximators.

A video presenting an Intel research paper on enhancing the photorealism of synthetic images. A convolutional network uses rendering intermediates, shown on GTA V footage that comes out looking like real street scenes.

3Blue1Brown's chapter 4 on deep learning gives a more formal, symbol-based account of backpropagation calculus, connecting the intuition from the previous chapter to the math found in textbooks and code.

The first lecture of Stanford's CS231n Winter 2016 course on convolutional neural networks for visual recognition, introducing the course and the historical context of computer vision.

Microsoft Research's Rick Rashid demonstrates a speech recognition and translation system, built on deep neural networks, that turns his spoken English into Chinese speech in 2012.

Machine Learning Street Talk interviews Paul Lessard about Categorical Deep Learning, which uses category theory and monads to make neural network architectures more interpretable and composable. They also discuss neural networks' limits in reasoning and unbounded computation.

Andrej Karpathy, then Tesla's head of AI, presents at ScaledML 2020 on how Tesla builds full self-driving using vision-based neural nets and fleet data collection. He covers training custom detectors and neural nets gradually replacing traditional code.

3Blue1Brown's third deep learning chapter gives a visual, intuitive account of backpropagation and what happens to a neural network as it learns. A follow-up video covers the underlying calculus.

A Welch Labs explainer on neural scaling laws and the compute-efficient frontier, asking why model loss seems to hit a line it cannot cross. It draws an analogy to an ideal gas law for AI.

A six-minute journey through the layers of a neural network using Deep Dream, with each frame fed back into the network starting from random noise and the targeted layer changing every four seconds. It is set to downtempo music.

A short video accompanying the CVPR 2018 paper Learning to See in the Dark. It shows a neural network producing sharp images from very low-light raw sensor data, compared against denoising approaches.

Andrej Karpathy's Databricks talk on Software 2.0, the shift from human-written code to code produced by neural network training. It covers how developer work moves toward curating datasets, the advantages and challenges, and opportunities for new tooling.

A short demo video for NVIDIA's StyleGAN paper, which proposes a style-based generator for GANs. It shows face synthesis with style mixing and scale-specific control over attributes like pose and freckles.

Andrew Ng presents unsupervised feature learning and deep learning, showing how algorithms can learn representations from unlabeled data instead of hand-designed features. He gives examples in vision and audio, including visualizations of features learned by successive layers.

Geoffrey Hinton's Google Tech Talk linking machine learning to biology, explaining dropout as a form of model averaging and why sexual reproduction and stochastic spiking neurons may prevent overfitting.

Tom Scott-style absurdist research by tom7 for SIGBOVIK 2021, using machine learning to discover letterforms that are 'uppercaser' than uppercase A and lowercaser than lowercase. It is a comedic, technically inventive deep dive with fonts and a paper.

Lex Fridman interviews George Hotz, founder of Comma.ai, in a three-hour conversation. Topics include self-driving, Tesla and Waymo, crypto, GPT-3, immortality and book recommendations.

A university lecture from CS480/680 by Pascal Poupart on attention mechanisms and transformer networks, worked through on a chalkboard.

Andrej Karpathy builds a bigram character-level language model from scratch, introducing torch.Tensor and the framework of training, sampling and loss evaluation. It is the first step toward a GPT-style Transformer in his makemore series.

Ilya Sutskever gives a Simons Institute talk on why neural networks generalize, framing learning as finding short descriptions, like compression. It was part of the Large Language Models and Transformers program.

A 2007 Google Tech Talk introducing Polyworld, an artificial life simulator used to evolve artificial intelligence by designing worlds rather than minds. It presents results from ongoing attempts to evolve intelligent behavior.

Footage shot in New York City in 1911 by a Swedish film company, upscaled to 4K and 60 fps with neural networks and optionally colorized. The creator notes the colors are artificial and added only for ambiance.

Geoff Hinton's 2010 Google Tech Talk on deep belief networks, learning layers of representation one at a time from unlabeled data, and extending them with multiplicative interactions.

A Two Minute Papers episode on NVIDIA's recurrent denoising autoencoder, which cleans up noisy Monte Carlo path-traced image sequences. It shows how a neural network can produce usable renders from very few samples.