occupancy networks

Andrej Karpathy's first Neural Networks: Zero to Hero lecture, which builds micrograd, a tiny autograd engine, from scratch in Python. It explains backpropagation and neural network training step by step, assuming only basic Python and some high-school calculus.

The first of a short series by Welch Labs on building a complete neural network in Python. It introduces the data and the architecture of a simple network before later parts cover forward propagation, gradient descent, and backpropagation.

Geoffrey Hinton's 2025 Royal Institution Discourse on what AI has learned from biological intelligence and how it might surpass it. He explains neural networks and argues that human language use resembles what backpropagation-trained networks do.

Geoffrey Hinton gives an MIT talk on what is wrong with convolutional neural networks and introduces his capsules project. The description points to the capsule network papers.

The first lecture of Stanford's CS231n course on convolutional neural networks for visual recognition. It surveys the history and key challenges of computer vision, from the Cambrian explosion and Hubel and Wiesel to ImageNet.

3Blue1Brown's second neural network chapter explains cost functions and gradient descent, showing how a network learns by minimizing error during training.

Tesla's full AI Day 2022 presentation, covering the Tesla Bot hardware and software, Full Self-Driving planning and perception, the occupancy network, data labeling, simulation and Dojo training infrastructure.

A short demo video for NVIDIA's StyleGAN paper, which proposes a style-based generator for GANs. It shows face synthesis with style mixing and scale-specific control over attributes like pose and freckles.

A university lecture from CS480/680 by Pascal Poupart on attention mechanisms and transformer networks, worked through on a chalkboard.

Steven Johnson's TED talk argues that good ideas come from connected, slow-building hunches and liquid networks, from London coffee houses to Darwin to the modern web, rather than lone Eureka moments.

Two Minute Papers covers a paper on speeding up fluid and smoke simulation with convolutional networks. It contrasts the approach with earlier regression forest methods.

3Blue1Brown's first chapter on deep learning explains what neurons and layers are and the math underlying a basic neural network, using handwritten digit recognition with visual animations.

SethBling shows MarI/O, a program using neural networks and the NEAT genetic algorithm to learn to play Super Mario World. The description links the source code, the paper and background reading.

Martin Gorner's Devoxx talk introduces deep learning with TensorFlow, walking through neural networks for MNIST, how to choose an architecture and how to diagnose networks that fail to train.

Alex Russell's Chrome Dev Summit 2016 talk on building web experiences that work on slow, flaky networks and low-power devices. It argues developers should focus on the performance of real-world mobile hardware rather than desktop machines.

Max Tegmark's MIT CBMM talk on why deep learning works so well. He argues that the laws of physics and the symmetries in physical data make it compressible into small neural networks, tying physics to network structure.

BobbyBroccoli documentary on the rise of Nortel Networks, once the dominant Canadian telecom giant, during the dot-com era. It is part one of a two-part story.

Mark Rober and Jabril build an app that uses machine learning to decode baseball sign sequences. The video also explains neural networks in a simple, accessible way.

Andrej Karpathy, then Tesla's head of AI, gives a CVPR 2021 workshop keynote on how Autopilot's neural networks work. A large part covers the move to a vision-only stack, including why radar was dropped and how vision produces depth estimates.

A short cartoon explaining how language models like ChatGPT work, from autocomplete and conditional probabilities to neural networks, gradient descent and backpropagation. It shows how networks act as universal function approximators.

Geoff Hinton's 2010 Google Tech Talk on deep belief networks, learning layers of representation one at a time from unlabeled data, and extending them with multiplicative interactions.

Footage shot in New York City in 1911 by a Swedish film company, upscaled to 4K and 60 fps with neural networks and optionally colorized. The creator notes the colors are artificial and added only for ambiance.

3Blue1Brown's chapter 4 on deep learning gives a more formal, symbol-based account of backpropagation calculus, connecting the intuition from the previous chapter to the math found in textbooks and code.

The first lecture of Stanford's CS231n Winter 2016 course on convolutional neural networks for visual recognition, introducing the course and the historical context of computer vision.

Tom 7 (suckerpinch) delivers a long, comedic digression on gradient descent and floating-point imprecision. He shows how rounding behavior can supply the nonlinearity needed to build a neural network.

Microsoft Research's Rick Rashid demonstrates a speech recognition and translation system, built on deep neural networks, that turns his spoken English into Chinese speech in 2012.

Urs Hoelzle, Google's SVP of infrastructure, gives the 2012 Open Networking Summit keynote on how Google adopted OpenFlow and software-defined networking in its own networks. The slides are linked in the description.

Cloth Map documents Havana's SNET, a community-built private network that Cubans created because internet prices were high. It is used for gaming and sharing.

Machine Learning Street Talk interviews Paul Lessard about Categorical Deep Learning, which uses category theory and monads to make neural network architectures more interpretable and composable. They also discuss neural networks' limits in reasoning and unbounded computation.

Andrew Ng presents unsupervised feature learning and deep learning, showing how algorithms can learn representations from unlabeled data instead of hand-designed features. He gives examples in vision and audio, including visualizations of features learned by successive layers.

Andrej Karpathy, then Tesla's head of AI, presents at ScaledML 2020 on how Tesla builds full self-driving using vision-based neural nets and fleet data collection. He covers training custom detectors and neural nets gradually replacing traditional code.

Geoffrey Hinton's Google Tech Talk linking machine learning to biology, explaining dropout as a form of model averaging and why sexual reproduction and stochastic spiking neurons may prevent overfitting.

Veritasium explains why analog computing may suit AI workloads, covering old analog computers, The Analog Thing and Mythic's analog chips that use flash memory for neural network inference. It argues digital hardware is hitting limits.

Tom Scott-style absurdist research by tom7 for SIGBOVIK 2021, using machine learning to discover letterforms that are 'uppercaser' than uppercase A and lowercaser than lowercase. It is a comedic, technically inventive deep dive with fonts and a paper.

3Blue1Brown's third deep learning chapter gives a visual, intuitive account of backpropagation and what happens to a neural network as it learns. A follow-up video covers the underlying calculus.

Andrej Karpathy builds a bigram character-level language model from scratch, introducing torch.Tensor and the framework of training, sampling and loss evaluation. It is the first step toward a GPT-style Transformer in his makemore series.

A Welch Labs explainer on neural scaling laws and the compute-efficient frontier, asking why model loss seems to hit a line it cannot cross. It draws an analogy to an ideal gas law for AI.

A NANOG 64 panel with speakers from Comcast, APNIC, T-Mobile and Facebook on deploying IPv6-only networks, including performance data showing IPv6 outperforming NAT-ed IPv4.

A demo of a raycasting 3D engine built entirely from in-game mechanisms in vanilla Factorio 0.17. It renders simple 3D scenes, producing a new frame every 45 game ticks, with the video sped up to compensate for low UPS.

A six-minute journey through the layers of a neural network using Deep Dream, with each frame fed back into the network starting from random noise and the targeted layer changing every four seconds. It is set to downtempo music.