datacenter networks

John Ousterhout's Netdev 0x16 keynote arguing that nearly every aspect of TCP is wrong for the datacenter, from connections and streams to reliable delivery and congestion control. He presents the case for a replacement protocol.

Scott Manley explains the thermal balance of spacecraft and how a satellite could be configured to radiate away the heat from datacenter-scale computing in space. His rough numbers suggest it is feasible.

James Hamilton of AWS gives a Velocity 2010 keynote on datacenter infrastructure innovation, focused on power and cooling costs and a breakdown of where datacenter spending goes.

Andrej Karpathy's first Neural Networks: Zero to Hero lecture, which builds micrograd, a tiny autograd engine, from scratch in Python. It explains backpropagation and neural network training step by step, assuming only basic Python and some high-school calculus.

The first of a short series by Welch Labs on building a complete neural network in Python. It introduces the data and the architecture of a simple network before later parts cover forward propagation, gradient descent, and backpropagation.

Geoffrey Hinton's 2025 Royal Institution Discourse on what AI has learned from biological intelligence and how it might surpass it. He explains neural networks and argues that human language use resembles what backpropagation-trained networks do.

Geoffrey Hinton gives an MIT talk on what is wrong with convolutional neural networks and introduces his capsules project. The description points to the capsule network papers.

The first lecture of Stanford's CS231n course on convolutional neural networks for visual recognition. It surveys the history and key challenges of computer vision, from the Cambrian explosion and Hubel and Wiesel to ImageNet.

3Blue1Brown's second neural network chapter explains cost functions and gradient descent, showing how a network learns by minimizing error during training.

Urs Hoelzle, Google's SVP of infrastructure, gives the 2012 Open Networking Summit keynote on how Google adopted OpenFlow and software-defined networking in its own networks. The slides are linked in the description.

A short demo video for NVIDIA's StyleGAN paper, which proposes a style-based generator for GANs. It shows face synthesis with style mixing and scale-specific control over attributes like pose and freckles.

A university lecture from CS480/680 by Pascal Poupart on attention mechanisms and transformer networks, worked through on a chalkboard.

Steven Johnson's TED talk argues that good ideas come from connected, slow-building hunches and liquid networks, from London coffee houses to Darwin to the modern web, rather than lone Eureka moments.

Two Minute Papers covers a paper on speeding up fluid and smoke simulation with convolutional networks. It contrasts the approach with earlier regression forest methods.

3Blue1Brown's first chapter on deep learning explains what neurons and layers are and the math underlying a basic neural network, using handwritten digit recognition with visual animations.

SethBling shows MarI/O, a program using neural networks and the NEAT genetic algorithm to learn to play Super Mario World. The description links the source code, the paper and background reading.

Martin Gorner's Devoxx talk introduces deep learning with TensorFlow, walking through neural networks for MNIST, how to choose an architecture and how to diagnose networks that fail to train.

Alex Russell's Chrome Dev Summit 2016 talk on building web experiences that work on slow, flaky networks and low-power devices. It argues developers should focus on the performance of real-world mobile hardware rather than desktop machines.

Benn Jordan investigates infrasound and low-frequency noise from data centers, including xAI's Colossus and a crypto mine, and reports his own experiments on the effects. The video covers the health and community impacts on nearby residents.

A two-minute 2008 clip from Sun's Fishworks team in which Brendan Gregg uses DTrace to show disk latency spiking while he shouts at a rack of drives. Bryan Cantrill narrates the demonstration.

Max Tegmark's MIT CBMM talk on why deep learning works so well. He argues that the laws of physics and the symmetries in physical data make it compressible into small neural networks, tying physics to network structure.

BobbyBroccoli documentary on the rise of Nortel Networks, once the dominant Canadian telecom giant, during the dot-com era. It is part one of a two-part story.

Mark Rober and Jabril build an app that uses machine learning to decode baseball sign sequences. The video also explains neural networks in a simple, accessible way.

Andrej Karpathy, then Tesla's head of AI, gives a CVPR 2021 workshop keynote on how Autopilot's neural networks work. A large part covers the move to a vision-only stack, including why radar was dropped and how vision produces depth estimates.

A short cartoon explaining how language models like ChatGPT work, from autocomplete and conditional probabilities to neural networks, gradient descent and backpropagation. It shows how networks act as universal function approximators.

Geoff Hinton's 2010 Google Tech Talk on deep belief networks, learning layers of representation one at a time from unlabeled data, and extending them with multiplicative interactions.

Footage shot in New York City in 1911 by a Swedish film company, upscaled to 4K and 60 fps with neural networks and optionally colorized. The creator notes the colors are artificial and added only for ambiance.

3Blue1Brown's chapter 4 on deep learning gives a more formal, symbol-based account of backpropagation calculus, connecting the intuition from the previous chapter to the math found in textbooks and code.

Google's Amin Vahdat gives the ONS 2015 keynote on Google's datacenter networking, including its custom hardware, software-defined networking and very high bisection bandwidth.

The first lecture of Stanford's CS231n Winter 2016 course on convolutional neural networks for visual recognition, introducing the course and the historical context of computer vision.

Tom 7 (suckerpinch) delivers a long, comedic digression on gradient descent and floating-point imprecision. He shows how rounding behavior can supply the nonlinearity needed to build a neural network.

Microsoft Research's Rick Rashid demonstrates a speech recognition and translation system, built on deep neural networks, that turns his spoken English into Chinese speech in 2012.

Cloth Map documents Havana's SNET, a community-built private network that Cubans created because internet prices were high. It is used for gaming and sharing.

Machine Learning Street Talk interviews Paul Lessard about Categorical Deep Learning, which uses category theory and monads to make neural network architectures more interpretable and composable. They also discuss neural networks' limits in reasoning and unbounded computation.

A short video of five people carrying a live, running web server across the city by public transport to a new datacenter. It stayed powered and connected to the internet the whole way, so there was no downtime.

Andrew Ng presents unsupervised feature learning and deep learning, showing how algorithms can learn representations from unlabeled data instead of hand-designed features. He gives examples in vision and audio, including visualizations of features learned by successive layers.

Andrej Karpathy, then Tesla's head of AI, presents at ScaledML 2020 on how Tesla builds full self-driving using vision-based neural nets and fleet data collection. He covers training custom detectors and neural nets gradually replacing traditional code.

Geoffrey Hinton's Google Tech Talk linking machine learning to biology, explaining dropout as a form of model averaging and why sexual reproduction and stochastic spiking neurons may prevent overfitting.

Bryan Cantrill of Oxide Computer Company gives a Stanford EE380 talk on rethinking the server. He argues that hyperscaler-style rack-scale machines with open firmware, RISC-V and Rust should be available to everyone, not just the giants.

James Hamilton's Tuesday Night Live keynote at AWS re:Invent 2016, touring AWS infrastructure at scale including networking, power and custom silicon. It also features NASA JPL's Tom Soderstrom.