neural rendering

Andrej Karpathy's first Neural Networks: Zero to Hero lecture, which builds micrograd, a tiny autograd engine, from scratch in Python. It explains backpropagation and neural network training step by step, assuming only basic Python and some high-school calculus.

The first of a short series by Welch Labs on building a complete neural network in Python. It introduces the data and the architecture of a simple network before later parts cover forward propagation, gradient descent, and backpropagation.

A video presenting an Intel research paper on enhancing the photorealism of synthetic images. A convolutional network uses rendering intermediates, shown on GTA V footage that comes out looking like real street scenes.

A Graphics Programming Conference talk on how the game Tiny Glade renders meadows, trees and player-built castles. It covers real-time global illumination, tilt-shift depth of field, stable shadows with continuous time of day, ray marching and GPU-driven rendering on old hardware.

A one-minute NVIDIA research demo of foveated rendering for VR using SMI eye tracking. It renders at full detail only where the eye is looking, matching the physiology of human vision.

3Blue1Brown's first chapter on deep learning explains what neurons and layers are and the math underlying a basic neural network, using handwritten digit recognition with visual animations.

Sentdex demonstrates GAN Theft Auto, a GameGAN-based neural network that recreates GTA V so the network itself is the playable environment. He shows the results and releases the code.

A Remedy engineer explains how Alan Wake 2 moved to a GPU-driven rendering pipeline built on mesh shaders, culling and rendering meshlets for dense vegetation on PS5 and Xbox Series consoles.

Brian Karis's SIGGRAPH 2021 talk explains how Unreal Engine 5's Nanite virtualized geometry works. It covers mesh import, the data structures, streaming, decompression, culling, software and hardware rasterization, and shading.

3Blue1Brown's second neural network chapter explains cost functions and gradient descent, showing how a network learns by minimizing error during training.

Geoffrey Hinton's 2025 Royal Institution Discourse on what AI has learned from biological intelligence and how it might surpass it. He explains neural networks and argues that human language use resembles what backpropagation-trained networks do.

Dan Abramov's JSConf Iceland talk on what came after React 16, with live-coded prototypes of async rendering and time slicing. It previews the ideas that became concurrent mode and suspense.

Geoffrey Hinton gives an MIT talk on what is wrong with convolutional neural networks and introduces his capsules project. The description points to the capsule network papers.

The first lecture of Stanford's CS231n course on convolutional neural networks for visual recognition. It surveys the history and key challenges of computer vision, from the Cambrian explosion and Hubel and Wiesel to ImageNet.

An excerpt from a Bay Area Rust meetup on Servo's WebRender, which renders CSS display lists on the GPU at very high frame rates. It explains the approach and demos rendering at around 500 FPS.

Two Minute Papers covers a paper on speeding up fluid and smoke simulation with convolutional networks. It contrasts the approach with earlier regression forest methods.

Euclideon's 2011 technology preview of Unlimited Detail, a rendering approach claiming unlimited geometric detail in real time via point-cloud style data. The company promised more to come while keeping most work secret.

Ryan Florence's React.js Conf 2015 talk pushes React into unusual use cases, including a demo comparing rendering in React, Ember and Angular.

Bisqwit builds a Doom or Duke Nukem 3D style software-rendered 3D engine in C from scratch, supporting non-Euclidean geometry. It covers vector rotation and portal rendering with sped-up, tool-assisted coding.

A two-minute explainer of 3D Gaussian Splatting, the rendering technique from a 2023 research paper. It outlines why it could change how 3D graphics are made.

A Two Minute Papers episode on NVIDIA's recurrent denoising autoencoder, which cleans up noisy Monte Carlo path-traced image sequences. It shows how a neural network can produce usable renders from very few samples.

A GDC 2015 talk in which Natalya Tatarchuk describes the multithreaded renderer Bungie built for Destiny. It covers the low-latency architecture across platforms, its successes and its challenges.

Media Molecule's Alex Evans recounts the journey, including the failures, of finding the Dreams engine, based on his Siggraph 2015 talk. It covers the signed-distance-field and splat-based rendering approach and the artistic goals behind it.

Veritasium explains why analog computing may suit AI workloads, covering old analog computers, The Analog Thing and Mythic's analog chips that use flash memory for neural network inference. It argues digital hardware is hitting limits.

Lex Fridman interviews George Hotz, founder of Comma.ai, in a three-hour conversation. Topics include self-driving, Tesla and Waymo, crypto, GPT-3, immortality and book recommendations.

Casey Muratori follows up his refterm proof-of-concept, a GPU-based terminal renderer, with a second part covering memory usage, glyph sizing and correct Unicode and VT handling. It is built to show that a terminal can be fast and correct without heroic effort.

A demo of Marble Marcher, a game using a fractal physics engine and fully procedural ray-marched rendering. The player rolls a marble through shifting fractal levels.

Epic's Unreal Engine 5 tech demo set in the Matrix universe, with Keanu Reeves and Carrie-Anne Moss. It moves from a cinematic to a playable car chase through a large detailed city, with a behind-the-scenes look at the rendering at the end.

The Academy's 2014 Sci-Tech Awards segment honoring Matt Pharr, Greg Humphreys and Pat Hanrahan for their physically based rendering work and book. The authors give a short acceptance speech that thanks Donald Knuth for literate programming.

Tom 7 (suckerpinch) delivers a long, comedic digression on gradient descent and floating-point imprecision. He shows how rounding behavior can supply the nonlinearity needed to build a neural network.

A short demo of real-time sparse voxel octree rendering, shown by Jon Olick at Siggraph 2008. It shows a detailed model rendered by ray casting through the octree.

SethBling shows MarI/O, a program using neural networks and the NEAT genetic algorithm to learn to play Super Mario World. The description links the source code, the paper and background reading.

Andrej Karpathy, then Tesla's head of AI, presents at ScaledML 2020 on how Tesla builds full self-driving using vision-based neural nets and fleet data collection. He covers training custom detectors and neural nets gradually replacing traditional code.

Martin Gorner's Devoxx talk introduces deep learning with TensorFlow, walking through neural networks for MNIST, how to choose an architecture and how to diagnose networks that fail to train.

Steven Wittens' JSConf US 2013 talk on drawing with WebGL, using animated visuals to walk through vectors, matrices and the rendering pipeline. Despite the title, it works as a visual primer on the linear algebra behind 3D graphics.

Chris McCord's ElixirConf 2018 keynote introduces Phoenix LiveView, which renders HTML on the server and updates the page over websockets. It includes a demo showing that latency is not the bottleneck one might expect.

3Blue1Brown's third deep learning chapter gives a visual, intuitive account of backpropagation and what happens to a neural network as it learns. A follow-up video covers the underlying calculus.

Max Tegmark's MIT CBMM talk on why deep learning works so well. He argues that the laws of physics and the symmetries in physical data make it compressible into small neural networks, tying physics to network structure.

Mark Rober and Jabril build an app that uses machine learning to decode baseball sign sequences. The video also explains neural networks in a simple, accessible way.

Andrej Karpathy, then Tesla's head of AI, gives a CVPR 2021 workshop keynote on how Autopilot's neural networks work. A large part covers the move to a vision-only stack, including why radar was dropped and how vision produces depth estimates.