deep thinking

Rich Hickey's philosophical talk from the first Clojure Conj in 2010. He argues that good design comes from deliberate, away-from-keyboard thinking, and describes how to feed the background mind so it solves problems.

Max Tegmark's MIT CBMM talk on why deep learning works so well. He argues that the laws of physics and the symmetries in physical data make it compressible into small neural networks, tying physics to network structure.

A short smartphone-recorded clip from a 2014 event in Paris where DeepMind CEO Demis Hassabis presents an AI learning to play old Atari games. The system learns from only the video signal and the score. Audio quality is poor.

Full episode of Sassy Justice, a satirical local-news show from the South Park creators starring a deepfaked consumer advocate, Fred Sassy. It uses face-swapped public figures to comment on the dangers of manipulated video.

Geoff Hinton's 2010 Google Tech Talk on deep belief networks, learning layers of representation one at a time from unlabeled data, and extending them with multiplicative interactions.

A deepfake in which Bill Hader, impersonating Arnold Schwarzenegger on a talk show, visibly morphs into Schwarzenegger. It was made with DeepFaceLab by Ctrl Shift Face.

A Mandelbrot set zoom animation reaching a magnification of about 2.1x10^275, claimed as the deepest at the time (January 2010). Each frame was rendered separately over six months on 12 CPU cores.

Philosopher Daniel Dennett visits Google to discuss his book Intuition Pumps and Other Tools for Thinking, describing thought experiments and thinking tools across psychology, biology, computer science and physics.

A six-minute journey through the layers of a neural network using Deep Dream, with each frame fed back into the network starting from random noise and the targeted layer changing every four seconds. It is set to downtempo music.

Deepfake artist Shamook's improved version of the Luke Skywalker scene from The Mandalorian season 2 finale, replacing the official CGI face with a deepfake.

A short video built around an animation of the roughly 10,000 galaxies in the Hubble Ultra Deep Field, placed at their measured redshift distances. It gives a brief history of deep field images and ends with a 3D fly-through.

Martin Gorner's Devoxx talk introduces deep learning with TensorFlow, walking through neural networks for MNIST, how to choose an architecture and how to diagnose networks that fail to train.

Bret Victor's 2013 MIT Media Lab talk argues that the media we think with determine which thoughts are possible. He demos new representations for scientific papers, circuits, filters and signal processing.

Andrew Ng presents unsupervised feature learning and deep learning, showing how algorithms can learn representations from unlabeled data instead of hand-designed features. He gives examples in vision and audio, including visualizations of features learned by successive layers.

The first lecture of Stanford's Natural Language Processing with Deep Learning course, taught by Chris Manning and Richard Socher. It introduces NLP and word vectors, including SVD, skip-gram, CBOW, negative sampling and word2vec.

Lesson 1 of fast.ai's Practical Deep Learning for Coders, taught by Jeremy Howard. It takes coders from running code in a Jupyter notebook straight into training a real image model.

Alan Kay argues for rethinking how computer science is taught in schools, drawing on his work on object-oriented programming and GUIs. He discusses how poorly people understand complex systems and what education could do about it.

A Veritasium explainer on how the brain works and how we learn. It is inspired by Kahneman's Thinking, Fast and Slow and covers System 1 and System 2 thinking and why harder learning builds skill.

A two-minute Insider Tech clip on DeepMind's simulated agents that learned to walk, run, jump and climb without guidance, with the goofy resulting gaits.

Machine Learning Street Talk interviews Paul Lessard about Categorical Deep Learning, which uses category theory and monads to make neural network architectures more interpretable and composable. They also discuss neural networks' limits in reasoning and unbounded computation.

3Blue1Brown's first chapter on deep learning explains what neurons and layers are and the math underlying a basic neural network, using handwritten digit recognition with visual animations.

Valve's John McDonald presents at GDC 2018 on how Valve uses deep learning (VACnet) to detect cheaters in Counter-Strike: Global Offensive. He covers training on human-reviewed replays and how model output is confirmed by manual review.

3Blue1Brown's visual walkthrough of the attention mechanism inside transformers, covering self-attention, multiple heads and cross-attention. It is chapter 6 of his deep learning series.

A GDC 2018 talk by Valve's John McDonald on VACNet, a deep learning system that analyzes CS:GO gameplay to detect cheaters. It covers how the model is trained on human-reviewed Overwatch cases and how its output feeds human moderation.

Andrew, diagnosed with early onset Parkinson's at 35, shows how deep brain stimulation surgery has controlled his motor symptoms. The short video shows his tremors with the stimulator off and on.

3Blue1Brown's third deep learning chapter gives a visual, intuitive account of backpropagation and what happens to a neural network as it learns. A follow-up video covers the underlying calculus.

Julia Galef uses pictures to explain Bayes' rule, then tells three personal stories showing how she applies it to improve her thinking.

Daniel Kahneman's Talks at Google lecture on his book Thinking, Fast and Slow, covering System 1 and System 2 thinking and the psychology of judgment and decision making.

A roughly 10-minute guided Non-Sleep Deep Rest session voiced by neuroscientist Andrew Huberman. It is meant to reduce daytime fatigue, improve focus and aid sleep.

Bill Dally of NVIDIA gives a Hot Chips 2023 keynote on the challenges of building hardware for deep learning. He traces how GPUs have been adapted for machine learning, especially training.

Bill Dally of NVIDIA gives a University of Washington lecture on trends in deep learning hardware. It covers numeric formats, specialized instructions, energy per operation and co-design of hardware and software.

ArtosisTV's commentary on Game 1 of DeepMind's AlphaStar (Protoss) against top Zerg player Serral at Blizzcon 2019. Artosis analyzes the match with beginner-friendly commentary.

A DeepSec 2010 conference talk by Ralf Philipp Weinmann on attacking the baseband processor of mobile phones rather than the application processor. It covers how open-source GSM base stations make malicious-tower attacks practical against neglected GSM stacks.

A BuzzFeed PSA from 2018 in which a deepfaked Barack Obama, voiced by Jordan Peele, warns about fake video before revealing the trick. It is a one-minute demonstration of AI face and voice manipulation.

3Blue1Brown's visual explanation of how transformers and GPT-style language models work, tracing data through tokens, embeddings and attention.

John Ousterhout, Stanford professor and creator of Tcl, presents the ideas of his book A Philosophy of Software Design in a Talks at Google session. He covers managing complexity, deep modules and interfaces, and what he teaches in his Stanford software design course.

Leslie Lamport argues that programmers, like architects, should write rigorous specifications before coding, especially for concurrent and distributed systems. The talk introduces the case for writing specs (the TLA+ mindset) as a way of thinking above the code.

3Blue1Brown's second neural network chapter explains cost functions and gradient descent, showing how a network learns by minimizing error during training.

A 1994 talk by systems thinker Russ Ackoff, who knew W. Edwards Deming, on continuous versus discontinuous improvement. It ends with an unexpected lesson.

An RSA Animate drawing of Barbara Ehrenreich's talk on the darker side of positive thinking. It argues that forced positivity in culture, work and policy can be harmful and played a part in the financial crisis.