deep thinking

The first lecture of Stanford's Natural Language Processing with Deep Learning course, taught by Chris Manning and Richard Socher. It introduces NLP and word vectors, including SVD, skip-gram, CBOW, negative sampling and word2vec.

Bill Dally of NVIDIA gives a University of Washington lecture on trends in deep learning hardware. It covers numeric formats, specialized instructions, energy per operation and co-design of hardware and software.

Andrej Karpathy's three-hour-plus general-audience deep dive into how ChatGPT-style LLMs are built, covering the full training stack from pretraining to post-training. It also gives mental models for their quirks, like hallucination, and how to use them well.

Sacha Arnoud, Waymo's Director of Engineering, gives a lecture in MIT's 2018 deep learning for self-driving cars course on the rise of machine learning in autonomous vehicles. It covers how Waymo uses deep learning and how it handles tricky driving situations.

An RSA Animate drawn from a Sir Ken Robinson talk arguing that current education systems are modeled on the industrial age. It discusses divergent thinking, standardization, and how schools disengage kids.

The first lecture of Stanford's CS231n course on convolutional neural networks for visual recognition. It surveys the history and key challenges of computer vision, from the Cambrian explosion and Hubel and Wiesel to ImageNet.

Leo Dirac explains how transformer models have replaced LSTMs in NLP. He gives background from bag of words through RNNs and LSTMs, then a technical deep dive into multi-headed self-attention and positional encoding, with sample code.

Timothy Gowers thinks aloud, in real time, as he tries to understand why Pascal's determinant equals 1. He does not fully solve it in this episode, but shows his exploratory process.

A university lecture from CS480/680 by Pascal Poupart on attention mechanisms and transformer networks, worked through on a chalkboard.

Douglas Hofstadter lectures at Uppsala University on how analogy and daring thinking drove Einstein's discovery of light quanta. He argues deep insight in physics comes from imaginative leaps rather than rigid logic.

Geoffrey Hinton gives an MIT talk on what is wrong with convolutional neural networks and introduces his capsules project. The description points to the capsule network papers.

The opening lecture of Robert Sapolsky's Stanford course on Human Behavioral Biology, explaining the course premise and how to avoid categorical thinking. It starts a roughly 25-lecture series on how biology, genes, hormones and environment shape behavior.

Larry McEnerney of the University of Chicago Writing Program gives a graduate-student session on writing effectively. His core idea is that readers care about value to them, not the writer's thinking process.

The first lecture of MIT's 6.001 (SICP) with Abelson and Sussman, giving an overview of the course and an introduction to Lisp. It argues that computer science is about managing complexity and abstraction, not computers.

Alan Kay's Stanford Startup School lecture covers his career at Xerox PARC and how to think about inventing the future. He goes through what came out of PARC and why companies fail to capitalize on such ideas.

Andrej Karpathy builds the GPT tokenizer from scratch, implementing byte pair encoding along with encode and decode. He shows how many odd LLM behaviors trace back to tokenization, and covers Unicode byte encodings along the way.

Leonard Bernstein's 1973 Norton Lectures at Harvard, starting with Musical Phonology. He draws analogies between music and linguistics, including references to Chomsky.

Daniel Dennett's Royal Institution lecture on whether the brain is a computer and where its software comes from. He argues that minds are built by evolution and culture, with competence arising without comprehension.

The introductory lecture of Richard Hamming's capstone course, The Art of Doing Science and Engineering: Learning to Learn. It focuses on style of thinking and on educating rather than training, using technical material only as illustration.

Leonard Susskind lectures at Stanford on how gravity and quantum information theory have merged into a new way of thinking about physics. He covers entanglement and complexity, with applications from black holes to strange metals and the foundations of quantum mechanics.

Historian Timothy Snyder explains the politics of inevitability and the politics of eternity, two ways of thinking about time that frame his books On Tyranny and The Road to Unfreedom.

Chris Shore of the Centre for Computing History explains how Colossus was built, how it worked, and how it affected the course of World War II. It is an hour-long museum lecture on the codebreaking computer.

Energy scholar Vaclav Smil argues that global energy transitions are slow and that renewable targets should not rest on wishful thinking. He appraises how energy dependencies are likely to change and asks for using much less.

A segment from Carl Sagan's original Cosmos explaining the fourth spatial dimension through analogy with a two-dimensional world and shadow projection.

Chuck Spinney, a long-time associate of John Boyd, presents Boyd's thinking on the OODA loop and evolutionary epistemology. The talk runs about two and a half hours and includes Q&A.

Dan Britt lectures on the history of climate, covering orbital cycles, ice ages and conditions millions of years ago. It frames current climate debate with deep-time context.

A lecture by Swami Sarvapriyananda at IIT Kanpur on the Mandukya Upanishad and the question 'Who am I?'. It explains the four states of consciousness in Advaita Vedanta.

The second talk in the Mill CPU series by Mill Computing, covering the Belt, the Mill's operand-passing model that replaces general registers. It explains the cost of register renaming in conventional wide CPUs and how the Belt reaches high instruction-level parallelism.

Guy Kawasaki's UC Berkeley Haas talk on the top ten mistakes entrepreneurs make, covering a startup from inception to exit.