deep modules

The first lecture of Stanford's Natural Language Processing with Deep Learning course, taught by Chris Manning and Richard Socher. It introduces NLP and word vectors, including SVD, skip-gram, CBOW, negative sampling and word2vec.

Bill Dally of NVIDIA gives a University of Washington lecture on trends in deep learning hardware. It covers numeric formats, specialized instructions, energy per operation and co-design of hardware and software.

Andrej Karpathy's three-hour-plus general-audience deep dive into how ChatGPT-style LLMs are built, covering the full training stack from pretraining to post-training. It also gives mental models for their quirks, like hallucination, and how to use them well.

Sacha Arnoud, Waymo's Director of Engineering, gives a lecture in MIT's 2018 deep learning for self-driving cars course on the rise of machine learning in autonomous vehicles. It covers how Waymo uses deep learning and how it handles tricky driving situations.

The first lecture of Stanford's CS231n course on convolutional neural networks for visual recognition. It surveys the history and key challenges of computer vision, from the Cambrian explosion and Hubel and Wiesel to ImageNet.

Leo Dirac explains how transformer models have replaced LSTMs in NLP. He gives background from bag of words through RNNs and LSTMs, then a technical deep dive into multi-headed self-attention and positional encoding, with sample code.

A university lecture from CS480/680 by Pascal Poupart on attention mechanisms and transformer networks, worked through on a chalkboard.

Geoffrey Hinton gives an MIT talk on what is wrong with convolutional neural networks and introduces his capsules project. The description points to the capsule network papers.

The first lecture of MIT's 6.001 (SICP) with Abelson and Sussman, giving an overview of the course and an introduction to Lisp. It argues that computer science is about managing complexity and abstraction, not computers.

Andrej Karpathy builds the GPT tokenizer from scratch, implementing byte pair encoding along with encode and decode. He shows how many odd LLM behaviors trace back to tokenization, and covers Unicode byte encodings along the way.

Leonard Bernstein's 1973 Norton Lectures at Harvard, starting with Musical Phonology. He draws analogies between music and linguistics, including references to Chomsky.

Chris Shore of the Centre for Computing History explains how Colossus was built, how it worked, and how it affected the course of World War II. It is an hour-long museum lecture on the codebreaking computer.

Dan Britt lectures on the history of climate, covering orbital cycles, ice ages and conditions millions of years ago. It frames current climate debate with deep-time context.

Douglas Hofstadter lectures at Uppsala University on how analogy and daring thinking drove Einstein's discovery of light quanta. He argues deep insight in physics comes from imaginative leaps rather than rigid logic.

A lecture by Swami Sarvapriyananda at IIT Kanpur on the Mandukya Upanishad and the question 'Who am I?'. It explains the four states of consciousness in Advaita Vedanta.

The second talk in the Mill CPU series by Mill Computing, covering the Belt, the Mill's operand-passing model that replaces general registers. It explains the cost of register renaming in conventional wide CPUs and how the Belt reaches high instruction-level parallelism.