fourier transform

Leo Dirac explains how transformer models have replaced LSTMs in NLP. He gives background from bag of words through RNNs and LSTMs, then a technical deep dive into multi-headed self-attention and positional encoding, with sample code.

A university lecture from CS480/680 by Pascal Poupart on attention mechanisms and transformer networks, worked through on a chalkboard.

A 3Blue1Brown live lockdown math episode asking what Euler's formula e^(pi i) is actually saying. It builds intuition through exponentials, rotation and vectors landing on the unit circle.

The first lecture of David MacKay's Cambridge course on information theory, pattern recognition and neural networks. It introduces the core ideas of his book Information Theory, Inference, and Learning Algorithms.

Sasha Rush surveys linear RNNs and state space models as replacements for attention in transformer-style models, in a talk for MLSys. It covers the foundational preliminaries and predates Mamba.

Biochemist Nick Lane gives a Royal Institution lecture arguing that the Krebs cycle is central to what makes cells alive, not just genetic information. He draws on the themes of his book Transformer to link metabolism to the origin of life, death, and aging.