optimization

A guest lecture Jon Gjengset gave at Two Sigma on using Rust for large, high-performance systems. It covers why Rust suits such projects, the Noria research database, a concurrent data structure used in it, and practical experience.

Leo Dirac explains how transformer models have replaced LSTMs in NLP. He gives background from bag of words through RNNs and LSTMs, then a technical deep dive into multi-headed self-attention and positional encoding, with sample code.

Anders Hejlsberg whiteboards how compilers are taught versus how modern ones are actually built. He explains the shift to compilers as services with incremental, error-tolerant, IDE-oriented design, as in Roslyn and TypeScript.

The first lecture of Stanford's Natural Language Processing with Deep Learning course, taught by Chris Manning and Richard Socher. It introduces NLP and word vectors, including SVD, skip-gram, CBOW, negative sampling and word2vec.

Peter Norvig's UBC distinguished lecture on how massive amounts of simple data, such as for language modeling, can yield understanding that hand-built models could not.

The first lecture of David MacKay's Cambridge course on information theory, pattern recognition and neural networks. It introduces the core ideas of his book Information Theory, Inference, and Learning Algorithms.

A university lecture from CS480/680 by Pascal Poupart on attention mechanisms and transformer networks, worked through on a chalkboard.

John Gustafson presents posits, a proposed drop-in replacement for IEEE 754 floats that is simpler in hardware and more flexible in precision. The seminar includes a live demo and machine learning results.

Sacha Arnoud, Waymo's Director of Engineering, gives a lecture in MIT's 2018 deep learning for self-driving cars course on the rise of machine learning in autonomous vehicles. It covers how Waymo uses deep learning and how it handles tricky driving situations.

Geoffrey Hinton gives an MIT talk on what is wrong with convolutional neural networks and introduces his capsules project. The description points to the capsule network papers.

The first lecture of Stanford's CS231n course on convolutional neural networks for visual recognition. It surveys the history and key challenges of computer vision, from the Cambrian explosion and Hubel and Wiesel to ImageNet.

Lecture 7A from MIT 6.001 (SICP) by Abelson and Sussman introduces the metacircular evaluator, a Scheme interpreter written in Scheme. It shows how evaluating a program is itself a program.

A long Mill Computing lecture on the Mill CPU's compiler toolchain, covering the specializer that targets family members, scheduling and latency passes, and the belt. It explains how the compiler handles a wide-issue, exposed-pipeline design.