nlp

Sasha Rush surveys linear RNNs and state space models as replacements for attention in transformer-style models, in a talk for MLSys. It covers the foundational preliminaries and predates Mamba.

Leo Dirac explains how transformer models have replaced LSTMs in NLP. He gives background from bag of words through RNNs and LSTMs, then a technical deep dive into multi-headed self-attention and positional encoding, with sample code.

The first lecture of Stanford's Natural Language Processing with Deep Learning course, taught by Chris Manning and Richard Socher. It introduces NLP and word vectors, including SVD, skip-gram, CBOW, negative sampling and word2vec.

Peter Norvig's 2007 Google talk on theorizing from data, covering what huge datasets can do for language understanding, translation, information extraction and inference. It argues that more data often beats cleverer models.

Peter Norvig's UBC distinguished lecture on how massive amounts of simple data, such as for language modeling, can yield understanding that hand-built models could not.

SoundHound's CEO demos Hound, a voice assistant, handling fast compound natural language queries with context and detailed criteria. The video leaked before the product's official launch.

Andrew Ng presents unsupervised feature learning and deep learning, showing how algorithms can learn representations from unlabeled data instead of hand-designed features. He gives examples in vision and audio, including visualizations of features learned by successive layers.

Emily Shea explains how repetitive strain injury pushed her to program by voice using speech recognition (Talon), and gives a live coding demo. She argues voice-driven development can be pain-free, efficient and enjoyable.

Andrej Karpathy, then Tesla's head of AI, presents at ScaledML 2020 on how Tesla builds full self-driving using vision-based neural nets and fleet data collection. He covers training custom detectors and neural nets gradually replacing traditional code.

Martin Gorner's Devoxx talk introduces deep learning with TensorFlow, walking through neural networks for MNIST, how to choose an architecture and how to diagnose networks that fail to train.

Mike Pound explains the discrete cosine transform and how it underpins JPEG compression. It is part two of Computerphile's JPEG series.

Huygens Optics explains the Fourier transform and shows how a lens performs one optically. A DLP projector is used as a Fourier filter to do simple pattern recognition on lithographic character patterns.

Andrej Karpathy builds the GPT tokenizer from scratch, implementing byte pair encoding along with encode and decode. He shows how many odd LLM behaviors trace back to tokenization, and covers Unicode byte encodings along the way.

Luis von Ahn, creator of CAPTCHA and reCAPTCHA, explains how small contributions from many people online can digitize books. He then introduces Duolingo as a way to translate the web while people learn languages.

Benn Jordan explains how Flock Safety's license plate cameras log and share vehicle locations, and explores how adversarial AI can defeat them. It covers weak security and the surveillance risks of private police-camera networks.

A short conference talk from !!Con 2017 in which Kevin Chen explains the HDR imaging algorithm from a foundational paper, framing it as a system of linear equations, and implements it in a Microsoft Excel spreadsheet. No prior image processing knowledge is needed.

David MacKay's 2007 Google Tech Talk presents Dasher, a text-entry system driven by continuous gestures and a language model, reaching about 35 words per minute with one finger and 25 hands-free. It explains the information-theory reasoning behind the design.

The SIGGRAPH 2005 demo of dual photography from Stanford, which swaps the roles of projector and camera to render a scene from the light's point of view. It includes the reveal of a hidden playing card.

Mark Rober and Jabril build an app that uses machine learning to decode baseball sign sequences. The video also explains neural networks in a simple, accessible way.

Andrej Karpathy, then Tesla's head of AI, gives a CVPR 2021 workshop keynote on how Autopilot's neural networks work. A large part covers the move to a vision-only stack, including why radar was dropped and how vision produces depth estimates.

A Computerphile video in which Dr Mike Pound and PhD student Lewis Stuart explain 3D Gaussian splatting, a quick technique that turns photos of a scene into a 3D model without much compute. It includes a demo and compares the approach to NeRFs.

Tom Scott visits the USPS Remote Encoding Center in Salt Lake City, where keyers decipher mail images machines cannot read. It shows the lengths taken to decode bad handwriting.

A player piano is made to reproduce human speech, here reciting a European Environmental Criminal Court proclamation in German at the 2009 World Venice Forum. This upload adds subtitles so the words become understandable.

Rob Miles explains on Computerphile how language models generate plausible text and how transformers improved on recurrent networks, with a look at what comes next.

A ShortCircuit (Linus Tech Tips) review of Nvidia's RTX Voice, which uses GPU-based neural networks to remove background noise from microphone audio. It demonstrates how well the filtering works.

A University of Tokyo lab demonstrates a book scanner that flips pages automatically and scans over 250 pages per minute. It uses real-time 3D recognition of the flipped pages and then restores each one to a flat image.

A long livestream of George Hotz live-coding a toy SLAM implementation, twitchslam, over two days in May 2018. It is an unedited session of Python programming with computer vision and visualization.

Lesson 1 of fast.ai's Practical Deep Learning for Coders, taught by Jeremy Howard. It takes coders from running code in a Jupyter notebook straight into training a real image model.

Neural-network upscaled, 60 fps, stabilized and colorized version of 1902 footage of the Wuppertal Schwebebahn suspended railway in Germany. The source film comes from MoMA. It gives a vivid view of an early-1900s city and a train system that still runs.

A SIGGRAPH Asia 2013 research demo of 3-Sweep, a tool for extracting editable 3D objects from a single photo using a few user strokes. It predates modern AI image tools.

A PyCon 2012 talk by Kurt Grandis on building an automated water-squirting turret that uses Python and computer vision to detect and drive off squirrels from a bird feeder.

Part 2 of the IROS 2011 keynote by Sebastian Thrun and Chris Urmson on Google's self-driving car. It shows how the car perceives its surroundings with a lidar point cloud and recognizes traffic lights and pedestrians. The audio is poor because it was recorded on a pocket camcorder.

Lex Fridman interviews Douglas Lenat, founder of Cyc, a decades-long project to encode common-sense knowledge and reasoning for AI. They discuss symbolic AI and how it may complement neural networks.

Adobe's VoCo demo from the MAX 2016 Sneak Peeks, co-hosted by Jordan Peele, showing software that changes words in a voiceover by typing new ones.

Pokey Rule presents Cursorless, a spoken language designed for editing code by voice, built on tree-sitter parsing with smart structural and simple token-level primitives. The talk is from Strange Loop 2023.

Rob Miles explains "glitch" tokens, odd strings in large language model vocabularies that make the model produce gibberish. He covers how they likely arise from tokenizer training data, such as Reddit usernames, that never appeared in the model's own training.

Luis von Ahn, creator of CAPTCHA and reCAPTCHA, presents Duolingo at TEDxCMU 2011. The idea is to have millions of people translate the web for free while learning a language, an example of human computation.

A demonstration of Larynx, an offline open source neural text-to-speech system supporting nine languages. The video itself is narrated with a Larynx-generated voice.

Footage shot in New York City in 1911 by a Swedish film company, upscaled to 4K and 60 fps with neural networks and optionally colorized. The creator notes the colors are artificial and added only for ambiance.

Geoffrey Hinton gives an MIT talk on what is wrong with convolutional neural networks and introduces his capsules project. The description points to the capsule network papers.