image processing

A guest video by Welch Labs on 3Blue1Brown explaining how text-to-image and video models work. It covers CLIP, shared embedding spaces, diffusion models (DDPM, DDIM), conditioning, guidance and negative prompts.

Lesson 1 of fast.ai's Practical Deep Learning for Coders, taught by Jeremy Howard. It takes coders from running code in a Jupyter notebook straight into training a real image model.

GeoGuessr pro Rainbolt plays head-to-head against an AI built by Stanford students that guesses locations from street-view images. The video shows the AI repeatedly beating him.

Alex O'Connor shows that ChatGPT cannot draw a wine glass filled to the brim, and ties the failure to David Hume's missing shade of blue. It asks what the failure says about how image models learn.

A short side-by-side demo of a large image loading over a slow connection in JPEG, progressive mozjpeg, JPEG XL and AVIF. It shows how each codec renders partial data.

Benn Jordan's video about storing a PNG image in a living bird, a starling that learns to reproduce encoded sounds. It mixes ornithology, audio engineering, and data encoding.

Veritasium visits the US Navy's Crane facility to test top night vision goggles. It explains how image intensifier tubes and thermal or digital systems work and compares the generations of the technology.

A short Techquickie explainer on why laptop webcams are much worse than phone cameras. It covers sensor size, image processing, and the interface differences between laptops and phones, with input from a Logitech engineer.

Tom Scott visits the USPS Remote Encoding Center in Salt Lake City, where keyers decipher mail images machines cannot read. It shows the lengths taken to decode bad handwriting.

A short demo of an open-source Krita plugin that uses Stable Diffusion to generate and edit images in place. It adds ControlNet scribble and line-art support. You select an area, push a button, and new matching content appears.

A 2023 University of Toronto conversation between Geoffrey Hinton and Fei-Fei Li on responsible AI development. They look back at the history of deep learning and ImageNet and discuss the ethics and risks of AI.

Jim Keller's talk at the 2020 Scaled Machine Learning conference on Moore's Law in the age of AI chips. He compares scalar, vector and GPU-style parallel approaches to computation.

Acerola explores building a shader that turns games into ASCII art, walking through techniques such as brightness mapping and edge detection to see whether it looks good.

Andrej Karpathy, then Tesla's head of AI, presents at ScaledML 2020 on how Tesla builds full self-driving using vision-based neural nets and fleet data collection. He covers training custom detectors and neural nets gradually replacing traditional code.

Huygens Optics explains the Fourier transform and shows how a lens performs one optically. A DLP projector is used as a Fourier filter to do simple pattern recognition on lithographic character patterns.

Andrej Karpathy builds the GPT tokenizer from scratch, implementing byte pair encoding along with encode and decode. He shows how many odd LLM behaviors trace back to tokenization, and covers Unicode byte encodings along the way.

Benn Jordan explains how Flock Safety's license plate cameras log and share vehicle locations, and explores how adversarial AI can defeat them. It covers weak security and the surveillance risks of private police-camera networks.

Andrej Karpathy, then Tesla's head of AI, gives a CVPR 2021 workshop keynote on how Autopilot's neural networks work. A large part covers the move to a vision-only stack, including why radar was dropped and how vision produces depth estimates.

A Computerphile video in which Dr Mike Pound and PhD student Lewis Stuart explain 3D Gaussian splatting, a quick technique that turns photos of a scene into a 3D model without much compute. It includes a demo and compares the approach to NeRFs.

A ShortCircuit (Linus Tech Tips) review of Nvidia's RTX Voice, which uses GPU-based neural networks to remove background noise from microphone audio. It demonstrates how well the filtering works.

Neural-network upscaled, 60 fps, stabilized and colorized version of 1902 footage of the Wuppertal Schwebebahn suspended railway in Germany. The source film comes from MoMA. It gives a vivid view of an early-1900s city and a train system that still runs.

Lex Fridman interviews Douglas Lenat, founder of Cyc, a decades-long project to encode common-sense knowledge and reasoning for AI. They discuss symbolic AI and how it may complement neural networks.

Pokey Rule presents Cursorless, a spoken language designed for editing code by voice, built on tree-sitter parsing with smart structural and simple token-level primitives. The talk is from Strange Loop 2023.

Rob Miles explains "glitch" tokens, odd strings in large language model vocabularies that make the model produce gibberish. He covers how they likely arise from tokenizer training data, such as Reddit usernames, that never appeared in the model's own training.

A demonstration of Larynx, an offline open source neural text-to-speech system supporting nine languages. The video itself is narrated with a Larynx-generated voice.

Footage shot in New York City in 1911 by a Swedish film company, upscaled to 4K and 60 fps with neural networks and optionally colorized. The creator notes the colors are artificial and added only for ambiance.

Andrej Karpathy's CVPR 2021 presentation on Tesla's Autopilot, covering its vision-based approach, sensor choices and the scale of Tesla's deep learning compute cluster. This upload is a lower-quality copy of the talk.

Denis Shiryaev's neural-network-upscaled, colorized 60 fps version of film footage shot in Paris between 1896 and 1900.

Steve Mould explains motion amplification and video magnification, techniques that find tiny changes in video invisible to the eye and exaggerate them. He covers the MIT research and industrial uses such as diagnosing vibrating machinery.

FortNine looks at why Tesla Autopilot has rear-ended motorcyclists, speculating that the vision system misreads motorcycle taillights as a distant car. It argues that radar and other sensors would help.

Tom Gally, a former professional translator and University of Tokyo professor, discusses in March 2023 whether GPT-4 and similar models could translate literary and popular fiction. He gives his assessment of Japanese-to-English quality.

Tom Gally, a former professional translator and University of Tokyo professor, discusses in April 2023 how GPT-4 and similar models may affect translators' careers. He draws on Japanese-English translation experience.

Short Airbus footage of its ATTOL project, in which a test aircraft performed a fully automatic, vision-based take-off at Toulouse-Blagnac. The crew ran eight take-offs over several hours.

Michael Reeves rigs up a camera and software that tracks a goldfish's movements to choose stock trades, tied to wallstreetbets sentiment. It is a comedic programming and electronics stunt.

Reducible's visual walkthrough of the Fast Fourier Transform, motivated by polynomial multiplication. It shows how the FFT's key ideas can be discovered by asking the right questions.

Veritasium tells the story of the Fast Fourier Transform, from its use in detecting nuclear tests to Cold War politics, and how it could have changed the arms race. It includes a short, intuitive explanation of what the Fourier transform does.

Robert Erdmann, a materials scientist at the Rijksmuseum, gives a PyCon keynote on the ultra-high-resolution imaging of Rembrandt's Night Watch. He covers the automated gantry camera system, the Python software that drives it, and the large volumes of image data involved.

Inigo Quilez live-codes a cartoon character painting entirely from math, using signed distance fields in a Shadertoy shader. He explains each step of building the image from formulas.

Explains the proposed NASA JPL plan to use the Sun as a gravitational lens, sending solar sail craft to its focal region to image an exoplanet's surface. It covers the distances involved, how imaging with the Sun would work and the mission design.

PBS Space Time explains the proposal to use the Sun's gravity as a giant lens, sending a train of small light-sail spacecraft to the focal point roughly 550 AU out. It covers how such a telescope could image exoplanets in surprising detail and the engineering challenges involved.