image processing

A six-minute clip of Adobe's MAX 2011 sneak peek showing a Photoshop prototype that removes motion blur from photos. The audience reacts with astonishment as blurry images become legible.

A short explainer from Nat and Friends on where Google Earth's 3D imagery comes from, how it is stitched together, and how often it is updated. It describes aircraft carrying camera rigs and photogrammetry that turns the photos into 3D models.

A short conference talk from !!Con 2017 in which Kevin Chen explains the HDR imaging algorithm from a foundational paper, framing it as a system of linear equations, and implements it in a Microsoft Excel spreadsheet. No prior image processing knowledge is needed.

A SIGGRAPH Asia 2013 research demo of 3-Sweep, a tool for extracting editable 3D objects from a single photo using a few user strokes. It predates modern AI image tools.

The Thought Emporium shows how to receive clear images directly from GOES-15, 16 and 17 and Himawari 8 geostationary weather satellites with software-defined radio. It covers the extra effort needed to receive GOES data.

Adobe's sneak-peek demo of Content-Aware Fill ahead of Photoshop CS5, showing objects being removed and areas repaired seamlessly with the delete key.

MinutePhysics explains that computers store brightness nonlinearly, so blurring and blending colors in raw RGB gives wrong, dark results. It shows why mixed colors, such as in gradients, look off.

Mike Pound explains the discrete cosine transform and how it underpins JPEG compression. It is part two of Computerphile's JPEG series.

MIT researchers show that sound can be recovered from silent video by analyzing tiny vibrations in objects such as a potato chip bag. It works even with consumer cameras by using rolling shutter.

A short demo video for NVIDIA's StyleGAN paper, which proposes a style-based generator for GANs. It shows face synthesis with style mixing and scale-specific control over attributes like pose and freckles.

A short Techquickie explainer on why laptop webcams are much worse than phone cameras. It covers sensor size, image processing, and the interface differences between laptops and phones, with input from a Logitech engineer.

A spectrogram analysis of Aphex Twin's track commonly called [Equation] from Windowlicker, rendered from a lossless version. At about 5:30 the spectrogram reveals an image of Richard D. James's face.

The SIGGRAPH 2005 demo of dual photography from Stanford, which swaps the roles of projector and camera to render a scene from the light's point of view. It includes the reveal of a hidden playing card.

Tom Scott visits the USPS Remote Encoding Center in Salt Lake City, where keyers decipher mail images machines cannot read. It shows the lengths taken to decode bad handwriting.

Berkeley neuroscientist Jack Gallant shows how fMRI brain mapping can be used to decode what a person sees, with visualizations of brain activity and its implications.

Technology Connections explains why the superheterodyne radio has its name and how mixing frequencies lets it tune stations well. It walks through the underlying wave phenomena and algebra and why the design solved a hard tuning problem.

An artist describes discovering she has aphantasia, the inability to form mental images, and how it affects her work. She asks viewers about their own visualization.

Neuroscientist Anil Seth argues in this TED talk that perception is a controlled hallucination: the brain constantly predicts the world and our shared agreement is what we call reality. He also touches on the construction of the self.

Acerola explores building a shader that turns games into ASCII art, walking through techniques such as brightness mapping and edge detection to see whether it looks good.

Huygens Optics explains the Fourier transform and shows how a lens performs one optically. A DLP projector is used as a Fourier filter to do simple pattern recognition on lithographic character patterns.

Luis von Ahn, creator of CAPTCHA and reCAPTCHA, explains how small contributions from many people online can digitize books. He then introduces Duolingo as a way to translate the web while people learn languages.

Mark Rober and Jabril build an app that uses machine learning to decode baseball sign sequences. The video also explains neural networks in a simple, accessible way.

A Computerphile video in which Dr Mike Pound and PhD student Lewis Stuart explain 3D Gaussian splatting, a quick technique that turns photos of a scene into a 3D model without much compute. It includes a demo and compares the approach to NeRFs.

A ShortCircuit (Linus Tech Tips) review of Nvidia's RTX Voice, which uses GPU-based neural networks to remove background noise from microphone audio. It demonstrates how well the filtering works.

Part 2 of the IROS 2011 keynote by Sebastian Thrun and Chris Urmson on Google's self-driving car. It shows how the car perceives its surroundings with a lidar point cloud and recognizes traffic lights and pedestrians. The audio is poor because it was recorded on a pocket camcorder.

Adobe's VoCo demo from the MAX 2016 Sneak Peeks, co-hosted by Jordan Peele, showing software that changes words in a voiceover by typing new ones.

Rob Miles explains "glitch" tokens, odd strings in large language model vocabularies that make the model produce gibberish. He covers how they likely arise from tokenizer training data, such as Reddit usernames, that never appeared in the model's own training.

Luis von Ahn, creator of CAPTCHA and reCAPTCHA, presents Duolingo at TEDxCMU 2011. The idea is to have millions of people translate the web for free while learning a language, an example of human computation.

Footage shot in New York City in 1911 by a Swedish film company, upscaled to 4K and 60 fps with neural networks and optionally colorized. The creator notes the colors are artificial and added only for ambiance.

A demo of PREDATOR (TLD), a real-time object tracker that learns the appearance of its target and improves while it tracks. The technology was later developed commercially.

VICE News reporter Elle Reeve investigates China's rollout of facial recognition and mass surveillance technology and what it means for citizens. The report originally aired on VICE on HBO.

Microsoft Research's Rick Rashid demonstrates a speech recognition and translation system, built on deep neural networks, that turns his spoken English into Chinese speech in 2012.

Ashley Feniello demonstrates VimSpeak, a voice-controlled interface for Vim inspired by Tavis Rudd's voice programming talk. The demo includes working through a VimGolf challenge by speech.

Ramesh Raskar presents femto-photography, a technique that captures a trillion frames per second, fast enough to show light itself moving through a scene. He discusses possible uses such as cameras that see around corners or inside the body.

Denis Shiryaev's neural-network-upscaled, colorized 60 fps version of film footage shot in Paris between 1896 and 1900.

Steve Mould explains motion amplification and video magnification, techniques that find tiny changes in video invisible to the eye and exaggerate them. He covers the MIT research and industrial uses such as diagnosing vibrating machinery.

FortNine looks at why Tesla Autopilot has rear-ended motorcyclists, speculating that the vision system misreads motorcycle taillights as a distant car. It argues that radar and other sensors would help.

Tom Gally, a former professional translator and University of Tokyo professor, discusses in March 2023 whether GPT-4 and similar models could translate literary and popular fiction. He gives his assessment of Japanese-to-English quality.

A short clip of the Google I/O 2018 demo in which Google Assistant's Duplex phones a salon and a restaurant to make appointments. The calls sound like a natural human conversation.

A 2007 video of someone trying to write Perl code using Windows Vista's speech recognition, with hilariously bad results.