transformers

Rob Miles explains on Computerphile how language models generate plausible text and how transformers improved on recurrent networks, with a look at what comes next.

Leo Dirac explains how transformer models have replaced LSTMs in NLP. He gives background from bag of words through RNNs and LSTMs, then a technical deep dive into multi-headed self-attention and positional encoding, with sample code.

A university lecture from CS480/680 by Pascal Poupart on attention mechanisms and transformer networks, worked through on a chalkboard.

Yannic Kilcher walks through the Attention Is All You Need paper, which introduced the Transformer, a sequence model based solely on attention without recurrence or convolutions. He reviews the architecture and the paper's translation results.

Adobe's VoCo demo from the MAX 2016 Sneak Peeks, co-hosted by Jordan Peele, showing software that changes words in a voiceover by typing new ones.

A deepfake in which Bill Hader, impersonating Arnold Schwarzenegger on a talk show, visibly morphs into Schwarzenegger. It was made with DeepFaceLab by Ctrl Shift Face.

Sunspring is a sci-fi short film starring Thomas Middleditch. Its screenplay was written by an LSTM neural network trained on sci-fi scripts, and the team shot and edited it in 48 hours.

A BBC News segment on University of Washington research that synthesized a photorealistic Obama video, modeling mouth movement so any audio can be put in his mouth.

A short clip of the Google I/O 2018 demo in which Google Assistant's Duplex phones a salon and a restaurant to make appointments. The calls sound like a natural human conversation.

Sony CSL's Daddy's Car is a pop song composed with AI in the style of the Beatles, with lead sheets generated by software from Beatles-style training material. Humans produced, arranged and performed it.

A BuzzFeed PSA from 2018 in which a deepfaked Barack Obama, voiced by Jordan Peele, warns about fake video before revealing the trick. It is a one-minute demonstration of AI face and voice manipulation.

A short demo video for NVIDIA's StyleGAN paper, which proposes a style-based generator for GANs. It shows face synthesis with style mixing and scale-specific control over attributes like pose and freckles.

The Google I/O 2018 keynote led by Sundar Pichai, covering Google's product and platform announcements. Highlights included AI features such as smart replies and the Assistant.

The CVPR 2016 oral demo of Face2Face, a system that captures a person's face and reenacts another video's face in real time using only RGB video. The authors frame it as research on modern vision and graphics.