transformers

Rob Miles explains on Computerphile how language models generate plausible text and how transformers improved on recurrent networks, with a look at what comes next.

Yannic Kilcher walks through the Attention Is All You Need paper, which introduced the Transformer, a sequence model based solely on attention without recurrence or convolutions. He reviews the architecture and the paper's translation results.