research

Andrej Karpathy builds a GPT from scratch in code, following Attention Is All You Need and GPT-2/3, in about two hours using PyTorch. He connects it to ChatGPT and assumes familiarity with his earlier makemore videos.

A short video of stretching and hand exercises for preventing and easing tendinitis and carpal tunnel syndrome, from a percussion teacher. The exercises were shown to him by a doctor.

A short promotional video demonstrating a head-thumping technique that is claimed to relieve tinnitus. Covering the ears and tapping the back of the head is said to silence the ringing temporarily.

A University of Michigan Medicine demonstration of building a cheap DIY air purifier from a box fan and a HEPA or high-MERV filter using hardware-store parts. It explains why this helps against dust, smoke and viruses and ties it to local air quality readings.

The first of a short series by Welch Labs on building a complete neural network in Python. It introduces the data and the architecture of a simple network before later parts cover forward propagation, gradient descent, and backpropagation.

Andrej Karpathy builds a bigram character-level language model from scratch, introducing torch.Tensor and the framework of training, sampling and loss evaluation. It is the first step toward a GPT-style Transformer in his makemore series.

A long Tsoding Daily programming stream in which he implements hot code reloading in C by loading a dynamic library at runtime without stopping the main program. It also touches on migrating struct versions across reloads.

NightHawkInLight shows how to make ultra-white radiative paint that cools surfaces below ambient temperature without electricity. He walks through the research papers and the household-materials process.