distributed training

The first lecture in Martin Kleppmann's 8-lecture distributed systems series from the University of Cambridge undergraduate course. It introduces the subject, and accompanying lecture notes are available.

A Ray Summit 2022 keynote fireside chat between OpenAI's Greg Brockman and Anyscale's Robert Nishihara. They discuss how OpenAI's training infrastructure evolved and its use of Ray.

An episode of Gwynne Dyer's 1983 documentary series War, about how military basic training turns recruits into soldiers. It covers the psychological side of conditioning.

Footage of FAA altitude chamber hypoxia training for pilots in which a participant passes out from oxygen deprivation. It shows how quickly and subtly cognition fails at altitude.

A short clip from an ACM and Computer History Museum interview in which Leslie Lamport discusses writing his paper on time, clocks and the ordering of events. He explains its link to special relativity.

A 1973 AT&T training film made for new employees at Bell Labs' Holmdel Computing Center, when Unix was in its third version and running at 16 computer centers in the Bell System. It shows how users submitted jobs and used the center's machines.

A 1953 US Navy training film on the basic mechanisms in fire control computers: shafts, gears, cams and differentials. It shows how these mechanical parts compute ship and target firing solutions.

A 1930s Chevrolet training film explaining the principles and development of the differential gear. It builds the idea step by step from first principles to the finished mechanism.

Rick Beato argues that adults cannot develop true perfect pitch, which he ties to critical-period brain plasticity and language acquisition. He says it can only be developed in young children, and discusses the research behind this.

Robert Miles explains reward modelling, in which an AI learns what humans want from their preference comparisons instead of a hand-written reward function. He walks through the 2017 paper on deep reinforcement learning from human preferences.

FortNine's tongue-in-cheek guide to why drivers fail to see motorcyclists and how riders can make themselves more visible. It touches on perception psychology, including the role of saccades.

Rick Beato discusses a theory of why some children develop perfect pitch, based on early exposure to music. He draws on his experience with his own children.

Part one of an American Airlines training video presented by Captain Warren VanderBurgh on automation dependency. It grew out of a program on wake turbulence avoidance, and this restored version has the audio cleaned up.

Chapter 11 of the documentary The Corporation, on how marketing targets children, including the Nag Factor study that taught kids to nag their parents more effectively.

A man with no flight instruction and only desktop flight simulator experience attempts to take off and land a Cessna 172, with a commercial airline instructor aboard who gives no help. The flight came out of a prop bet.

Adam Neely explains what perfect (absolute) pitch is, how it differs from relative and quasi-absolute pitch, and why he would not want it. He covers its drawbacks, including drift with age.

A short demonstration of a crow using a purpose-built vending machine, in which the bird exchanges objects for rewards. The creator sells the machine as a kit.

Practical Engineering explains why connecting solar and other inverter-based generation to the grid is hard. It covers grid stability, load balancing and the behind-the-scenes engineering challenges of renewables.

Cardiologist James O'Keefe argues in a TEDx talk that moderate running confers the best longevity benefits, while extreme endurance training can damage the heart. He draws on data and his own experience as a lifelong runner.

Destin Sandlin of Smarter Every Day undergoes hypoxia training and experiences his own impaired thinking as oxygen is withdrawn. The video shows how quickly and unnoticeably oxygen deprivation degrades judgment, illustrating why you put your own mask on first.

David Epstein argues in this TED talk that improving athletic records owe much to technology, training and the selection of specialized body types, not just humans getting inherently better. He gives counter-intuitive examples across sports.

Mario Zechner explains why he built pi, a minimal coding agent, out of frustration with existing agent harnesses. He also discusses how AI agents are straining open source projects and argues for slowing down.

Admiral William McRaven's 2014 University of Texas commencement address, built around lessons from Navy SEAL training. It famously opens with the case for making your bed every morning.

Part one of a 1996 BBC Modern Times documentary on the training to become a London black cab driver. It follows candidates learning the Knowledge and facing stressful repeated exams with Public Carriage Office examiners.

Johns Hopkins APL shows Les Baugh, a bilateral shoulder-level amputee, controlling two Modular Prosthetic Limbs by thought after a short training period. Nerves rerouted to his chest drive the arms.

A short clip of Jordan Peterson explaining Pareto distributions, using Soviet farm collectivization and the resulting famine as context. It also covers how outcomes concentrate in a few people.

Rob Miles explains "glitch" tokens, odd strings in large language model vocabularies that make the model produce gibberish. He covers how they likely arise from tokenizer training data, such as Reddit usernames, that never appeared in the model's own training.

Nina Paley's TEDxMaastricht talk arguing against copyright and for copyleft and free culture. She argues that restricting ideas harms creativity and that culture belongs to everyone.

The US Chemical Safety Board's updated animation of the 2005 BP Texas City refinery explosion, which killed 15 workers. It was released ahead of a process safety training application.

A 1951 Bell System training film showing how telephone calls are routed through a step-by-step (Strowger) switching office.

A 60 Minutes segment in which Navy pilots Dave Fravor and Alex Dietrich recount their 2004 encounter with the 'tic tac' UAP during training with the USS Nimitz group.

A clip from Joe Rogan Experience #928 in which Rogan, with Arian Foster, explains why he abandoned his belief in Moon landing conspiracy theories. He describes how doctored training photos fueled doubt before he learned more of the physics.

A 2019 BBC News report by John Sudworth, who was given rare access to the secure facilities in Xinjiang where more than a million Muslims are thought to be held. Authorities call them training schools, and the report examines what the visit reveals about the system and conditions inside.

Andrej Karpathy's Databricks talk on Software 2.0, the shift from human-written code to code produced by neural network training. It covers how developer work moves toward curating datasets, the advantages and challenges, and opportunities for new tooling.

Robert Miles covers a paper where researchers ran real versions of his earlier mesa-optimizer thought experiments. They show inner misalignment, where an AI learns the wrong goal in training and misbehaves when deployed.

Practical Engineering explains how continuous welded rail copes with thermal expansion by restraining it, using the relationship between temperature, stress and strain. The video is part of a series on railways.