rlhf

Robert Miles explains reward modelling, in which an AI learns what humans want from their preference comparisons instead of a hand-written reward function. He walks through the 2017 paper on deep reinforcement learning from human preferences.

UC Berkeley EECS colloquium by OpenAI's John Schulman on reinforcement learning from human feedback. He covers why RL helps with truthfulness, hallucination, and the limits of behavior cloning.

Andrej Karpathy walks through how GPT assistants are trained, from tokenization and pretraining to supervised finetuning and RLHF, and gives practical prompting strategies and a view of the tooling ecosystem.

A podcast episode in which GPT-4 red teamer Nathan Labenz describes using the model before release, finding it human-level but not human-like, and discussing its weaknesses and worrying capabilities. It closes with speculation about GPT-5.

Computerphile interviews AI safety researcher Rob Miles about ChatGPT: how it was trained with human feedback, why it tends to produce confident nonsense, and where it may or may not be dangerous.