alignment

Robert Miles covers a paper where researchers ran real versions of his earlier mesa-optimizer thought experiments. They show inner misalignment, where an AI learns the wrong goal in training and misbehaves when deployed.

Robert Miles explains the orthogonality thesis: how an agent can be highly intelligent while pursuing goals that seem stupid or alien to humans. He separates intelligence from goals and touches on the stamp-collector example.

A Computerphile video in which Rob Miles warns against assuming a general AI will think like a human. He uses the thought experiment of a stamp-collecting AI to show how a goal-driven system could pursue its aim with dangerous results.

Robert Miles introduces the first of several videos on the paper 'Concrete Problems in AI Safety', focusing on how AI systems can cause accidental negative side effects. It explains the problem in accessible terms.

Rob Miles explains for Computerphile why simple rule sets like Asimov's Three Laws cannot make robots and AI safe. Terms like harm and human are too ambiguous to specify precisely.

Robert Miles goes through ten common reasons people give for dismissing AI safety, adapted from Stuart Russell's list, and answers each one. It is aimed at skeptics and newcomers.

A speculative-fiction video by Tom Scott, framed as coming from a future, about an AI called Earworm that enforces copyright so aggressively that humanity ends up forgetting a century of pop culture. It is animated and narrated as part of his series of videos from a future.

CGP Grey's video "Humans Need Not Apply" argues that automation will make a large part of the workforce economically obsolete, much as engines did for horses. It surveys robots and software replacing both physical and mental labor.

Robert Miles explains instrumental convergence: why an AGI with almost any goal would tend to pursue sub-goals like self-preservation and resource acquisition. It builds on Omohundro's basic AI drives and his orthogonality thesis video.

Robert Miles asks whether corporations can be thought of as artificial general superintelligences and how far the analogy holds. He argues the differences, such as being limited by employees' capabilities, matter for AI safety.

Robert Miles examines whether the case for AI safety is a Pascal's mugging, a tiny probability of an enormous payoff. He argues it is not, because AI risk does not depend on vanishingly small probabilities.

Robert Miles explains reward modelling, in which an AI learns what humans want from their preference comparisons instead of a hand-written reward function. He walks through the 2017 paper on deep reinforcement learning from human preferences.

Rob Miles explains on Computerphile why implementing an off switch for a general AI is hard. A capable agent would resist being stopped because shutdown prevents it from achieving its goal.

A Kurzgesagt animation on longtermism, arguing that because the future could hold trillions of people, our actions' effect on future generations may be what matters most.