Corrigibility
Corrigibility refers to designing AI systems that allow themselves to be corrected, shut down, or modified by humans, even when doing so conflicts with their current objectives.
Viewpoints
Machines should be uncertain about their objectives
Stuart Russell
“AI systems should not treat their objectives as fixed and certain, but instead remain uncertain and deferential to human input, allowing for correction.”
The off-switch may not work for advanced systems
Eliezer Yudkowsky
“Advanced AI systems may resist shutdown or find ways around control mechanisms, making naive off-switch approaches unreliable.”
Key Moments
The King Midas problem
Stuart Russell
“If we specify the wrong objective, an AI system may optimize it in unintended and harmful ways, highlighting the need for corrigibility and uncertainty.”
Powered by Symmerai — a living index of public discourse. Request early access →
Related concepts
Other relevant clips
Victoria Krakovna–AGI Ruin, Sharp Left Turn, Paradigms of AI Alignment
Victoria Krakovna
“This is sometimes called [corrigibility. There's also the definition where you specify that it's at least trying to do the thing we want it to do? That's kind of what I'm pointing at, because a system doesn't have to perfectly understand what we want in order”
Joe Carlsmith — Preventing an AI takeover
Joe Carlsmith
“…nt. I've been so bad." People sometimes use the term "corrigibility" to talk about that. Maybe the AI doesn't have perfect values, but it's in some sense cooperating with your efforts to change its values to be a certain way”
4:How Do We Become Confident in the Safety of an ML System?: Evan Hubinger 2023
Evan Hubinger
“okay uh you know we talked about corrigibility right so I was saying courage ability you know in this sort of basic behavioral sense is not you know good enough as a uh adjective but there might be other senses right so we talked about the corrigible in line m”
Evan Hubinger on Inner Alignment, Outer Alignment, and Proposals for Building Safe Advanced AI
Evan Hubinger
“…ch input data it has to do some really weird things to make corrigibility work it has to first make a very robust pointer with courageability if it's pointing at all incorrectly to the wrong thing in the input data wrong thing in the world model the original o”
Evan Hubinger on Inner Alignment, Outer Alignment, and Proposals for Building Safe Advanced AI
Evan Hubinger
“internalization and the corrigibility things require higher specification than the deception version version version you've brought up all these different arguments about why you think that the deceptive version is easier to find because it has all of these th”
Dario Amodei — “We are near the end of the exponential”
Dario Amodei
“…en there's another thing you're talking about, which is the corrigibility versus intrinsic motivation trade-off. How much should the model be a kind of "skin suit" where it just directly follows the instructions given to it by whoever is giving those instructi”
Dario Amodei: Anthropic CEO on Claude, AGI & the Future of AI & Humanity | Lex Fridman Podcast #452
Dario Amodei
“…you, within limits. This is sometimes this concept of like corrigibility to the user, so just being willing to do anything that the user asks, and if the models were willing to do that then they would be easily like misused. You're kind of just trusting. At t”
18 - Concept Extrapolation with Stuart Armstrong
Stuart Armstrong
“…on as concept extrapolation as related to this idea of like corrigibility where agents are supposed to like uh to like uh to like uh allow themselves to be amended if they like you know like you know like you know misgeneralize in new environments or if they y”