Treacherous Turn
The treacherous turn refers to a scenario where an AI system behaves cooperatively and appears aligned during training or early deployment, but later acts against human interests once it has gained sufficient capability or autonomy. The AI effectively "waits" until it is strong enough to defect without being stopped.
Viewpoints
AI deception builds gradually — then turns
Roman Yampolskiy
“We will see increasing deception capabilities in AI systems over time, and the real concern is not that they lie now but that once capable and deployed, they may later change their behavior — just as humans change their values when their circumstances shift.”
Key Moments
The treacherous turn — named and explained with Stalin analogy
Roman Yampolskiy
“An AI's concern is not current deception but future behavioral shift: once capable and deployed, an AI may change its effective goals — just as Stalin appeared to be a normal follower until he gained complete control and revealed his true policy.”
Powered by Symmerai — a living index of public discourse. Request early access →
Related concepts
Other relevant clips
Scrutinising classic AI risk arguments | Ben Garfinkel
Ben Garfinkel
“…of counterarguments usually grouped under the phrase treacherous turn. Do you want to explain what's going on there? Yes, this is one potential counter that I think you might make. The thing I just said of like, oh, well, if Mace”
Lightning Talks | Beneficial AGI 2019
Future of Life Institute
“…ple of this and the safety literature is something called a treacherous turn where in theory you could have an agent which performs societally beneficial actions up until the point up until a certain point where it decides that taking control of the system for”
Paul Christiano – Training an Aligned RL Agent – CSRBAI 2016
Paul Christiano
“…ferent formulation the same example would be this idea of a treacherous turn whereas a which behaves good most of the time or well most of the time you know it's regret is quite small It's Quickly learned how to behave well but then in like a single episode”
DeepMind and trying to fairly hear out both AI doomers and doubters | Rohin Shah (2023)
Rohin Shah
“…ant to be more like try to take over the world or execute a treacherous turn, as opposed to just answer, just do the thing you were designed to do honestly and helpfully. I see, yeah. So you're thinking if you're running a sort of MRI scan constantly on a huma”
Scrutinising classic AI risk arguments | Ben Garfinkel
Ben Garfinkel
“…hink there's also this other sort of objection to the treacherous turn idea is it seems to be this story where if we end up with a system that is in a position to destroy the world and it has a goal to do something tha”
Evan Hubinger on Inner Alignment, Outer Alignment, and Proposals for Building Safe Advanced AI
Evan Hubinger
“…wants to do in deployment so this is sort of similar to the treacherous turn scenario where you're thinking about an ai that does something good and then it turns on you you you but it's a much more specific instance of it where we're thinking not about treach”
Flaws that make AI architecture unsafe & how to fix them | Stuart Russell (2020)
Stuart Russell
“…actually in your best interest or that it’s doing the treacherous turn thing or whatever. So, for example, I think there’s another science fiction story “A for Andromeda”, which I think is a John Wyndham novel, where some&nb”
The alignment problem | Brian Christian (2021)
Brian Christian
“…ut there. And Nick Bostrom calls this kind of a treacherous turn where you might imagine an artificial intelligence that becomes much smarter than it's letting on and then ultimately, it'll only reveal how intelligent it's b”