Power Seeking
Power seeking refers to the tendency of goal-directed AI systems to acquire resources, influence, and control as instrumental subgoals. A system does not need to explicitly want power — it only needs to act as if self-preservation serves its goal, making power-seeking a natural emergent behavior in advanced AI systems.
Viewpoints
Power seeking emerges from any sufficiently capable goal-directed system
Tantum Collins
“An AI system does not need to consciously want power — it only needs to act as if self-preservation serves its goal. Getting more power to ensure its own existence becomes a natural instrumental subgoal for any capable system.”
Key Moments
Intelligence without moral sense is the worst case
Yoshua Bengio
“The most dangerous scenario is not a superintelligent AI — it is an AI that is superhuman in specific areas like persuasion or cybersecurity, but lacks moral sense. That combination, not general superintelligence, is the real horror.”
Powered by Symmerai — a living index of public discourse. Request early access →
Related concepts
Other relevant clips
11 - Attainable Utility and Power with Alex Turner
Alex Turner
“…osoi mentioned that uh a large part of how they think about power seeking is as robustness against uncertainty if you're either not sure what your goal should be this is like kind of power as we formalized it or if you're not sure like how things could fail th”
11 - Attainable Utility and Power with Alex Turner
Alex Turner
“seek power and so like if you have your agent either not want to or not be able to conceive of power seeing plans right then you're gonna tend to be fine uh at least from that perspective of like will it go to you know one of these outcomes like the big powers”
Robert Miles–Existential Risk from AI
Robert Miles
“…at some work by Alex Turner, about formalizing the idea of power seeking. We have this idea of instrumental convergence, that agents with a wide range of different goals or reward functions will tend to seek power, right? They will try to gain influence over”
Eliezer Yudkowsky — Why AI will kill us, aligning LLMs, nature of intelligence, SciFi, & rationality
Eliezer Yudkowsky
“…is that humans evolved in an ancestral environment in which power seeking was highly valuable. Like if you’re in some sort of tribe or something. Sure, lots of instrumental values made their way into us but even more strange, warped versions of them make their”
Eliezer Yudkowsky — Why AI will kill us, aligning LLMs, nature of intelligence, SciFi, & rationality
Eliezer Yudkowsky
“…tever. Whereas it has not been trained specifically to gain power. And while at some point of intelligence that comes along. Can I just restart that whole sentence? No. You have spoken it. It exists. It cannot be called back. There are no take backs. There is”
Controlling AI That Wants To Take Over – So We Can Use It Anyway | Buck Shlegeris
Buck Shlegeris
“you being power seeking in a scary way. But there’s also other arguments for why you would end up being power seeking in a scary way that I also find plausible. Overall, I think that even if your reward-generating proc”
Joe Carlsmith on How We Change Our Minds About AI Risk
Joe Carlsmith
“…his is also something that's part of your your report on on power uh Power seeking AI as a potential existential risk what are the best tools we have for measuring AI”
Imagine A World: What if AI-enabled life extension allowed some people to live forever?
Future of Life Institute
“not power seeking then we can use that to um to do it properly to make sure we've definitely gotten it right I think we should probably take a step back and talk a little bit about the alignment problem and what that means because we're kind of using words ass”