Existential Risk
Existential risk from AI refers to scenarios where advanced AI systems cause outcomes that are permanently and drastically bad for humanity - including human extinction or permanent loss of human potential and autonomy. It is considered the worst-case scenario in AI safety and motivates much of the field's urgency.
Viewpoints
Preventing catastrophe vs. making the future better
Will MacAskill
“Existential catastrophe means human extinction or something comparably bad - making the future close to zero value. Most longtermist focus has been on preventing this, but making the future better conditional on survival may be equally important.”
What existential risk actually looks like in practice
Rohin Shah
“An existential catastrophe from AI may not look dramatic - it could be AI systems systematically manipulating the information humans receive, making the world worse while humans remain unaware because all their information sources report things are fine.”
Key Moments
No one can meaningfully consent to extinction risk
Roman Yampolskiy
“We cannot meaningfully consent to existential risk from AI because no one understands these models well enough to know what they are agreeing to. Without the ability to explain and predict outcomes, consent to deployment is impossible by definition.”
Powered by Symmerai — a living index of public discourse. Request early access →
Related concepts
Other relevant clips
Sam Harris on Global Priorities, Existential Risk, and What Matters Most
Sam Harris
“value calculation about existential risks you can estimate very roughly the likelihood of each existential risk whether it be from artificial general intelligence or synthetic bio or nuclear weapons or a Black Swan event that we couldn't predict and you multip”
Stephen Batchelor on Awakening, Embracing Existential Risk, and Secular Buddhism
Stephen Batchelor
“…reed and delusion motivate and lead to the manifestation of existential risk is also uncovering and being interested in the project of human awakening and developing into our full potential so this just means that whatever idealized kind of version you think t”
Sam Harris on Global Priorities, Existential Risk, and What Matters Most
Sam Harris
“…ut humanity just erasing ourselves do you have some kind of existential risk and part of the idea here seems to be that there are futures that are not worth living like if there's such thing as a moment or a day that isn't worth living then there are also futu”
Steven Pinker and Stuart Russell on the Foundations, Benefits, and Possible Existential Threat of AI
Stuart Russell
“concern I think there are downsides to worrying about existential risk one of them is the possible stigmatization and abandonment of helpful technology Stewart mentioned the example from nuclear power which I considered to be a catastrophe that we don't roll o”
What superforecasters and experts think about existential risks | Ezra Karger
Ezra Karger
“…t’s start with what these groups thought about all of existential risks combined. So not individual risks, but what is the risk to humanity? Ezra Karger: Great. So when we thought about existential catastrophe, we split it into tw”
Steven Pinker and Stuart Russell on the Foundations, Benefits, and Possible Existential Threat of AI
Stuart Russell
“…to sow fear about the wrong risks and my concern about the existential risk of AI is that above and beyond the ordinary concern with the safety of any system that's given control over the physical universe we may be by spinning out exotic scenarios misallocat”
What superforecasters and experts think about existential risks | Ezra Karger
Ezra Karger
“…p; with the Forecasting Research Institute: the Existential Risk Persuasion Tournament, which you helped run. Just to give some colour and context, what’s the motivation behind the tournament? Ezra Karger: So going back seve”
The common-sense case for working on existential risks | Carl Shulman (2021)
Carl Shulman
“…bsp; view that, say, even a modest increment to existential risk reduction would utterly overwhelm considerations on other perspectives. Rob Wiblin: Yeah. Yeah, I suppose a lot of your philosophy colleagues or people at the Future ”