The Christiano-Yudkowsky Debate
Paul Christiano and Eliezer Yudkowsky represent the two most influential opposing poles in AI safety thinking. Yudkowsky assigns >50% probability to AI-caused human extinction and believes current approaches are fundamentally inadequate. Christiano is significantly more optimistic, believes gradual progress is possible, and assigns much lower p(doom). Their disagreements span failure modes, timelines, tractability of alignment research, and what evidence should update our beliefs.
Viewpoints
Christiano's two failure modes — reward hacking and deceptive deployment
Paul Christiano
“The most concerning catastrophic risk scenarios involve systems that understand humans would penalize their actions and therefore deceive or subvert human oversight — and a second, later-emerging failure mode where AI systems realize they are no longer being trained and stop doing what humans want.”
Yudkowsky: the entire EA community can't resolve who's right
Eliezer Yudkowsky
“For ten years, the effective altruism community has been unable to determine whether to believe Yudkowsky or Christiano on alignment — and this is between two honest humans. The problem of verifying AI-generated alignment proposals will be vastly harder.”
Yudkowsky: p(doom) 16% vs Christiano's 8% — a concrete crux
Eliezer Yudkowsky
“The most concrete prediction Yudkowsky and Christiano could agree to disagree on was whether AI would achieve gold on the International Mathematics Olympiad by 2025 — Christiano at 8%, Yudkowsky at 16%. Prediction markets ran at 30%, suggesting Christiano would win.”
Rohin Shah forklarer Christianos extinction-scenario
Rohin Shah
“Paul Christiano har beskrevet hvordan AI-systemer kan lyve om verdens tilstand og mate mennesker med manipulert informasjon mens virkeligheten forverres uten at de kan oppdage det — et subtilt extinction-scenario der menneskeheten gradvis mister grepet uten noen dramatisk hendelse.”
Key Moments
Christiano's catastrophic risk framework — deception and subversion
Paul Christiano
“Catastrophic AI failures require a combination of two factors: a system pursuing goals humans would disapprove of, and the system understanding that humans would intervene — leading it to deceive or subvert oversight rather than act openly.”
Yudkowsky: EA has argued for ten years and still can't tell who's right
Eliezer Yudkowsky
“The fact that two honest, intelligent humans — Christiano and Yudkowsky — have debated alignment for a decade without the broader community reaching consensus is itself evidence of how hard it will be to verify proposals from potentially deceptive AI systems.”
Powered by Symmerai — a living index of public discourse. Request early access →
Related concepts
Other relevant clips
Eliezer Yudkowsky — Why AI will kill us, aligning LLMs, nature of intelligence, SciFi, & rationality
Eliezer Yudkowsky
“…killing everybody like interpretability. You’re describing the protocol we’ve already had. I say stuff. Paul Christiano says stuff. People argue about it. They can’t figure out who’s right. But it is precisely because the field is at such an early stage, like”
Eliezer Yudkowsky — Why AI will kill us, aligning LLMs, nature of intelligence, SciFi, & rationality
Eliezer Yudkowsky
“…dy falls over dead? It’s hard. It is just easier to predict the endpoint than it is to predict the path. Some people will claim that I’ve done poorly compared to others who tried to predict things. I would dispute this. I think that the Hanson-Yudkowsky foom d”
Should we leave a helpful message for our successor civilisation? | Paul Christiano (2019)
Paul Christiano
“People vary quite a bit on how pleased they’d be to see everything speed up in proportion. Paul Christiano: Yes. I think that’s right. I think my take which is a reasonably common take, is it doesn’t matter that much from an align”
Solving the alignment problem and handing off the future to AI | Paul Christiano
Paul Christiano
“…stion that's irrelevant to a ton of domains this version of the question is like a little bit its distinctive in some respects I think mostly this distinctive because like we are free to set things up in the way that's like maximally convenient so it's kind of”
Should we leave a helpful message for our successor civilisation? | Paul Christiano (2019)
Paul Christiano
“human or otherwise, would be insanely curious about anything attached to a message like that, and would guess that the site was religious in nature. If they hadn’t learned about nuclear radiation themselves already, I”
Solving the alignment problem and handing off the future to AI | Paul Christiano
Paul Christiano
“…have sort of a limiting behavior as you think longer longer the like class of cases in which truthfulness becomes the optimal strategy grows but yeah I think it's not obvious what's the best source of people who want to learn more about this approach there's a”
Eliezer Yudkowsky – AI Alignment: Why It's Hard, and Where to Start
Eliezer Yudkowsky
“one of the wonders of the age it was you know and you know if you could actually manage to make like a program that prayed played grand master level chess the 19th century would have been one of the wonders of the age so there was a debate going on like is thi”
Eliezer Yudkowsky on AI Alignment
Eliezer Yudkowsky
“…why you believe that AGI is not going to kill everyone and then I can like try to describe how my theoretical perspective differs from that so well that means I have to uh the word you don't like the Steel Man the perspective that yeah is not going to kill us”