The concrete crux with Christiano: 16% vs 8% on AI winning IMO gold
Eliezer YudkowskyEliezer Yudkowsky — Why AI will kill us, aligning LLMs, nature of intelligence, SciFi, & rationalityat 1:46:00
From the conversation
What part of this is different for credibility than dollars? Presumably you would have different predictions before the world ends. It would be weird if the model that this world ends and the model that says the world doesn’t end have the same predictions up until the world ends. Yeah. Paul Christiano and I cooperatively fought it out really hard at trying to find a place where we both had predictions about the same thing that concretely differed and what we ended up with was Paul’s 8% versus my 16% for an AI getting gold on International Mathematics Olympics problem set by, I believe, 2025. And prediction markets odds on that are currently running around 30%. So probably Paul’s going to win, but slight moral victory. I guess people like Paul have had the perspective that you’re going to see these sorts of gradual improvements in the capabilities of these models from like GPT-2 to GPT-3.
Summary
The most concrete prediction Yudkowsky and Christiano could agree to disagree on was whether AI would achieve gold on the International Mathematics Olympiad by 2025 — Christiano at 8%, Yudkowsky at 16%. Prediction markets ran at 30%, suggesting Christiano would win.
Watch the clip on YouTubeStarts at 1:46:00