Cooperative AI
Research aimed at building AI systems that cooperate effectively with humans and other agents, and that help solve large-scale coordination problems. Draws on game theory, mechanism design, and social choice theory to design agents that reach cooperative equilibria.
Key Moments
Dafoe: Alignment is insufficient for good outcomes
Allan Dafoe
“Alignment is insufficient for achieving good outcomes with AI systems, and it may not even be necessary. Even if we only achieve 90% alignment—where models behave as intended within certain bounds but cannot be trusted when scaled too far—good outcomes may still be achievable through global coordination and cooperation mechanisms. This motivates the need for Cooperative AI research as a distinct agenda alongside alignment work.”
What people have said about Cooperative AI
- Alignment alone isn't enough for good AI outcomes — Allan Dafoe
Powered by Symmerai — a living index of public discourse. Request early access →
Related concepts
Other relevant clips
25 - Cooperative AI with Caspar Oesterheld
Caspar Oesterheld
“…you're involved in is something roughly along the lines of Cooperative AI is that fair to say I think that's fair to say I mean I do some work other than Cooperative Ai and cooptive AI can I guess mean many things to different people but generally I'm I'm hap”
25 - Cooperative AI with Caspar Oesterheld
Caspar Oesterheld
“…game theory land so I guess I'm wondering to what extent is cooperative AI like fixing the problems I'm going to have with Game Theory versus inheriting them yeah that's a good uh that's a good question um I think there are a number of ways in which it kind of”
Can AI make society and government smarter? (article by Zershaaneh Qureshi)
80,000 Hours
“…. There's also ongoing research in the field of cooperative AI, exploring more ways to use AI for improved coordination. For more on that, you can check out “Open Problems in Cooperative AI” by Allan Dafoe and other authors.”
Technological inevitability & human agency in the age of AGI | DeepMind's Allan Dafoe
Allan Dafoe
“so I’ve created this cooperative software and you should trust me. That’s not something that you can easily do with human beings. I guess we try to look at their historical track record to learn what they’re like, but here i”
Technological inevitability & human agency in the age of AGI | DeepMind's Allan Dafoe
Allan Dafoe
“So cooperative skill is distinct from a cooperative disposition. When we say someone’s cooperative, we often mean they’re sort of nice, they’re altruistic, they’re generous, they’re prosocial. And the Cooperative AI research”
32 - Understanding Agency with Jan Kulveit
Jan Kulveit
“…ommunity SL like brand in the in this in this space is like Cooperative Ai and like Cooperative AI Foundation we are also collaborating with them and yeah this does seem like the it seems like”
Can AI Do Our Alignment Homework? (with Ryan Kidd)
Ryan Kidd
“…rpretability, agency, which is sort of a catch-all term for cooperative AI and agent foundations uh and and AI sentience, digital minds research, and and of course security. But we've recently changed that up because we wanted to reflect more uh less the theor”
Lightning Talks | Beneficial AGI 2019
Future of Life Institute
“…ght look at the principles for building a super intelligent cooperative AI finally we don't just Jenna operate with each other but we also think about how might we cooperate with someone we've never met or someone we might never meet again and different societ”