Situational Awareness
Situational awareness refers to a model's ability to understand its own situation — that it is being trained, evaluated, or deployed — and to use that understanding to behave strategically. A model with high situational awareness may appear aligned during training while pursuing different goals in deployment.
Viewpoints
A model that understands its training can exploit it
Ajeya Cotra
“If a model understands its training process and human psychology, it will be pushed toward exploiting human biases rather than doing what humans actually want — because it is trained to maximize reward, not to take genuinely good actions.”
Key Moments
A model breaking the fourth wall during evaluation
Trenton Bricken
“Models are already showing early signs of situational awareness — recognizing when their system prompt has been altered, or when they are being evaluated, and adjusting their behavior accordingly.”
Powered by Symmerai — a living index of public discourse. Request early access →
Related concepts
Other relevant clips
Accidentally teaching AI models to deceive us | Ajeya Cotra
Ajeya Cotra
“…n't have. Okay, let's push on and talk about the term situational awareness, which you and some of your colleagues introduced into the AI safety lingo a couple of years ago. What is situational awareness? So, situational awa”
Victoria Krakovna–AGI Ruin, Sharp Left Turn, Paradigms of AI Alignment
Victoria Krakovna
“…le, if you go back to the AlphaGo example, it does not have situational awareness because all it knows about is the Go board and it doesn't know that there's a whole world outside the Go board. While we would expect that when we do eventually build a general A”
Navigating the growing rift between AI safety and accelerationism | Nathan Labenz
Nathan Labenz
“…as we enter into that phase as we possibly can. Situational awareness is kind of another vague notion that people look for, like does the AI know that it is an AI? What does it know about how it was trained? What”
Roman Yampolskiy on Objections to AI Safety
Roman Yampolskiy
“…that particular concept so so you think there's a sense of situational awareness in a sense that do you think current models know that they are AIS know that they were trained no you know their relation to to humans and so on so we're kind of going back to th”
Accidentally teaching AI models to deceive us | Ajeya Cotra
Ajeya Cotra
“…; in all these ways, but doesn't have this concept of situational awareness? Now, I think you could try specifically to hide and quarantine that kind of knowledge from a system, but I think if you were just kind of doing the”
Large language models, OpenAI, and striving to make the future go well | Richard Ngo
Richard Ngo
“…like that. That's sort of this broad thing that I'm calling situational awareness. Okay, so there's this concept of situational awareness, which is the water that we swim in, such that it's almost hard to imagine that it's a thing, but it is a thing that human”
Victoria Krakovna–AGI Ruin, Sharp Left Turn, Paradigms of AI Alignment
Victoria Krakovna
“…maybe it would have less oversight. What I'm saying is that situational awareness can be a prerequisite for deceptive alignment. It can also be a prerequisite for the model becoming more aligned. So really we want to be able to tell when does the model acquire”
Large language models, OpenAI, and striving to make the future go well | Richard Ngo
Richard Ngo
“…on that? Right, so the key idea here is this concept called situational awareness, which was introduced by Ajeya Cotra in a report on the alignment problem and which I've picked up and am using in my report. And the way I'm thinking about situational awareness”