Neural networks are grown, not built

From the conversation

Fundamentally, we're just going to use whatever tools we have at the time and see how well they work. Ideally, we have this enumerative safety case where we can almost verify or prove that the model will behave in particular ways. In the worst case, we use the current tools like when we won the auditing game of seeing what features are active when the assistant tag lights off. Can you back up? Can you explain, what is mechanistic interpretability? What are features? What are circuits? Totally. Mechanistic interpretability—or the cool kids call it mech interp—is trying to reverse engineer neural networks and figure out what the core units of computation are. Lots of people think that because we made neural networks, because they're artificial intelligence, we have a perfect understanding of how they work. It couldn't be further from the truth. Neural networks, AI models that you use today, are grown, not built. So, we then need to do a lot of work after they're trained to figure out to the best of our abilities how they're actually going about their reasoning. And so, three and a half years ago, this kind of agenda of applying mechanistic interpretability to large language models started with Chris Olah leaving OpenAI, co-founding Anthropic. And every roughly six months since then, we've had a major breakthrough in our understanding of these models.…