The black box has seams: what Anthropic's circuit tracing mapped

We built machines nobody fully understands, and in 2025 researchers started mapping them. In March 2025, Anthropic published two paired studies, a methods paper called "Circuit Tracing: Revealing Computational Graphs in Language Models" and a findings paper called "On the Biology of a Large Language Model". The framing is deliberate. A language model grows through training rather than being engineered line by line, so understanding one looks less like reading code and more like neuroscience: probe, map, intervene. The papers trace parts of the internal computations behind specific behaviours in Claude 3.5 Haiku, a production model.

The core problem the method solves is that a model's raw neurons carry many meanings at once. The team trained replacement components that approximate parts of the model using sparse, more interpretable features, and from these built attribution graphs: maps of how information flows from the words you type, through intermediate concepts, to the word the model outputs. The graphs support interventions, suppressing or amplifying a feature and watching the behaviour change, which moves the evidence beyond correlation toward causal testing.

Three of the traced behaviours stand out. The model can plan ahead: writing rhyming poetry, it selects candidate rhyme words first, then composes the line to arrive at them, planning visible in the circuitry. Concepts cross languages: shared features activate for the same idea across English, French and Chinese, and some representations appear to capture meaning before language. And reasoning steps can be traced: for "the capital of the state containing Dallas", the model passed through an internal Texas representation on its way to Austin.

Why this matters connects to the faithfulness research covered earlier in this series, which showed that a model's own explanation is not necessarily a faithful account of how it produced an answer. Circuit tracing offers a different kind of evidence: inspection of internal mechanisms rather than self-description. In May 2025 the tools were open-sourced, so external researchers now generate these graphs on open models, and mechanistic auditing is beginning to move from aspiration toward a practical research discipline. The honest limits belong next to that: attribution graphs currently explain a fraction of behaviours, on short prompts, at high research cost. This is early anatomy, a first sketch of the organism rather than a full map.

The professional translation is about a phrase. "Explainable AI" appears in nearly every vendor deck and every draft regulation, and it can mean very different things. A model-generated rationale explains an answer in words. Mechanistic interpretability looks for evidence inside the system itself. Circuit tracing gives that distinction substance, though it is not yet a mature audit standard, and as the tools improve, the difference may start to matter in procurement, assurance and regulation.

Two questions follow. What kind of explanation is this: a rationale generated by the model, evidence about which inputs mattered, or inspection of internal mechanisms? These are different forms of evidence. And how much evidence does the risk require: for low-risk work, output testing may be enough, while for consequential decisions, do not rely only on the model's own explanation.

Attribution graphs have turned "nobody knows how these models work" from a fact into a research programme. Planning, shared concepts and intermediate reasoning steps are now partially visible in the circuitry. The black box is not opening all at once, but it is no longer completely sealed. Know the difference between a generated explanation and evidence of computation.