Neural Networks Are Already Clean: Rethinking Mechanistic Interpretability
New evidence suggests neural networks are inherently legible, debunking the superposition theory of polysemanticity.
1Challenging the Superposition Hypothesis
The field previously assumed that polysemanticity in neural networks was caused by superposition, where models fold unrelated concepts into single neurons due to space constraints. Recent analysis of models up to trillions of parameters suggests this was a misinterpretation. Instead, networks appear to be inherently clean, with legible building blocks throughout their architecture.
2Recursive Generalization as the Driver
What appeared to be arbitrary packing of concepts is actually a motif called recursive generalization. Neurons sprout offshoots like tree branches, expanding from simple concepts to more complex, related ones. This fractal nature of learning means that neurons are not knotted, but rather organized in a way that remains logically consistent over time.
3Revisiting InceptionV1 and Vision Models
Early studies of InceptionV1 suggested its deep layers were full of arbitrary mixtures, making them difficult to study. However, retraining the architecture with modern improvements reveals that 99-100% of neurons are easily interpretable. The original confusion stemmed from architectural overflows and weak interpretability tools rather than an inherent failure of the network's learning principles.
4Scaling Interpretability Across Model Sizes
Experiments across a 300x range in model size demonstrate that legibility persists at every scale. In smaller models, neurons simply cover coarser-grained concepts, while larger models differentiate these into specific, granular features. This confirms that the network does not switch learning principles when space is limited; it merely adjusts the granularity of its neurons.
5Modern Language Model Audits
Preliminary audits of Qwen3-32B show that approximately 90% of neuron poles have one primary job. Only a small fraction, roughly 2-4%, remain currently unexplained. These findings suggest that modern language models are far more interpretable than previous benchmarks, such as the 2023 OpenAI automated interpretability project, had led the research community to believe.
6The Role of Weak Activations
Weak activations often reveal related context rather than unrelated jobs. For example, a neuron tracking a specific person might activate for related figures or associated technologies. When researchers analyze these activations in depth, they find that the neuron is usually performing a consistent, logical function, even if initial automated tools failed to categorize it.
7Limitations of Early Interpretability Tools
The gap in understanding was largely due to compounding technical issues. Early explainer models could not effectively test conjectures, and metrics often rewarded simulations over accurate explanations. Furthermore, researchers lacked expectations for language neuron motifs and relied on small, simplified models that were prone to the very superposition effects they were trying to study.
8The Momentum of Interpretability Mapping
Mechanistic interpretability functions like a jigsaw puzzle; every piece of understanding constrains the rest of the system. Once a researcher maps a significant portion of the network, the remaining unexplained neurons become easier to decode. This creates a positive feedback loop where the map fills in faster as more circuits are successfully identified and verified.
9Contextualizing Social and Feeling Detectors
Neurons tracking subtle social cues or feelings often fire on punctuation or contextual markers rather than nouns. Without a map of surrounding circuits, these neurons appear to fire randomly. However, when the broader ecosystem is understood, these activations align perfectly with the model's predictive behavior, confirming the neuron's specific role in the network.
10Refining the Cleanliness Score
While a single 'cleanliness' score provides a rough pointer to a network's state, it should not be treated as a precise metric to optimize. The field benefits more from treating each neuron and circuit differently, respecting their unique motifs, depth, and roles within the model, rather than forcing them into a single, reductive classification.
11Direct Study Over Decomposition
The shift away from decomposition methods, like sparse autoencoders, is now possible. Because networks are not knotted, researchers can study them directly. This approach allows for testing explanations at an agentic scale, watching data flow through circuits, and verifying hypotheses against the model's actual performance, rather than relying on an approximated, unfolded version.
12Future Outlook for Interpretability
The outlook for mechanistic interpretability is significantly better than previously feared. By recognizing that neural networks are already clean, the field can move toward more effective, direct analysis. This shift in perspective simplifies the path toward understanding trillion-parameter models, as the focus moves from untangling knots to mapping the fractal nature of generalization.