The most "active" parts of an AI's brain are almost entirely unrelated to the actual decisions the AI makes.
April 24, 2026
Original Paper
Variance Is Not Importance: Structural Analysis of Transformer Compressibility Across Model Scales
arXiv · 2604.20682
The Takeaway
A common intuition in AI research is that high-variance signals are the most important carriers of information. This study proves that 96% of those signals are actually uncorrelated with the model's final predictions. This means that researchers have been focusing on the wrong parts of the network when trying to compress or understand models. The true logic of the model is hidden in much quieter, low-variance directions. This discovery will force a total redesign of the tools we use to analyze and prune neural networks. We must learn to listen to the whispers of the network rather than the shouts.
From the abstract
We present a systematic empirical study of transformer compression through over 40 experiments on GPT-2 (124M parameters) and Mistral 7B (7.24B parameters). Our analysis covers spectral compression, block-level function replacement, rotation-based quantization, activation geometry, and adaptive early exit.We identify five structural properties relevant to compression. (1) Variance is not importance: high-variance activation directions are approximately 96 percent uncorrelated with predictive dir