Factual selection in LLMs is driven by rotational dynamics on a hypersphere rather than scalar magnitude shifts, with the behavior emerging suddenly at the 1.6B parameter mark.
arXiv · March 17, 2026 · 2603.13259
The Takeaway
The discovery of rotational geometry for factuality provides a new way to probe and monitor model truthfulness. The identified phase transition at 1.6B parameters suggests that factual processing capability is a threshold-based emergent property rather than a linear scaling result.
From the abstract
When a language model is fed a wrong answer, what happens inside the network? Current understanding treats truthfulness as a static property of individual-layer representations-a direction to be probed, a feature to be extracted. Less is known about the dynamics: how internal representations diverge across the full depth of the network when the model processes correct versus incorrect continuations.We introduce forced-completion probing, a method that presents identical queries with known correc