Proposes 'semantic sections' as a replacement for global feature vectors to interpret LLMs in complex, non-linear representation spaces.
March 24, 2026
Original Paper
Semantic Sections: An Atlas-Native Feature Ontology for Obstructed Representation Spaces
arXiv · 2603.20867
The Takeaway
Challenges the prevailing assumption that features in LLMs correspond to single global directions. This framework allows researchers to track features that change meaning across contexts (twisted sections), providing a more mathematically rigorous path for mechanistic interpretability.
From the abstract
Recent interpretability work often treats a feature as a single global direction, dictionary atom, or latent coordinate shared across contexts. We argue that this ontology can fail in obstructed representation spaces, where locally coherent meanings need not assemble into one globally consistent feature. We introduce an atlas-native replacement object, the semantic section: a transport-compatible family of local feature representatives defined over a context atlas. We formalize semantic sections