Proposes multi-cluster memory for test-time adaptation, proving that a single unstructured memory pool is fundamentally insufficient for non-i.i.d. data streams.
March 24, 2026
Original Paper
One Pool Is Not Enough: Multi-Cluster Memory for Practical Test-Time Adaptation
arXiv · 2603.21135
The Takeaway
Most TTA methods assume a single distribution, but real-world streams are multi-modal. This method uses lightweight descriptors to organize memory into clusters, preventing catastrophic interference and improving adaptation performance by up to 12% on complex benchmarks.
From the abstract
Test-time adaptation (TTA) adapts pre-trained models to distribution shifts at inference using only unlabeled test data. Under the Practical TTA (PTTA) setting, where test streams are temporally correlated and non-i.i.d., memory has become an indispensable component for stable adaptation, yet existing methods universally store amples in a single unstructured pool. We show that this single-cluster design is fundamentally mismatched to PTTA: a stream clusterability analysis reveals that test strea