Achieves high-performance online continual learning without the massive memory overhead of traditional experience replay buffers.
March 19, 2026
Original Paper
Abstraction as a Memory-Efficient Inductive Bias for Continual Learning
arXiv · 2603.17198
The Takeaway
By using abstraction-augmented training to capture relational structures rather than raw instances, models can learn from non-stationary streams without forgetting. This challenges the assumption that replay buffers are necessary for stable continual learning.
From the abstract
The real world is non-stationary and infinitely complex, requiring intelligent agents to learn continually without the prohibitive cost of retraining from scratch. While online continual learning offers a framework for this setting, learning new information often interferes with previously acquired knowledge, causes forgetting and degraded generalization. To address this, we propose Abstraction-Augmented Training (AAT), a loss-level modification encouraging models to capture the latent relationa