This paper demonstrates that the order of training examples alone can encode information not present in any individual example, allowing models to bypass established sample complexity bounds.
March 27, 2026
Original Paper
The Order Is The Message
arXiv · 2603.25047
The Takeaway
It challenges the fundamental assumption that IID training is optimal, showing that a structured sequence can achieve high accuracy with 0.3% of the data where IID training fails. This suggests a massive, untapped channel for training efficiency and data curation.
From the abstract
In a controlled experiment on modular arithmetic ($p = 9973$), varying only example ordering while holding all else constant, two fixed-ordering strategies achieve 99.5\% test accuracy by epochs 487 and 659 respectively from a training set comprising 0.3\% of the input space, well below established sample complexity lower bounds for this task under IID ordering. The IID baseline achieves 0.30\% after 5{,}000 epochs from identical data. An adversarially structured ordering suppresses learning ent