A backbone-agnostic denoising objective that allows small GNNs to outperform large models pretrained on much larger supervised datasets in physical sciences.
March 19, 2026
Original Paper
Self-Conditioned Denoising for Atomistic Representation Learning
arXiv · 2603.17196
The Takeaway
It demonstrates that self-conditioned denoising can match or exceed the performance of expensive DFT force-energy pretraining. This significantly lowers the compute barrier for creating high-performance foundation models in chemistry and materials science.
From the abstract
The success of large-scale pretraining in NLP and computer vision has catalyzed growing efforts to develop analogous foundation models for the physical sciences. However, pretraining strategies using atomistic data remain underexplored. To date, large-scale supervised pretraining on DFT force-energy labels has provided the strongest performance gains to downstream property prediction, out-performing existing methods of self-supervised learning (SSL) which remain limited to ground-state geometrie