SDZE enables the training of 10-million-dimensional Physics-Informed Neural Networks (PINNs) on a single GPU.
March 26, 2026
Original Paper
Stochastic Dimension-Free Zeroth-Order Estimator for High-Dimensional and High-Order PINNs
arXiv · 2603.24002
The Takeaway
It solves the memory and complexity bottlenecks of high-order PDEs by using backpropagation-free zeroth-order optimization. This effectively bypasses the O(d^k) complexity of spatial derivatives, opening the door for massive-scale scientific machine learning on consumer hardware.
From the abstract
Physics-Informed Neural Networks (PINNs) for high-dimensional and high-order partial differential equations (PDEs) are primarily constrained by the $\mathcal{O}(d^k)$ spatial derivative complexity and the $\mathcal{O}(P)$ memory overhead of backpropagation (BP). While randomized spatial estimators successfully reduce the spatial complexity to $\mathcal{O}(1)$, their reliance on first-order optimization still leads to prohibitive memory consumption at scale. Zeroth-order (ZO) optimization offers