arXiv · 2610.30641
How good are universal MLIPs for alloys? A benchmark is only as trustworthy as its controls
Abstract
Universal machine-learned interatomic potentials (uMLIPs) are ranked by their formation-energy error against a DFT database, as if that database were ground truth, but independent DFT databases disagree. We benchmark fourteen uMLIPs on 1.45 million ordered intermetallics against AFLOW, which none trained on, and compare every energy and stability error with that disagreement. For half the structures, AFLOW, Alexandria, and OQMD differ by less than 6-8 meV/atom, but a minority differ by far more, raising the average to 16-30. That disagreement sets a floor on any model's measurable error. Re-referenced, the best uMLIP agrees with AFLOW as closely as Alexandria does, and the next four sit 2-7 meV/atom above the floor. Once broken elemental references are screened, the energy and stability rankings agree almost everywhere. Three apparent model failures shrink or vanish under the right control: poorly correlated shear moduli, thousands of "bad" DFT references, and the leader's inflated lead.
Explore related subjects
Keep this discovery
Explore connections, maps & timelines
Gus L. W. Hart, Jacob N. Pursley. 2026-10-22. How good are universal MLIPs for alloys? A benchmark is only as trustworthy as its controls. https://arxiv.org/abs/2610.30641
Cite the original work for its findings. Save a collection to share your selection of sources.