arXiv · 2611.11248
FESOM2.8 in single precision: climate fidelity over six decades and performance on CPUs and GPUs
Abstract
Halving the floating-point precision of an ocean model can make it substantially faster, but whether the rounding errors stay negligible over climate time scales has rarely been tested. We implement single precision in the global ocean-sea-ice model FESOM2 and compare 60-year hindcasts with double-precision controls. A first implementation lost salt steadily, accumulated a heat-content deficit of 129 ZJ, and shifted major ocean fronts. Controlled experiments trace the drift to the tracer update: small increments added to a large salinity value are rounded away, and the increments themselves are biased. Accumulating tracer tendencies and layer thicknesses in 64 bits, and carrying the rounding error of each tracer update into the next, removes the drift for at most 8% additional runtime. Single- and double-precision simulations then differ in heat content and mean state about as much as two double-precision runs with slightly perturbed initial conditions, sea ice differs by at most 0.2%, and he salinity difference is four orders of magnitude smaller than before. In the coupled model AWI-ESM3, single precision adds no detectable heat source or sink over 60 years. Single precision speeds up the 60-year hindcasts by a factor of 1.35, and the model by up to 1.66 on meshes of up to 5 km resolution; on GPUs, a C++/Kokkos version runs up to 1.27 times faster and needs 43% less memory. Whether reduced precision is safe for climate integrations is decided by long-term budgets, not by short tests.
Explore related subjects
Keep this discovery
Explore connections, maps & timelines
Suvarchal Kumar Cheedela, Nikolay V. Koldunov, Sergey Danilov, Jan Streffing, Dmitry Sidorenko, Thomas Jung. 2026-11-08. FESOM2.8 in single precision: climate fidelity over six decades and performance on CPUs and GPUs. https://arxiv.science/abs/2611.11248
Citations retain the original authors and DOI and include this site’s record URL. If this discovery service contributed to your research, you can also cite arXiv Science by Indiciae: BibTeX · RIS.