arXiv Science⌕ Search
Home/Archive/arXiv

arXiv · 2610.20519

Probabilistic AI models outperform numerical weather prediction models on record breaking heat extremes

Jakob Benjamin Wessel·Fiona Spuler·Christopher A. T. Ferro·Marlene Kretschmer·Sebastian Sippel
View original source ↗Read source document ↗Export metadata ↓

Abstract

AI weather prediction (AIWP) models have emerged as a key complement to physics-based numerical weather prediction (NWP) models, and now often outperform them on aggregate skill. Yet, several studies report that physics-based models still forecast extreme events better, in particular record-breaking extremes, based on the evaluation of deterministic forecasts. Here we evaluate heat extreme forecasts from probabilistic AIWP models over global land and show that they clearly outperform the ECMWF IFS ensemble, both raw and post-processed, including for record-breaking events, in all regions and at all lead times considered. AIWP models better forecast the occurrence and intensity of extremes, are better calibrated in the tail and miss fewer records. Station observations confirm these results. We argue that many reported AIWP limitations stem from unsuitable verification choices, specifically comparing deterministic forecasts that target different quantities. Instead, a fair evaluation of extremes necessitates probabilistic verification.

Explore related subjects

stat.APphysics.geo-ph

Keep this discovery

Explore connections, maps & timelines

BibTeXRIS

Jakob Benjamin Wessel, Fiona Spuler, Christopher A. T. Ferro, Marlene Kretschmer, Sebastian Sippel. 2026-10-14. Probabilistic AI models outperform numerical weather prediction models on record breaking heat extremes. https://arxiv.org/abs/2610.20519

Cite the original work for its findings. Save a collection to share your selection of sources.

KEEP EXPLORING

Related papers

arXiv2026

Bayesian Adaptive Tensor Smoothing for Multiway Demographic Counts

Demographic data are often collected across multiple dimensions such as time, age, subnational region, and sex. Modeling such data is challenging due to the interplay of high dimensionality, heterogeneity between groups, and stochastic variability within small groups. This paper proposes a Bayesian adaptive tensor smoothing model for multiway demographic counts that represents the latent log-rate array as a sum of rank-one components. Global-local shrinkage priors control the effective rank of the decomposition, the number of modes along which each component varies, and the smoothness that Gaussian process priors impose along ordered modes such as age and time. Posterior inference relies on a data augmentation Markov chain Monte Carlo algorithm that interweaves centered and noncentered parameterizations in every layer of the model to keep mixing satisfactory in large and sparse count arrays. Applied to Swedish county-level and sex-specific mortality from 1969 to 2024, the model recovers stable demographic surfaces and life table quantities from noisy small-population data. In cross-validation exercises, the method outperforms both canonical demographic benchmarks and existing count tensor factorizations.

stat.AP↗
arXiv2026

Leveraging Multi-Model Land-Use Emissions Reveals a Rising Airborne Fraction of Atmospheric $CO_2$ Over the Past Six Decades

The airborne fraction, the share of anthropogenic $CO_2$ emissions that remains in the atmosphere rather than being taken up by land and ocean, is a compact diagnostic of sink efficiency and a direct input to the remaining carbon budgets behind climate targets. Here we show that it has risen over the observational record, from $0.32$-$0.40$ in $1959$ to $0.47$-$0.50$ by $2024$ (ranges across estimators), above the $0.44$ commonly reported for recent decades. Applied to IPCC AR6 assessments, this rise reduces the remaining carbon budget by $23$-$48$ $GtCO_2$ for a given temperature target, up to roughly one year of current global fossil emissions. The trend has been difficult to detect because two features of the standard analysis obscure it. First, three high-leverage years following the $1991$ Pinatubo eruption dominate the volcanic term of the natural-variability adjustment and attenuate the estimated trend; smoothing those years, or using a volcanically robust $ENSO$ index, removes the artifact. Second, collapsing land-use and land-cover change emissions to a single series discards the disagreement among models that carries the information needed to resolve a small trend. Propagating the dispersion of $69$ land-use series together with the reported fossil-emission uncertainty into delta-method weights, weighted and generalized least squares recover a positive and significant trend, corroborated by a mixed-effects model and robust across a grid of $49$ natural-variability specifications that maps which treatment choices reproduce which published result.

stat.AP↗
arXiv2026

LLM Benchmarking via Representation Multi-task Learning

Quantifying and evaluating the capabilities of Large Language Models (LLMs) remains a fundamental challenge in modern data science and artificial intelligence. In this paper, we consider LLM evaluation based on their performance across items in multiple benchmark domains (e.g., mathematical reasoning and coding) within a leaderboard framework. Our goal is to address two core questions: (1) How do we derive more accurate domain-specific scores by borrowing information across domains? and (2) How do we define and estimate an overall score that aggregates performance across multiple domains? To solve these problems, we propose a novel statistical framework based on representation Multi-task Learning (MTL) and an item response theory model. Specifically, we define overall and domain-specific LLM traits through an Item Response Theory (IRT) model, and propose an MTL approach to estimate these traits from item-level response data. We develop a computationally efficient estimator and establish its minimax optimality under certain asymptotic regimes. This framework provides a rigorous measurement foundation for systematic LLM evaluation. We conduct extensive simulations, demonstrating the superior performance of the proposed method over competing methods. Crucially for the Applications and Case Studies section, we apply the proposed framework to MMLU response data from the Hugging Face Open LLM Leaderboard, covering 4,272 LLMs and 13,232 items across 56 subjects. The empirical analysis reveals substantial heterogeneity in domain size and difficulty, together with strong positive cross-domain dependence, highlighting the practical value and substantive insights generated by our approach.

stat.AP↗