arXiv Science⌕ Search

arXiv · 2611.16315

Shared Item-Level Error Limits the Aggregation of Small Language Models in Fermi Estimation

Abstract

Multi-agent systems of language models are often motivated by an averaging argument: if the agents' errors are unbiased and independent, the error of an aggregate of N answers falls as 1/sqrt(N). Fermi estimation, whose continuous answers are aggregated on the logarithmic scale, permits a direct test. We decompose the log10 error of each answer into the mean error of its model on the item, common to all its draws on the item, and a residual, and estimate both from 2.51M answers given before any communication by 15 open-weight models (3B-72B) on realFP, with replication under a single prompt pair on realFP, Science Olympiad Fermi questions and a guesstimation set. Given the item, teammates' errors are uncorrelated and teams behave as independent draws, but the item-level component accounts for a median 0.906 of the error variance over items. Teams of thirty therefore reduce a single agent's error by a median 1.7% (at most 5.4%), whereas draws that were also unbiased on each item would reduce it by 74-79%. All models compress their estimates toward about 10^6; this trend explains a median 17% of the item-level component, and a cross-validated linear recalibration on labelled items removes the overall bias but not that component. Three rounds of debate reduce within-team dispersion by 93% and leave the item-level component unchanged; revision without peers attains the same small gain. Mixing five models of similar accuracy reduces error by 6.6% relative to their own homogeneous teams, but no mixture improves on the most accurate model. Higher sampling temperature changes team error by about 1% at most, and contrastive activation steering not detectably; per-agent scaffold prompts (not significant after multiplicity correction) and skipping one layer in a single-model screen improve it. The results are conditional on the models, benchmarks, prompts and settings tested.

Explore related subjects

Keep this discovery

Explore connections, maps & timelines

BibTeXRIS

Carolina Fortuna, Blaz Bertalanic. 2026-11-12. Shared Item-Level Error Limits the Aggregation of Small Language Models in Fermi Estimation. https://arxiv.science/abs/2611.16315

Citations retain the original authors and DOI and include this site’s record URL. If this discovery service contributed to your research, you can also cite arXiv Science by Indiciae: BibTeX · RIS.

KEEP EXPLORING

Related papers

Equilibrium decisions under partial observation: dynamical error bounds and information requirements

Reduced monitoring can preserve stable dynamics while changing an equilibrium decision. Motivated by heterogeneous urban-mobility indicator coverage, we study fixed linear observations of a nonlinear recovery flow coupled to a strategic fixed-point map, and ask when a decision based on the reduced model can be trusted. The main result is a joint certificate: bounds on trajectory tracking, equilibrium displacement and unresolved output dependence are assembled into one decision-margin error which, under a stated Gaussian observation model, yields a conservative one-sided test and an explicit information requirement for prescribed worst-case power. In the scalar experiment this requirement is sharp: when the effect does not exceed twice the error budget, no valid test has worst-case power above its size, however many observations are taken. A second result connects manifold-adapted observations to defect bounds through a locally invertible observation chart, while distinguishing observation initialisation from asymptotic phase initialisation. Supporting results, resting on established lumping, contraction and fixed-point theory, cover exact quotients, an observation-fibre obstruction and the dependence of recovery-time comparisons on the tolerance convention. Seven reproducible synthetic experiments examine each component. The results give conditional criteria for certifying partial monitoring and separate mathematical guarantees from empirical validation.

cs.MA↗

Even Loose Proximity Constraints Make Makespan-optimal Unlabeled Multi-agent Pathfinding Hard

We study $r$-Connected Unlabeled Multi-agent Pathfinding ($r$CUMAPF), a variant of MAPF where interchangeable agents have a proximity range $r\in \mathbb{N}$, and agents must form a single group in a given graph $G$ through limited-range communication at all times. The problem arises in fundamental applications such as self-reconfiguration and platooning, while standard MAPF does not capture proximity constraints. For the problem of finding a makespan-optimal solution, prior work shows NP-hardness for $r=1$ and polynomial-time solvability when $r$ is sufficiently large; the complexity for the intermediate case $2\le r$ is largely unknown. To narrow this gap, we show that finding a solution to $r$CUMAPF with makespan at most $2$ is NP-hard for $2\le r= O(|V|^{1-\varepsilon})$, where $0<\varepsilon\le 1$ is a fixed constant and $|V|$ is the number of vertices. As a consequence, we show that the labeled analogue of $r$CUMAPF is also NP-hard under the same condition.

cs.MA↗

Addressing Exogenous Variability in Cooperative Multi-Agent Reinforcement Learning

Cooperative multi-agent reinforcement learning (MARL) often fails under exogenous variability such as shifts in opponents or environment regimes that cannot be controlled by the team but alter the transition dynamics. We formalize this challenge as Exogenous Dec-POMDP (ED-POMDP), in which an episode-level exogenous regime, selected independently of the team, indexes the dynamics, and the global state is partitioned into components that are directly responsive to the current team action and components that are not directly action-responsive at the current step. This formulation exposes mediated influence: while team actions do not directly update the latter at the current step, they can shape their later evolution through induced changes in the former. Based on this, we propose LEICA, a CTDE-compatible algorithm that learns history-conditioned context representations for both components and shapes policy updates using agent-wise, influence-weighted counterfactual intrinsic rewards. Across SMAX benchmarks with opponent strategy shifts, LEICA consistently improves both training performance and generalization to unseen opponents' strategies over existing baselines, supporting the usefulness of influence-weighted shaping under exogenous train-test regime shifts.

cs.MA↗