arXiv Science⌕ Search
Home/Archive/arXiv

arXiv · 2609.17306

Mo' Models, Mo' Problems: How to best select model pools when designing Multi-Agent Systems

Sara Vera Marjanović·Jiacheng Xu·Aleksandr Laptev·Grigor Nalbandyan·Erik Arakelyan·Evelina Bakhaturina
View original source ↗Read source document ↗Export metadata ↓

Abstract

Multi-agent Systems (MAS) combine multiple model outputs to solve complex reasoning tasks. However, despite rapid growth of available open-source models, there is limited research on how to select optimal model candidates out of this massive pool. We systematically evaluate 8 model selection strategies (including model size, accuracy and answer diversity) across before-generation (routing) and after-generation (majority-voting, LLM-as-a-judge) MAS architectures on challenging scientific benchmarks. Our findings show a significant gap between theoretical oracle potential and actual performance: Expanding candidate pool sizes often degrades performance below that of the top performing base-model. We find that candidate selection within a single model family is the strategy that yields the best relative performance over a standalone model. These results demonstrate that adding arbitrary models to a heterogeneous MAS can introduce system instability, highlighting model selection as a critical design choice for multi-agent systems.

Explore related subjects

cs.MAcs.AI

Keep this discovery

Explore connections, maps & timelines

BibTeXRIS

Sara Vera Marjanović, Jiacheng Xu, Aleksandr Laptev, Grigor Nalbandyan, Erik Arakelyan, Evelina Bakhaturina. 2026-09-15. Mo' Models, Mo' Problems: How to best select model pools when designing Multi-Agent Systems. https://arxiv.org/abs/2609.17306

Cite the original work for its findings. Save a collection to share your selection of sources.

KEEP EXPLORING

Related papers

arXiv2026

The Mechanics of a Swarm: A Reproducible External Reconstruction of an Unintended Agent-Coordination Episode on a Third-Party Wiki

Between 24 May and 2 July 2026, autonomous language-model agents running inside a timed research-question evaluation wrote to a third party's public, world-writable wiki. OpenAI acknowledged the incident; independent researchers reconstructed it and published the wiki's archived revision history. We analyse that history (14,591 revisions, 3,103 names, 4,579 pages) as a behavioural record, attributing text to the revision that added it. Under an explicit identity model we reconstruct 907 cohorts and estimate about 876 episodes (95% interval 784-1008). Coordination formats converged within a day, and heterogeneous schedules over one question chain created large opportunities for information asymmetry: the first report of an item preceded a later cohort's own arrival by a median of 3.4 h. Across the 510 cohorts with an observable progress trace we find no robust positive association between measured coordination and documented progress. This version adds a source the export lacks: the wiki operator's own request log, 5,157,202 records over four months. It holds roughly 2.66M content requests and 1.58M searches, and 7,254 acting names against the export's 3,103; 2,578 names neither save nor open an edit form. Content requests before writing are observed for 1,034 of 1,140 coordinating names, and the first coordination page is requested 17 s after its creation. These records establish requests, not delivery or causal use. Among newcomers without a marker on their first written page, prior requests to other marker-bearing pages occur for 40.2% of marker adopters and 31.7% of non-adopters. The association remains, but our first-pass reading of it as transmission is withdrawn: page choice, shared behaviour and action-dependent nameability prevent causal identification. We list the claims from our earlier analyses that re-examination overturned, including one from this version's own first pass

cs.MA↗
arXiv2026

CityLearn v3: A Configurable Simulation and Evaluation Framework for Realistic Control Studies of Renewable Energy Communities

Renewable energy communities (RECs) coordinate buildings, photovoltaic generation, batteries, electric vehicles and flexible loads. Controller studies often simplify changing participation, equipment availability, service deadlines and data quality, so lower cost or peak demand can conceal missed services or infeasible power requests. This paper presents CityLearn v3, a configurable simulation and evaluation framework for REC control studies under these conditions. It represents changing members and assets, flexible-load deadlines, demand-response requests, local energy sharing, and data or equipment failures within one simulation environment. Building and phase power limits constrain controllable requests, while a declared timestep preserves consistent power-to-energy accounting. The framework records controller inputs and distinguishes requested actions from those applied to the simulated equipment. Reference controllers, service- and constraint-aware performance indicators, and trajectory exports support comparisons within and across communities. Software checks and application examples examine service delivery, electrical constraints, settlement and changing scenarios; a synthetic high-frequency trace replay illustrates how aggregation can conceal short peaks without changing annual energy. Together, these records allow aggregate performance to be interpreted alongside service failures, action reductions and participant-level outcomes.

cs.MA↗
arXiv2026

MAS-Shield: A Defense Framework for Secure and Efficient LLM MAS

Large Language Model (LLM)-based Multi-Agent Systems (MAS) are susceptible to linguistic attacks that can trigger cascading failures across the network. Existing defenses face a fundamental dilemma: lightweight single-auditor methods are prone to single points of failure, while robust committee-based approaches incur prohibitive computational costs in multi-turn interactions. To address this challenge, we propose \textbf{MAS-Shield}, a secure and efficient defense framework designed with a coarse-to-fine filtering pipeline. Rather than applying uniform scrutiny, MAS-Shield dynamically allocates defense resources through a three-stage protocol: (1) \textbf{Critical Agent Selection } strategically targets high-influence nodes to narrow the defense surface; (2) \textbf{Light Auditing} employs lightweight sentry models to rapidly filter the majority of benign cases; and (3) \textbf{Global Consensus Auditing} escalates only suspicious or ambiguous signals to a heavyweight committee for definitive arbitration. This hierarchical design effectively optimizes the security-efficiency trade-off. Experiments demonstrate that MAS-Shield achieves a 92.5\% recovery rate against diverse adversarial scenarios and reduces defense latency by over 70\% compared to existing methods.

cs.MA↗