arXiv · 2611.21452
A Taxonomy and Taxonomy-Guided Benchmark Reduction for Spreadsheet Tasks
Abstract
Spreadsheets let millions of end users without programming training do computational work, and teams are increasingly building AI agents to help them. Evaluating each candidate agent configuration across many tasks is costly. Benchmark reduction can lower this cost by selecting a small subset of tasks whose outcomes predict full-benchmark scores, but selecting tasks only for predictive accuracy can leave whole categories of spreadsheet work untested. We contribute a taxonomy of spreadsheet tasks, grounded in a review of benchmarks and refined with the Excel Agent product team at Microsoft, and a benchmark-reduction method that favors taxonomic diversity among similarly predictive choices. On benchmark logs from Microsoft, taxonomy-guided selection broadens coverage at little or no cost in predictive accuracy and often improves prediction on configurations excluded from selection. Broader coverage can therefore support, rather than compromise, predictive accuracy. Generalization remains limited across time: subsets selected using little recent history predict poorly for configurations much later in time. Beyond benchmark reduction, the taxonomy provides a reusable framework for auditing gaps in task coverage, as teams at Microsoft have done.
Explore related subjects
Keep this discovery
Explore connections, maps & timelines
Bogdan Vasilescu, Emerson Murphy-Hill, Anikesh Parashar, Aayush Kumar, Gustavo Soares, Mar Ginés Marín, Sumit Gulwani. 2026-11-16. A Taxonomy and Taxonomy-Guided Benchmark Reduction for Spreadsheet Tasks. https://arxiv.science/abs/2611.21452
Citations retain the original authors and DOI and include this site’s record URL. If this discovery service contributed to your research, you can also cite arXiv Science by Indiciae: BibTeX · RIS.