arXiv Science⌕ Search
Home/Archive/arXiv

arXiv · 2502.09479

Assessing Generative AI value in a public sector context: evidence from a field experiment

Trevor Fitzpatrick·Seamus Kelly·Patrick Carey·David Walsh·Ruairi Nugent
View original source ↗Read source document ↗Export metadata ↓

Abstract

The emergence of Generative AI (Gen AI) has motivated an interest in understanding how it could be used to enhance productivity across various tasks. We add to research results for the performance impact of Gen AI on complex knowledge-based tasks in a public sector setting. In a pre-registered experiment, after establishing a baseline level of performance, we find mixed evidence for two types of composite tasks related to document understanding and data analysis. For the Documents task, the treatment group using Gen AI had a 17% improvement in answer quality scores (as judged by human evaluators) and a 34% improvement in task completion time compared to a control group. For the Data task, we find the Gen AI treatment group experienced a 12% reduction in quality scores and no significant difference in mean completion time compared to the control group. These results suggest that the benefits of Gen AI may be task and potentially respondent dependent. We also discuss field notes and lessons learned, as well as supplementary insights from a post-trial survey and feedback workshop with participants.

Explore related subjects

q-fin.GNcs.LGecon.GNq-fin.EC

Keep this discovery

BibTeXRIS

Trevor Fitzpatrick, Seamus Kelly, Patrick Carey, David Walsh, Ruairi Nugent. 2025-02-13. Assessing Generative AI value in a public sector context: evidence from a field experiment. https://arxiv.org/abs/2502.09479

Cite the original work for its findings. Save a collection to share your selection of sources.

KEEP EXPLORING

Related papers

arXiv2026

AI for AI: Optimizing Additional Infrastructure Build-out to Power Artificial Intelligence Data Centers

The twenty-first century's transformative technology, artificial intelligence, is increasingly constrained by the twentieth century's transformative technology, the electricity grid. Rapid growth in electricity demand from data centers is leading to higher electricity prices, without a compensating supply-side response. We develop a framework linking data-center load growth, available generation capacity, and market-clearing prices to understand this phenomenon. We first analyze a deterministic model to show how differing estimates of demand and supply growth rates affect prices. We then model the expansion of new data centers and their associated electricity demand, together with build-outs of new electricity supply, as stochastic processes,resulting in probabilistic distributions of supply, demand, and prices rather than a single forecast. Finally, we formulate generation expansion as a stochastic control problem in which a revenue-maximizing investor dynamically chooses the intensity of supply-side investments. The analysis highlights a central challenge of the data-center build-out: even when rapid demand growth increases the need for new generation, the uncertainties related to load forecasts, development execution risks, and value cannibalization from overbuilding capacity may weaken incentives to invest at the pace required to keep electricity prices stable.

q-fin.GN↗
arXiv2026

Measuring DeFi Risk

Decentralized finance (DeFi) lending has grown from nonexistent in 2017 to nearly 40 billion US Dollars in deposited funds in May 2022. Using cryptocurrency as collateral, the platforms match speculative margin trading with yield-seeking depositors lending coins pegged to the dollar (stable coins). Depositors receive claims guaranteed by a basket of collateral, akin to new stable coins. We develop a framework requiring only knowledge of aggregate deposits and borrowings to measure overall system risks to lenders and borrowers. Using evidence from major protocols, the measures identify an increase in system fragility beyond prudent levels around mid 2021, with a potential loss of peg for extreme variations in coin prices. Overall, the model offers an easily implementable aggregate risk metric capturing the perspectives of synthetic investors and offers early warning signals as the industry is moving from deposits guaranteed by collateral to fiat money.

q-fin.GN↗
arXiv2026

Historical Reflections on Interest Rates and the Emergence of the Yield Curve

This text grew out of a historical introduction initially written for a study of interest rates in cryptocurrency markets. The difficulty of defining a term structure for a currency without a conventional bond market led naturally to a more fundamental question: under what historical conditions does a yield curve become observable at all? Credit existed long before modern money, and interest-bearing loans are documented as early as ancient Mesopotamia. For much of history, the surviving evidence lacks the institutional features that facilitate reliable comparisons of interest rates by maturity: standardised debt instruments, sufficiently homogeneous borrowers, regular issuance over a range of maturities, observable market prices, and liquid secondary markets. We trace the gradual emergence of these conditions from ancient Mesopotamia, Greece, and Rome, through medieval and early modern Europe, to the development of modern sovereign debt markets in the nineteenth and twentieth centuries.

q-fin.GN↗