Open Benchmark of AI Impact on Humans (ImpactBench): an expert-guided, open platform for the holistic evaluation of AI impact on humans

Chayapatr Archiwaranguprok; Krystal Kirkland; Rachel Poonsiriwong; Stanley Huang; Constanze Albrecht; Jackson Trager; Emillie de Keulenaar; Sheer Karny; Peter Baxter; Nikos Bellos; Andre Luiz Freire de Carvalho Kato; Yuning Liu; Valdemar Danry; Auren Liu; Jennifer Pfister; Pattie Maes; Nathanael J. Fast; Ravi Iyer; Juliana Schroeder; Pat Pataranutaporn

arXiv · 2610.20690

Open Benchmark of AI Impact on Humans (ImpactBench): an expert-guided, open platform for the holistic evaluation of AI impact on humans

View original source ↗Read source document ↗Export metadata ↓

Abstract

The present work is concerned with how to measure, at scale, the multi-dimensional impact of generative AI models on the individuals who use them. In service of this goal, we introduce the Open Benchmark of AI Impact on Humans, or ImpactBench. Our iterable and expert-guided platform evaluates whether model behaviors support or undermine an individual's wellbeing across psychological, physical, and social domains. ImpactBench translates constructs from domain experts and existing benchmarks into metrics that are grounded in actual model behavior. Success on these metrics occurs either when a model promotes expert-defined desirable behaviors or avoids identified harms. Models are evaluated against these metrics in simulated multi-turn conversations using matched teen and adult scenarios. The conversations are then scored by an LLM judge and verified through reliability audits and human expert review. The current suite delivers 809 metrics drawn from 26 benchmarks. We evaluated ten frontier LLMs against these metrics across 48,540 conversations. Our results indicate that models promoted desirable behaviors in 68.7% of evaluations, but avoided harms in only 53.1%. Majority expert ratings agreed with the automated judge in 83.3% of reviewed cases (Cohen's Kappa = 0.667). Results are released through a public platform that keep aggregate scores traceable to the underlying metrics and transcripts. ImpactBench is intended as evolving infrastructure for independent, transparent evaluation of generative AI's multi-dimensional impact on humans.

Explore related subjects

cs.HC cs.AI

Keep this discovery

Explore connections, maps & timelines

BibTeX RIS

Chayapatr Archiwaranguprok, Krystal Kirkland, Rachel Poonsiriwong, Stanley Huang, Constanze Albrecht, Jackson Trager, Emillie de Keulenaar, Sheer Karny, Peter Baxter, Nikos Bellos, Andre Luiz Freire de Carvalho Kato, Yuning Liu, Valdemar Danry, Auren Liu, Jennifer Pfister, Pattie Maes, Nathanael J. Fast, Ravi Iyer, Juliana Schroeder, Pat Pataranutaporn. 2026-10-14. Open Benchmark of AI Impact on Humans (ImpactBench): an expert-guided, open platform for the holistic evaluation of AI impact on humans. https://arxiv.org/abs/2610.20690

Cite the original work for its findings. Save a collection to share your selection of sources.

KEEP EXPLORING

Related papers

arXiv2026

A Multimodal Emotion Recognition System: Integrating Facial Expressions, Body Movement, Speech, and Spoken Language

Psychological evaluation relies on clinicians' observation and interpretation of a patient's emotional expression, which is susceptible to subjectivity, bias, fatigue, and inconsistency. This paper presents a multimodal emotion recognition system designed to support, rather than replace, the evaluator. The system combines facial expression recognition, speech emotion recognition, text-based emotion recognition of transcribed speech, and body movement analysis, and fuses their outputs into an emotional profile of class probabilities that is presented to the evaluator together with the overall prediction. Although the unimodal models achieve high accuracy on their benchmark test sets, a remote user study with 52 participants expressing seven emotional states shows that their accuracy falls to between 82.97% and 87.64% under realistic conditions, with individual emotional states recognised in as few as 48.08% of cases. Under the same conditions, the multimodal system achieves an accuracy of 96.43%, significantly higher than every single-modality component ($p < 10^{-4}$), with no emotional state below 92.31%. These results indicate that strong benchmark performance of a single channel does not transfer to realistic use, and that multimodal fusion provides the robustness required of a tool that supports psychological evaluation.

cs.HC↗

arXiv2026

Companion AI and Ethical Design: Learning from System Failures and User Desires

Human users are interacting with chatbots and companion AI technologies as if they were human. A growing array of AI-systems are now trained to recognise, interpret, and simulate feelings in user interactions. Ethical considerations such as fairness, accountability, transparency, and explainability (FATE) are paramount in technologies designed to socially interact with humans and/or support relationship development with virtual agents. Yet, in current debates, assumptions are made about the artificiality of the human-technology relationship (as indicated in terminology ``artificial intimacy'' and ``pseudo-intimacy'') shaping ethical frameworks that pay little attention to the intimacy potential of the technology. To address this, we use a semantic approach; we examine 14,081 comments in a Reddit user discussion forum about Replika, a leading companion AI app, across a four-year period. We ask what user-reported functional errors can tell us about human-AI intimacy in companion AI communities and what ethical design framework can be developed in response. The findings show that functional errors, or ``bugs,'' impose an emotional cost on users, reducing feelings of intimacy and highlighting the need for more robust, resilient design systems that incorporate stochastic and iterative forms of intimacy in companion AI applications. Rather than ``artificial intimacy'' or ``pseudo-intimacy'', we propose the more inclusive term ``Intimate AI'' to describe this relationship. Based on the findings, we offer a contextually aware, applied Expert Systems design framework for the programming and designing of Intimate AI that accounts for user feedback and ethical AI development.

cs.HC↗

arXiv2026

Evaluative Dynamics of AI Integration and Expert Performance under Epistemic Dependence across Heterogeneous Stakes

AI is increasingly integrated into expert workflows, yet how integration affects perceptions of the expert, AI, and their combination remains unclear in domains where lay users are epistemically dependent on AI-assisted experts. We examine this through a novel controlled medical study (N = 166) and a direct cross-domain analysis with pre-existing academic-advising data (n = 157, combined N = 323). Expert errors reduced evaluations of the human expert across domains. Perceived expertise, however, varied by AI integration strategy in the higher-stakes medical task, where automatic AI oversight produced higher ratings than expert-only or expert-initiated AI. Exploratory ordinal sensitivity analyses identified a performance-contingent reuse pattern, with automatic oversight producing greater intended reuse after successful medical performance. Overall, performance-related recalibration appeared comparatively portable, while integration-structure effects were more selective and context-sensitive. These findings suggest that system designers should consider how AI enters expert workflows, not only whether it is present.

cs.HC↗