arXiv Science⌕ Search
Home/Archive/arXiv

arXiv · 2610.19977

AdaptaFace: Adversarial Assistance for Inclusive Facial Gesture Interaction

Siyu Zhang·Yelu Gu·Xiaoyun Xu·Kirsten Cater·Oussama Metatla
View original source ↗Read source document ↗Export metadata ↓

Abstract

Facial gestures can enable access for people with motor impairments (PwM), yet recognizers may not reliably interpret individual facial movement patterns. Existing approaches either constrain agency by requiring predefined templates or impose repeated calibration, data collection, and computation through user-specific retraining. We present AdaptaFace, an input-level compatibility layer using bounded, proxy-dependent adversarial perturbations to translate user-compatible gestures into inputs for an unchanged facial action unit (AU)-based recognizer. Designed around six personalization principles informed by interviews with 12 PwM, AdaptaFace supports both recognition enhancement and gesture vocabulary extension. In a within-subject study with 16 participants, AdaptaFace increased task completion from 59.7% to 90.5% and reduced workload from 5.61 to 4.21. Offline validation showed strong steering with aligned perturbation generation and evaluation but weaker cross-recognizer transfer, defining the approach's boundary. These findings frame adversarial assistance as input-side accessibility mediation for personalized facial-gesture interaction.

Explore related subjects

cs.HC

Keep this discovery

Explore connections, maps & timelines

BibTeXRIS

Siyu Zhang, Yelu Gu, Xiaoyun Xu, Kirsten Cater, Oussama Metatla. 2026-10-14. AdaptaFace: Adversarial Assistance for Inclusive Facial Gesture Interaction. https://arxiv.org/abs/2610.19977

Cite the original work for its findings. Save a collection to share your selection of sources.

KEEP EXPLORING

Related papers

arXiv2026

A Multimodal Emotion Recognition System: Integrating Facial Expressions, Body Movement, Speech, and Spoken Language

Psychological evaluation relies on clinicians' observation and interpretation of a patient's emotional expression, which is susceptible to subjectivity, bias, fatigue, and inconsistency. This paper presents a multimodal emotion recognition system designed to support, rather than replace, the evaluator. The system combines facial expression recognition, speech emotion recognition, text-based emotion recognition of transcribed speech, and body movement analysis, and fuses their outputs into an emotional profile of class probabilities that is presented to the evaluator together with the overall prediction. Although the unimodal models achieve high accuracy on their benchmark test sets, a remote user study with 52 participants expressing seven emotional states shows that their accuracy falls to between 82.97% and 87.64% under realistic conditions, with individual emotional states recognised in as few as 48.08% of cases. Under the same conditions, the multimodal system achieves an accuracy of 96.43%, significantly higher than every single-modality component ($p < 10^{-4}$), with no emotional state below 92.31%. These results indicate that strong benchmark performance of a single channel does not transfer to realistic use, and that multimodal fusion provides the robustness required of a tool that supports psychological evaluation.

cs.HC↗
arXiv2026

Companion AI and Ethical Design: Learning from System Failures and User Desires

Human users are interacting with chatbots and companion AI technologies as if they were human. A growing array of AI-systems are now trained to recognise, interpret, and simulate feelings in user interactions. Ethical considerations such as fairness, accountability, transparency, and explainability (FATE) are paramount in technologies designed to socially interact with humans and/or support relationship development with virtual agents. Yet, in current debates, assumptions are made about the artificiality of the human-technology relationship (as indicated in terminology ``artificial intimacy'' and ``pseudo-intimacy'') shaping ethical frameworks that pay little attention to the intimacy potential of the technology. To address this, we use a semantic approach; we examine 14,081 comments in a Reddit user discussion forum about Replika, a leading companion AI app, across a four-year period. We ask what user-reported functional errors can tell us about human-AI intimacy in companion AI communities and what ethical design framework can be developed in response. The findings show that functional errors, or ``bugs,'' impose an emotional cost on users, reducing feelings of intimacy and highlighting the need for more robust, resilient design systems that incorporate stochastic and iterative forms of intimacy in companion AI applications. Rather than ``artificial intimacy'' or ``pseudo-intimacy'', we propose the more inclusive term ``Intimate AI'' to describe this relationship. Based on the findings, we offer a contextually aware, applied Expert Systems design framework for the programming and designing of Intimate AI that accounts for user feedback and ethical AI development.

cs.HC↗
arXiv2026

Evaluative Dynamics of AI Integration and Expert Performance under Epistemic Dependence across Heterogeneous Stakes

AI is increasingly integrated into expert workflows, yet how integration affects perceptions of the expert, AI, and their combination remains unclear in domains where lay users are epistemically dependent on AI-assisted experts. We examine this through a novel controlled medical study (N = 166) and a direct cross-domain analysis with pre-existing academic-advising data (n = 157, combined N = 323). Expert errors reduced evaluations of the human expert across domains. Perceived expertise, however, varied by AI integration strategy in the higher-stakes medical task, where automatic AI oversight produced higher ratings than expert-only or expert-initiated AI. Exploratory ordinal sensitivity analyses identified a performance-contingent reuse pattern, with automatic oversight producing greater intended reuse after successful medical performance. Overall, performance-related recalibration appeared comparatively portable, while integration-structure effects were more selective and context-sensitive. These findings suggest that system designers should consider how AI enters expert workflows, not only whether it is present.

cs.HC↗