arXiv Science⌕ Search

subject

cs.HC

cs.HC: explore 158 source-linked works published from 2025 to 2026, with original documents and citations.

This collection is a preview while coverage and quality are evaluated.

Search within this collection

Coverage and selection

Includes records with this source-supplied label or an explicit phrase match in their metadata. Matches indicate a mention, not proof that a paper uses a method or tests a material. Source versions are consolidated by DOI.

Sources: arxiv. Collection updated 2026-09-15. Counts describe this index, not the complete source archives.

arXiv2026

From Prompt to Service: An SLM-Based Agent Orchestration Gateway for AI-Driven Virtual Worlds

As generative AI capabilities expand, AI-driven virtual worlds face a growing architectural challenge. Users interact through in-world interfaces in multimodal ways, yet their requests demand fundamentally different AI backend models and computational resources. Embedding these capabilities directly into virtual world systems reduces extensibility, complicates maintenance, and limits the ability to coordinate services distributed across edge and cloud infrastructure. This paper presents an SLM-based Agent Orchestration Gateway, a lightweight runtime coordination mechanism that decouples a virtual world client from heterogeneous AI backends through intent-driven service routing. An edge-deployed SLM classifies the semantic intent of each user prompt, a configurable service registry validates and resolves the routing decision, and the selected backend is invoked transparently, enabling new AI capabilities to be introduced in the virtual world without modifying the client application. The gateway is implemented and evaluated within the InterwovenXR virtual museum testbed. The evaluation shows that compact SLMs can serve as reliable intent routers on edge hardware, and that task-specific fine-tuning can transform sub-billion-parameter models into practical, low-latency routers. A layered configuration pairing a fine-tuned sub billion-parameter model as router with a larger SLM for conversational response generation is shown to be deployable on mid-range edge hardware and more efficient than delegating both responsibilities to a single model. The findings show that SLMs can support practical AI service orchestration in virtual worlds and the work contributes an evaluated architecture for scalable, extensible, and edge-supported AI interaction, enabling virtual agents become access points to distributed generative AI services.

cs.AI↗
arXiv2026

Personality Without Persons? A Psychometric Critique of Big Five Testing in Large Language Models

Human personality inventories are increasingly used to characterize large language models (LLMs), compare systems, and inform downstream governance claims. Yet, these inventories were developed and validated for humans, and it remains unclear whether they are valid for non-human systems. We present a systematic psychometric evaluation of Big Five personality measurement in LLMs. We ask three research questions: Do Big Five inventories a) appropriately describe LLMs, b) capture meaningful differences between models, and c) reflect internal factors consistent with human personality? We assess the content validity of five candidate Big Five inventories and administer the best-performing inventory to N = 264 LLMs spanning 50 model families. Our findings are threefold. First, Big Five items adapted for LLMs achieve acceptable content validity, whereas the original human-developed items do not. Second, Big Five inventories fail to capture meaningful differences across LLMs: between-model variance accounts for only 7% - 17% of the total score variance. Third, LLMs responses do not reproduce the canonical Big Five five-factor structure of human personality, with four of the five personality facets collapsing into one (r >= .90). Moreover, comparisons between base and instruction-tuned variants suggest that alignment training shifts Big Five scores toward socially desirable profiles. These findings demonstrate that Big Five inventories do not measure a construct equivalent to human personality in LLMs. Thus, using human personality frameworks to characterize, benchmark, compare, or govern LLMs risks producing misleading conclusions. We highlight the need for evaluation frameworks that are specifically designed and validated for LLMs, rather than transferring human psychological constructs without first establishing their validity.

cs.HC↗
arXiv2026

EITWatch: Smartwatch-Integrated Planar Electrical Impedance Tomography for Hand Gesture Recognition

Wrist Electrical Impedance Tomography (EIT) senses hand gestures from muscle- and tendon-driven impedance changes, but prior wrist-EIT systems require electrode coverage beyond the watch-back contact patch and separate analog front ends. We present EITWatch, the first wrist-EIT system built around smartwatch case-back geometry, asking whether this contact patch alone can support gesture recognition: eight planar electrodes in a 31 mm ring acquire 35 impedance measurements at 48 Hz. Because a planar array cannot encircle the wrist, EITWatch uses multi-depth scanning to sample multiple source-sink distances and current paths; it beat matched adjacent injection by 15.1/10.4 percentage points (macro/micro) across all 12 participants. In a prompted study, within-session leave-one-round-out accuracy reached 91.4%/92.5% (window/trial) for six macro-gestures, and 90.1%/91.5% (window/segment) for five micro-gestures plus relax; window-level cross-session and leave-one-user-out transfer reached 73.2%/70.4% and 63.1%/55.3% (macro/micro).

cs.HC↗
arXiv2026

Exploring Breathing-Music Coupling: Using the Breathing Mirror for Somatic Reflection in Piano Performance

While breathing is essential to living and for sound production in some instruments, for pianists, it is often a hidden and automatic process, making it difficult to analyze or refine. A critical gap exists between data and awareness: while sensors record precise physical metrics, they fail to capture the performer's somatic experience. Conversely, the high cognitive load of performance makes it nearly impossible for musicians to recall their internal states with temporal precision. To address this, we present a system, Breathing Mirror, and associated methodology designed to externalize the pianist's internal somatic experience through three analytical lenses: a Baseline View (synchronized signals), a First-Person View (subjective recall), and an Interpersonal View (collaborative reflection). Through a four-week longitudinal study with a skilled amateur pianist (35 years of experience), we evaluated the system's effectiveness by recording respiratory data using textile-integrated strain sensor belts. The results show that the Breathing Mirror reveals some patterns of breathing-music coupling and identifies critical blind spots where objective data diverges from subjective perception. Furthermore, we propose four somatic themes regarding the link between breathing and musical elements, offering a foundation for future large-scale validation across a broader range of pianists. This work provides a new way to study body signals, transforming breathing from an internal biological function into an articulate expressive parameter.

cs.HC↗
arXiv2026

Knowing Is Not Enough: Information Retrievability as a Precondition to Effective LLM Oversight

Large language models (LLMs) are increasingly embedded in organizational work, yet their errors often pass human review. Prior research locates such failures in users' capability to review LLM output or their engagement in doing so. We develop an alternative, retrieval-based account of human oversight and posit that error detection is more effective when oversight-relevant information is accessible to users at the moment of review. Across two randomized lab-in-the-field experiments with 640 customer-facing employees, we show that self-generated explanations improve error detection and strengthen recall of verification-relevant reasoning, while cues that reactivate such reasoning help sustain detection under repeated LLM use. Theoretically, we identify information retrievability as a distinct precondition for effective oversight and specify generative encoding and cue-supported reactivation as mechanisms that build and sustain it. Practically, lightweight onboarding self-explanations and daily retrieval cues can make human oversight more resilient as LLM use becomes routine.

cs.HC↗
arXiv2026

Reconciling Kinesthetic Mismatches: A Somatic Alignment Mindset for Musical Body Transformation

Mastering musical performance requires precise multisensory coordination, yet learners encounter a kinesthetic mismatch, which is a discrepancy between the internal perception of an action and the actual physiological state of the body. While multisensory Body Transformation Experiences (BTE) provide tools to bridge this gap, existing designs often focus on external correction rather than internal alignment. To address this, we propose the Somatic Alignment Mindset (SAM), a conceptual lens that integrates Taoist philosophy to shift the focus of HCI design from prescriptive feedback toward holistic embodied unity. By positioning technology as a reflective medium, SAM operationalizes the principles of Adaptation, Assessment, and Awareness to reconcile somatic discrepancies and foster deep, self-aligned musical mastery.

cs.HC↗
arXiv2026

Propose to Learn, Learn to Propose: Evaluability-Aware Assistance under Bounded Rationality

AI assistants often collaborate by proposing candidate edits, plans, or designs that users evaluate before adoption. Existing assistance methods focus on proposal quality or user-goal inference, often assuming that the user can reliably evaluate any proposal, which can fail in practice because of bounded rationality. We study evaluability-aware proposal planning, where proposals serve both as task interventions and as probes for learning latent preferences and evaluation constraints, where the resulting belief updates then guide later proposals. We formalise this setting as ProSE, a hidden-parameter sequential assistance problem, and instantiate it with a KL-regularised bounded-rational binary response model in which acceptance trades off value gain against a distance-dependent evaluability penalty. Analysing the planning consequence of this likelihood reveals that likely accepted proposals and informative probes need not coincide, which explains why planners that only pursue acceptance systematically underperform. We operationalise ProSE with \textsc{ProSE-Plan}, a depth-2 Bayes-adaptive planner that scores proposals by possible responses and response-induced posterior beliefs. In controlled graph simulations, \textsc{ProSE-Plan} improves over evaluability-unaware and myopic baselines when evaluation cost is the bottleneck, and a probe-commit ablation confirms that our approach selects informative proposals that simpler methods miss. Our results thus identify user evaluability as a planning-relevant dimension of AI assistance, complementary to generation quality and preference inference.

cs.AI↗
arXiv2026

Removing Speech, Keeping Activities: A Privacy Firewall for Acoustic Sensing in Assisted Living

Acoustic sensing offers a promising non-intrusive approach for monitoring daily activities of older adults, yet speech privacy concerns remain a critical barrier to real-world deployment. We present a privacy firewall pipeline based on a U-Net encoder-decoder, trained entirely on synthetic data, that removes speech from ambient audio while preserving environmental sounds indicative of daily activities. Activity recognition is performed using VGGish transfer learning with an SVM classifier. Evaluated on the ESC-50 and SINS datasets across multiple speech content levels, the proposed model reduced residual speech to 0% VAD-detectable speech (Silero Voice Activity Detection) under all tested conditions, outperforming Facebook Denoiser (6.55% residual), SepFormer (36.34%) and ConvTasNet (47.21%) on ESC-50 at the 100\% speech level. On ESC-50 at 40% speech level, classification performance recovers to 85% precision and 85% recall after speech removal, compared with 81%/75% before removal and an 84%/83% speech-free baseline. Evaluation on real-world participant home recordings collected with the AudioHive app showed 0% VAD-detectable speech after processing while maintaining 76% precision and recall. The pipeline enables privacy-preserving acoustic sensing without sacrificing activity recognition performance, addressing a key obstacle to the adoption of ambient monitoring in elderly care.

cs.SD↗
arXiv2026

Decoding Decision Correctness from EEG Under High Cognitive Workload in Virtual Reality: Implications for Collaborative Brain-Computer Interface Teams

Collaborative Brain-Computer Interfaces (cBCIs) offer a promising mechanism to augment team decision-making, but existing approaches rely exclusively on evidence available only after a decision has been made and reported, such as reaction time or stated confidence. This limits their use to explaining or discounting a decision after the fact, rather than informing a team's response before it is finalised. We tested whether spatial-covariance EEG features could instead provide a genuinely pre-emptive signal of an operator's decision correctness, available within the response window itself, and whether such a signal depends on cognitive workload. Using a continuous virtual reality target-detection task, participants (N = 23) completed a within-subject workload manipulation (High vs. Low). At the team level, weighting votes by this pre-emptive neural signal, available before a response is committed, produced substantial accuracy gains on contested (evenly-split) trials under High Workload (57% to 88% as team size increased from 2 to 16), but was actively detrimental under Low Workload. Critically, this advantage held even against post-hoc behavioural signals: confidence was the strongest single team-level signal overall, but by definition cannot inform a decision still in progress, whereas the neural signal can. These findings indicate that EEG-based decision-reliability signals are not a general-purpose team augmentation tool, but a workload-conditional one, with clear implications for when and how cBCI systems should be deployed in operational teams.

cs.HC↗
arXiv2026

Modeling and Optimizing User Preferences in AI Copilots: A Comprehensive Survey and Taxonomy

AI copilots represent a new generation of AI-powered systems designed to assist users, particularly knowledge workers and developers, in complex, context-rich tasks. As these systems become more embedded in daily workflows, personalization has emerged as a critical factor for improving usability, effectiveness, and user satisfaction. Central to this personalization is preference optimization: the system's ability to detect, interpret, and align with individual user preferences. While prior work in intelligent assistants and optimization algorithms is extensive, their intersection within AI copilots remains underexplored. This survey addresses that gap by examining how user preferences are operationalized in AI copilots. We investigate how preference signals are sourced, modeled across different interaction stages, and refined through feedback loops. Building on a comprehensive literature review, we define the concept of an AI copilot and introduce a taxonomy of preference optimization techniques across pre-, mid-, and post-interaction phases. Each technique is evaluated in terms of advantages, limitations, and design implications. By consolidating fragmented efforts across AI personalization, human-AI interaction, and language model adaptation, this work offers both a unified conceptual foundation and a practical design perspective for building user-aligned, persona-aware AI copilots that support end-to-end adaptability and deployment.

cs.AI↗
arXiv2026

Unsupervised Partner Design Enables Robust Ad-hoc Teamwork

We introduce Unsupervised Partner Design (UPD), a population-free multi-agent reinforcement learning method for robust ad-hoc teamwork. UPD generates training partners on-the-fly and selects them adaptively based on a learnability criterion, removing the need for pre-trained partner populations or manual parameter tuning. We show that this simple mechanism enables effective partner diversity and can be extended to joint partner-environment selection when a procedural level generator is available. Across Level-Based Foraging, Overcooked-AI, and the Overcooked Generalisation Challenge, UPD consistently achieves strong performance compared to both population-based and population-free baselines. In a human-AI user study, agents trained with UPD achieve higher returns and are rated as more adaptive, more human-like, and less frustrating than all evaluated baseline methods.

cs.LG↗
arXiv2026

Fact-Checking Comments Precede Community Notes to Misleading Posts on X

Community-based fact-checking systems, such as X's Community Notes program, offer a potentially scalable approach against online misinformation diffusion. However, their efficacy is substantially undermined by the delay of fact-check delivery. To address this challenge, we investigated the promise of organic user comments as rapid corrective signals by analyzing a large-scale dataset of over 2.2 million comments directed to 1,841 community fact-checked misleading posts. We developed a high-performance language model pipeline to identify ``fact-checking (FC) comments'', i.e., comments that correct source posts with reasoning or evidence (91% accuracy). Using this pipeline, we find that 99.4% of misleading posts receive their first FC comments before the official community note is created. Notably, the median time to the initial FC comment is only 0.1 hours, while the creation of community notes has a median latency of 9.7 hours since the publication of misleading posts. Additionally, compared to random posts, those misleading posts with displayed notes have significantly more FC comments. This suggests that FC comments are a unique pattern for misleading posts. We further identified characteristics of FC comments' intensity and speed: (i) FC comments' volume is significantly associated with content richness and misinformation type, favoring multimedia content and missing-context scenarios; (ii) the speed of FC comments remains robust across diverse topics and emotional intensities. These results suggest that platforms could use FC comments as early-warning signals, and potential resource for complementing community notes or automated synthesis.

cs.HC↗
arXiv2026

Toward Interaction Dynamics: A Predictive Framework for Safe Physical Human Robot Interaction

Physical human-robot interaction requires yielding transiently to contact yet recovering the commanded reference under sustained load. Finite-stiffness impedance control retains a static deflection there, while predictive alternatives typically optimize a nonlinear robot or impedance model online. Operational-space cancellation instead exposes a translational error double integrator with a fixed transition matrix and a configuration-scheduled input map, making interaction a predictive quantity rather than a property re-derived per configuration. We build on it a compact offset-free interaction-error MPC for torque-controlled manipulators: a force-domain random-walk state estimates persistent interaction and model error, and a 30-variable convex QP maps the correction through the current task inertia while constraining the applied joint torque. Conditional results establish impedance equivalence of the unconstrained passive feedback, offset-free regulation at feasible frozen configurations, and quadratic stabilizability of the scheduled backbone. In a 1kHz MuJoCo simulation of a 7-DOF Franka FR3, the estimator cuts steady-state error under a repeated 15N step from 2.77mm to 0.042mm when added to the otherwise identical 100Hz MPC. A stiffness-and-damping-calibrated impedance baseline attains 2.59mm but briefly saturates and needs 3.3x the peak positive joint power. Adding ideal measured-force cancellation to that baseline gives 1.39mm, so constant-load rejection is not unique to MPC; the sensorless controller still reaches 0.042mm in the moving task, a 65x reduction without force sensing and without the baseline's saturation or power cost. Demonstrated in simulation under a shared actuator budget, the contribution is an efficient operational-space realization complementing rather than replacing broader interaction-control architectures.

cs.RO↗
arXiv2026

Self-EmoQ: Plutchik-Guided Value-based Planning to Drive Streaming Emotional TTS

Emotional interaction is increasingly crucial for conversational AI, yet current systems lack a self-emotion determination mechanism to drive the streaming text-to-speech (TTS) synthesis. We propose an emotion-planning framework that determines the emotion prior to the textual generation, grounding the downstream emotional TTS in a streaming manner. The framework is implemented by a plug-and-play LLM module, initialized from pretrained LLMs, and trained by reinforcement learning (RL) with emotions as the actions. A hybrid reward is employed which combines imitation signals with theory-driven scoring, in which the theory of Plutchik's wheel of emotions is adopted. By experiments on DailyDialog, EmoryNLP, IMEOCAP, and MELD, our method outperforms prompting and finetuning baselines on both emotion determination and response quality. We finally implement an entire streaming pipeline for real-time deployment, with the speech quality confirming the framework's emotional alignment, contextual coherence, and expressive fluency. Codes, cases, and demos are available in https://sixingdeguo.github.io/EmoQ-page/.

cs.HC↗
arXiv2026

DigitalCoach: Communication and Grounding Gaps in Human and Agentic Computer Use Coaching

Agents are increasingly capable of automating software tasks, but can they teach humans how to use software themselves? We introduce DigitalCoach, a multimodal dataset of 72 human expert-novice computer use coaching sessions consisting of 22,752 dialogue turns grounded in 28.1 hours of screen and input event recordings across five software applications. We use DigitalCoach to evaluate whether state-of-the-art models can teach humans how to use computers. Automated evaluation shows that models differ from humans in how they coach: models provide more direct instructions, but fewer explanations, error diagnoses, and knowledge-check questions. When we fix the coaching method, models produce utterances similar to human references yet poorly grounded in visual context. Interactive evaluation confirms that model coaches cause learners to passively follow instructions without deeper engagement and fall short in visual grounding. DigitalCoach lays a foundation for collaborative and proactive computer use coaching agents. Data and code are available at https://project-digital-coach.vercel.app.

cs.CL↗
arXiv2026

Deceptive Patterns as a Sociotechnical Phenomenon: Review, Catalog, and Discussion

Background: Deceptive patterns are interface design strategies aimed at misleading users or favoring specific interests, compromising user experiences and ethical privacy principles. These patterns involve exploit different issues of the interaction between humans and technology, revealing gaps in the scientific literature regarding their understanding and working mechanisms. Purpose: This research characterizes deceptive patterns as a sociotechnical phenomenon, integrating human and technical dimensions while offering a self-explainable interactive catalog to raise awareness among users and designers. Methods: To characterize deceptive patterns as a sociotechnical phenomenon, a systematic literature review in Computer Science was conducted, and the Semiotic Framework was applied to analyze and organize the sociotechnical aspects of identified patterns in an integrated manner. An exploratory evaluation of the interactive catalog was conducted, combining heuristic evaluation and focus group. Results: The research identified multiple deceptive patterns in scientific literature. Analysis revealed that existing studies frequently addressed specific aspects of patterns, such as typologies or effects, and do not focus on sociotechnical issues. The exploratory evaluation suggested that the self-explainable catalog is useful and easy to use, with potential to inform users and promote awareness regarding the deceptive patterns existence and functioning. Conclusion: The study contributes to an informed understanding of deceptive patterns, highlighting the need for an approach that integrates human and technical dimensions. The self-explainable catalog is a promising tool to inform and promote awareness about the topic. By exposing the risks and mechanisms of these patterns, the research seeks to promote awareness among users and designers towards ethical interface design practices.

cs.HC↗
arXiv2026

DOBI: Dynamic Opportunistic Body Input via Spare Joint Recruitment for Hands-Free XR

Extended Reality (XR) systems are often most useful when users are engaged in ongoing physical tasks, yet current interaction techniques still largely assume the hands are available. We present opportunistic body input, an interaction paradigm that redirects continuous XR control to whichever available body region remains free in the moment. To investigate how users naturally coordinate these spare-body movements, we conducted an elicitation study across six hand-busy scenarios. We found that while users' preferred spare body regions shift dynamically based on physical constraints, the resulting spontaneous movements share a consistent, low-dimensional kinematic structure organized around a dominant principal axis. Building on these findings, we present DOBI (Dynamic Opportunistic Body Input), a real-time XR technique that uses gaze to target a UI element, a brief trigger gesture to identify the recruited spare body region, and the region's subsequent motion to drive continuous 1D control. A 1D Fitts' law study establishes the baseline motor performance of this paradigm across four distinct body regions, achieving throughputs up to 2.62 bits/s with an overall 5.0% error rate, and a dual-task usability study shows that DOBI supports reliable, low-effort control (SUS = 84.2) while users remain engaged in realistic hand-busy activities.

cs.HC↗
arXiv2026

UniScale: Exploring Unimanual Gesture Mapping Strategies for Gaze+Pinch-based Scaling Interaction

Object scaling serves as a fundamental spatial manipulation that enables complex and productive tasks in XR environments. This paper investigates unimanual scaling techniques for XR using gaze and hand interactions. We propose UniScale, a set of unimanual alternatives to the standard bimanual pinch, allowing users to scale objects while preserving hand availability for concurrent spatial manipulations. We design five distinct mapping strategies based on physical metaphors, exploring unimanual control that varies depth, angle, micro-gestures, and finger-distance input. We then compare these techniques against a standard bimanual baseline, in which users adjust the inter-hand distance via a bimanual pinch gesture. In a user study, we evaluate their effectiveness in a 3D object scaling task under both clutching and clutching-free conditions. The results indicate that while bimanual scaling relies on clutching for stable control, unimanual techniques excel in clutching-free conditions, significantly reducing physical hand movement. From the results, we derive valuable design implications for developing efficient 3D multimodal interactions in XR.

cs.HC↗
Compare source metadata on this page
WorkPublishedSource identifierSource
From Prompt to Service: An SLM-Based Agent Orchestration Gateway for AI-Driven Virtual Worlds2026-09-022606.03557arxiv
Personality Without Persons? A Psychometric Critique of Big Five Testing in Large Language Models2026-09-022607.02325arxiv
EITWatch: Smartwatch-Integrated Planar Electrical Impedance Tomography for Hand Gesture Recognition2026-09-022608.29415arxiv
Exploring Breathing-Music Coupling: Using the Breathing Mirror for Somatic Reflection in Piano Performance2026-09-022609.01974arxiv
Knowing Is Not Enough: Information Retrievability as a Precondition to Effective LLM Oversight2026-09-022609.01976arxiv
Reconciling Kinesthetic Mismatches: A Somatic Alignment Mindset for Musical Body Transformation2026-09-022609.01981arxiv
Propose to Learn, Learn to Propose: Evaluability-Aware Assistance under Bounded Rationality2026-09-022609.02242arxiv
Removing Speech, Keeping Activities: A Privacy Firewall for Acoustic Sensing in Assisted Living2026-09-022609.02376arxiv
Decoding Decision Correctness from EEG Under High Cognitive Workload in Virtual Reality: Implications for Collaborative Brain-Computer Interface Teams2026-09-022609.02436arxiv
Modeling and Optimizing User Preferences in AI Copilots: A Comprehensive Survey and Taxonomy2026-09-012505.21907arxiv
Unsupervised Partner Design Enables Robust Ad-hoc Teamwork2026-09-012508.06336arxiv
Fact-Checking Comments Precede Community Notes to Misleading Posts on X2026-09-012509.11052arxiv
Toward Interaction Dynamics: A Predictive Framework for Safe Physical Human Robot Interaction2026-09-012606.08281arxiv
Self-EmoQ: Plutchik-Guided Value-based Planning to Drive Streaming Emotional TTS2026-09-012606.09837arxiv
DigitalCoach: Communication and Grounding Gaps in Human and Agentic Computer Use Coaching2026-09-012606.31980arxiv
Deceptive Patterns as a Sociotechnical Phenomenon: Review, Catalog, and Discussion2026-09-012608.27684arxiv
DOBI: Dynamic Opportunistic Body Input via Spare Joint Recruitment for Hands-Free XR2026-09-012608.30341arxiv
UniScale: Exploring Unimanual Gesture Mapping Strategies for Gaze+Pinch-based Scaling Interaction2026-09-012609.00500arxiv

These are bibliographic comparisons, not experimental rankings. Follow the original document for methods and conditions.