arXiv subjects

Aditya Raikwar

Publications and source records attributed to Aditya Raikwar.

5 recordsLinked to original sources

arXiv2026

Do Video-LLMs Actually Watch? Diagnosing Character-Tracking Failures in Long-Form Video

Can a Video Large Language Model (Video-LLM) follow one person through a long video, keeping track of who they are well enough to report, in order, how their outfit changes across a full TV episode? Benchmarks increasingly score this kind of task, and the strongest open-source 7--8B models now reach 37--38% on InfiniBench's global appearance task, which asks exactly that. But does that score come from tracking the named character, or from something easier? We test this with a nine-condition diagnostic protocol applied to three architecturally distinct open-source Video-LLMs, with Gemini~2.5~Flash as a frontier reference, and find the accuracy does not come from character tracking. When we change the character named in the question to a different cast member, leaving the video and answer options untouched, the models change their answer only 4--31% of the time, so they are largely ignoring who the question asks about. Breaking that test down by the gender of the swapped name shows why: the models react more when the name is changed to a different-gender character than to a same-gender one (a 13--28 point gap), picking up coarse gender cues but unable to tell same-gender individuals apart. This shallow processing surfaces again when we drop the multiple-choice options and ask the same questions open-endedly: open-source accuracy drops 18--25 points, with none of 151 answers fully correct, versus a 12-point drop for Gemini. Further checks rule out the obvious innocent explanations, adding subtitles, using the most informative frames, or doubling the number of frames all leave character tracking unimproved, so the bottleneck is not how much video the model sees but how it ties that video to the person the question names. We release a diagnostic toolkit for auditing what such benchmark scores actually measure.

cs.CV↗

arXiv2026

AFA: Identity-Aware Memory for Preventing Persona Confusion in Multi-User Dialogue

When multiple people share a single voice assistant, the system conflates their histories: one resident's preferences can leak into another's responses, eroding utility and trust. We call this failure mode persona confusion, and we show it is a measurable problem in today's single-user dialogue systems when deployed in shared environments. We present the Adaptive Friend Agent (AFA), a modular framework that combines voice-based speaker identification with per-user memory stores to enable identity-aware, personalized dialogue across multiple users. To support training and evaluation, we construct PAT (Personalized Agent chaT), a synthetic dataset of 58,289 persona-grounded dialogue turns spanning 133 user profiles and 12 real-world scenarios. We evaluate AFA across five LLM back-ends in a standard response-quality benchmark, with a LLaMA-2-70B model fine-tuned on PAT achieving the highest overall performance. To directly measure persona confusion prevention, we introduce an interleaved multi-user evaluation protocol with a novel metric, Persona Attribution Accuracy (PAA), demonstrating that identity-aware routing improves PAA from 35.7% to 61.3%. Human evaluation confirms annotators perceive significantly higher personalization in routing-enabled responses. Our results establish that identity-aware user routing is the critical component for preventing persona confusion in multi-user conversational systems.

cs.HC↗

arXiv2025

Beyond Cognitive Load: AI-Based Estimation of Cognitive Effort Using Brain Signals During Digital Tasks

Cognitive effort, defined as the relationship between cognitive load and task performance, provides insight into how individuals allocate mental resources during demanding tasks. This construct is particularly important in high-stakes public health and clinical training, where excessive cognitive load is associated with medical errors and burnout. This study investigates whether cognitive effort varies across task segments and whether it can be estimated at the individual level using brain signal data and machine learning. Functional near-infrared spectroscopy (fNIRS) data were collected from 16 participants performing a structured digital cognitive task consisting of four sequential segments separated by short and long rest intervals. Cognitive effort was operationalized using relative neural efficiency and relative neural involvement, integrating prefrontal hemodynamic activity with task performance. The analysis followed a two-stage approach. First, segment-level group analysis tested whether cognitive effort differed across task segments, assessing whether the task structure induced meaningful variation in cognitive demand. Second, participant-independent machine learning models were used to predict task performance from brain signal features. These predicted scores were then combined with neural measures to estimate individual-level cognitive effort. Results showed significant differences in cognitive effort across the four task segments, indicating that variations in task structure influence collective cognitive efficiency. In addition, machine learning models successfully predicted performance from fNIRS data. Cognitive effort derived from predicted scores closely matched that based on actual performance, suggesting that the proposed metric primarily reflects brain signal patterns.

cs.HC↗

arXiv2024

Neurophysiological Insights into Multimedia-based Education: A PRISMA-ScR Review of fNIRS in Game-Integrated Learning Systems

Game-integrated learning systems (GILS) are a growing form of multimedia education. Brain-based evidence can help researchers and designers understand how GILS design choices shape how learners think and process information. This scoping review follows PRISMA-ScR and synthesizes 20 empirical studies (2014-2025) in which functional near-infrared spectroscopy (fNIRS) measured brain activity during GILS use. This corpus shows that fNIRS can capture brain responses across GILS platforms and game elements, and points to how neurophysiological evidence can inform multimedia design decisions, such as that different platforms activate different brain regions, that adaptive difficulty reduces cognitive load and improves performance simultaneously, and that collaborative gameplay predicts knowledge retention. The 20 studies in this corpus reflect a field with substantial room to grow. Causal links between brain activation and learning outcomes would give designers more reliable evidence for platform decisions. As fNIRS and multimedia devices improve, standardized methods, classroom settings, and real-time neural adaptation represent directions where future work can translate these findings into practical multimedia learning systems.

cs.HC↗

arXiv2024

Ping! Your Food is Ready: Comparing Different Notification Techniques in 3D AR Cooking Environment

Implementing visual and audio notifications on augmented reality devices is a crucial element of intuitive and easy-to-use interfaces. In this paper, we explored creating intuitive interfaces through visual and audio notifications. The study evaluated user performance and preference across three conditions: visual notifications in fixed positions, visual notifications above objects, and no visual notifications with monaural sounds. The users were tasked with cooking and serving customers in an open-source Augmented-Reality sandbox environment called ARtisan Bistro. The results indicated that visual notifications above objects combined with localized audio feedback were the most effective and preferred method by participants. The findings highlight the importance of strategic placement of visual and audio notifications in AR, providing insights for engineers and developers to design intuitive 3D user interfaces.

cs.HC↗