arXiv ScienceSearch

arXiv subjects

Xiaojiang Peng

Publications and source records attributed to Xiaojiang Peng.

2 recordsLinked to original sources

OneEmo: A Unified Multimodal Reasoning Model for Emotion Perception, Understanding, and Interaction

Multimodal Large Language Models (MLLMs) have demonstrated remarkable capabilities in emotional intelligence. However, prevailing research predominantly focuses on task-specific specialization, often neglecting inter-task synergy and leaving latent reasoning potential underexplored. To bridge this gap, we introduce OneEmo, a unified affective generalist capable of mastering emotion perception, comprehension, and interaction. For this purpose, we first construct EmoWorld-130K, a comprehensive dataset that distills specialized affective knowledge into explicit reasoning trajectories via a human-in-the-loop workflow. Supervised fine-tuning on this corpus reveals significant mutual benefits derived from multi-task learning. Second, to fully unlock the latent reasoning potential, we propose Emo-Chord, a novel reinforcement learning strategy that stabilizes optimization through unified multi-task reward allocation. Extensive experiments demonstrate that OneEmo achieves state-of-the-art performance against similarly sized baselines across most benchmarks. Notably, despite having significantly fewer parameters than commercial models, OneEmo delivers highly competitive results. This paper paves the way for more reliable and interpretable affective computing. The code is available at https://github.com/waHAHJIAHAO/OneEmo.

cs.HC

Robust Speech Emotion Recognition under Tone-Word Conflict: A Benchmark and Framework

Speech emotion recognition (SER) is a crucial component of human-computer interaction, attracting extensive attention from both industry and academia. However, existing SER systems typically assume alignment between vocal tone and lexical semantics, overlooking the real-world scenarios that involve tone-word conflict-where the emotion conveyed by speech contradicts the literal meaning of the words. To bridge this gap, we introduce TWIN-SER (Tone-Word Incongruent SER), a benchmark for systematic evaluation under acoustic-semantic incongruence, and show that state-of-the-art models degrade severely under such incongruence. To address this, we propose DAS (Disentangled Acoustic-Semantic fusion), a framework that mitigates tone-word conflict by explicitly disentangling acoustic and semantic pathways, selecting informative high-energy embeddings, and adaptively fusing them via a lightweight query-based attention mechanism. Specifically, DAS comprises three crucial modules: i) a heterogeneous feature extraction module that separately captures complementary acoustic and semantic representations from raw input; ii) a high-energy embedding selection module that identifies and retains the most discriminative embeddings; and iii) a Q-Former combination module that bridges the two pathways through cross-attention, enabling robust emotion prediction under incongruent conditions. Extensive experiments demonstrate that DAS consistently outperforms existing methods in tone-word conflict scenarios, as well as in standard in-domain and zero-shot settings. Our code and datasets are available at https://github.com/24DavidHuang/FAS

eess.AS