arXiv · 2611.15463
Right Answers, Inconsistent Evidence: Source-Level Counterfactual Auditing of Large Audio-Language Models
Abstract
A large audio-language model (LALM) can answer "is a cat present?" about a mixture correctly, and its confidence can drop when the cat is removed, yet this does not show that it can tell the cat's presence from its absence. We introduce TRACE. It deletes the queried source and each rival in turn and asks whether every view still containing that sound outscores the view with it removed. Across 8 LALMs and 3,456 controlled mixtures, 6.3% of correct answers fail this ordering: the target-absent view scores at or above some target-present view, a conflict no shared threshold repairs. A target-only score check misses 42.5% of them. The share rises from 4.9% to 8.4% as target SNR falls from +5 to -5 dB, even on units correct at every level, and survives loudness and re-pairing controls. Accuracy certifies the answer; TRACE audits whether it orders presence above absence. Code and data: https://github.com/yikise/TRACE.
Explore related subjects
Keep this discovery
Explore connections, maps & timelines
Yi Zhang, Yi Li, Hongwei Du. 2026-11-11. Right Answers, Inconsistent Evidence: Source-Level Counterfactual Auditing of Large Audio-Language Models. https://arxiv.science/abs/2611.15463
Citations retain the original authors and DOI and include this site’s record URL. If this discovery service contributed to your research, you can also cite arXiv Science by Indiciae: BibTeX · RIS.