Under the Skin: Backdoor Infection in Multimodal Large Language Models

Jiawen Wang; Ruo Yang; Haoran Dai; Binghui Wang; Ning Luo

Home/Archive/arXiv

arXiv · 2610.24058

Under the Skin: Backdoor Infection in Multimodal Large Language Models

Jiawen Wang·Ruo Yang·Haoran Dai·Binghui Wang·Ning Luo

View original source ↗Read source document ↗Export metadata ↓

Abstract

Multimodal Large Language Models (MLLMs) have achieved impressive performance in generating text conditioned on inputs from different modalities. In this work, we introduce backdoor infection, a cross-modal attack framework targeting MLLMs. Starting from a model already injected with a backdoor in one modality, our attack constructs a corresponding trigger in another modality, called the infected modality. Exploiting the model's shared latent space, the trigger in the infected modality reliably activates the malicious behavior, without modifying or poisoning the infected modality during training. We further develop a lightweight, constraint-aware optimization method for efficient and practical trigger generation. Extensive experiments across several state-of-the-art MLLMs, including LLaVA, InternVL, Qwen-VL, and Gemma, demonstrate that our approach identifies cross-modal triggers that are more efficient, effective, and less detectable than direct optimization baselines, highlighting a critical new threat vector in multimodal systems.

Explore related subjects

cs.CR

Keep this discovery

Explore connections, maps & timelines

BibTeX RIS

Jiawen Wang, Ruo Yang, Haoran Dai, Binghui Wang, Ning Luo. 2026-10-16. Under the Skin: Backdoor Infection in Multimodal Large Language Models. https://arxiv.org/abs/2610.24058

Cite the original work for its findings. Save a collection to share your selection of sources.

KEEP EXPLORING

Related papers

arXiv2026

Mitigating Data Reconstruction Attacks in Federated Learning: Conditions, Bounds, and Theoretical Guarantees

Privacy attacks, particularly data reconstruction attacks, pose significant challenges to federated learning despite its promise in privacy-preserving collaborative training. This paper presents a comprehensive theoretical analysis of privacy leakage in federated learning through dual perspectives: linear algebra and optimization theory. By analyzing the client-update Jacobian, we identify conditions under which distinct data batches produce the same model update and derive a sufficient batch-size criterion for non-unique reconstruction. Through optimization theory, we establish a novel upper bound for privacy leakage that incorporates batch size, distortion extent, and other critical parameters. These theoretical insights elucidate the fundamental relationships between system parameters and privacy vulnerability, laying the groundwork for developing more robust privacy-preserving federated learning frameworks.

cs.CR↗

arXiv2026

A Privacy-Preserving Federated Learning Method for Heterogeneous Clients

Federated Learning (FL) with Differential Privacy (DP) can ensure the protection of healthcare data privacy against malicious user attacks. However, FL with the DP method faces an inherent trade-off: stronger privacy protection degrades predictive accuracy due to injected noise. On the other hand, Homomorphic Encryption (HE) enables computations on encrypted data and the aggregation of encrypted gradients without introducing DP-induced noise, thereby potentially increasing the predictive accuracy. However, it may increase the computation cost. To improve the predictive accuracy while considering the computational ability of heterogeneous clients, we propose a Privacy-Preserving Machine Learning (PPML)-Hybrid method. In the proposed PPML-Hybrid method, clients distributedly select either HE or DP based on their computational resources, so that HE clients contribute noise-free updates while DP clients reduce computational overhead. Meanwhile, clients with high computational resources can flexibly adopt HE or DP according to their privacy needs. Performance evaluation on omics datasets shows that our proposed method achieves comparable predictive accuracy while significantly reducing computation time relative to HE-only. Additionally, it outperforms DP-only methods under equivalent or stricter privacy budgets.

cs.CR↗

arXiv2026

Memory-Induced Tool-Drift in LLM Agents

LLM agents increasingly combine long-term memory for personalization with tool calling for taking actions. We study a failure of this combination: personality-driven preferences stored in memory (e.g., cost-consciousness, impatience, risk tolerance) silently shifting tool-call parameters in contexts where they do not apply. We call this memory-induced tool-drift and stress-test it with MEMDRIFT, an iteratively refined benchmark of 105 scenarios spanning five bias dimensions and seven professional domains. Measured as excess parameter degradation over a neutral-memory baseline, biased memories in MEMDRIFT induce drift in all eight models we evaluate, raising degradation rates to 0.38--0.83 versus 0.05--0.12 for neutral memories. The drift persists under three widely used memory frameworks. A vulnerability audit of 6,062 tools across 288 MCP servers demonstrates that real-world tools are also susceptible to this failure mode. Mechanistically, we find that biased memories act as implicit steering vectors along the same direction as explicit behavioral instructions and draw attention away from task-relevant context. Neither prompt-based defenses nor relevance filters offer a complete solution, motivating defenses at the intersection of memory management and tool calling.

cs.CR↗