arXiv Science⌕ Search
Home/Archive/arXiv

arXiv · 2610.12893

Vulnerable Models in the AI Supply Chain: An Empirical Study of Adoption, Propagation, and Usage Patterns

The Anh Nguyen·Triet Huynh Minh Le·M. Ali Babar
View original source ↗Read source document ↗Export metadata ↓

Abstract

The rapid integration of Artificial Intelligence (AI) models has established a complex supply chain, but many of these shared artifacts harbor severe vulnerabilities such as insecure serialization and architectural backdoors. While prior efforts have extensively scanned model registries to flag malicious artifacts, a critical gap remains in understanding how developers actually adopt and integrate these unsafe models into downstream software systems. We bridge this gap by conducting a large-scale empirical study on the usage and propagation of 16,977 known vulnerable models on HuggingFace flagged by InsightDB. We track their prevalence across GitHub repositories, software packages, fine-tuning chains, and deployed applications. Our analysis reveals that, despite being publicly flagged, these vulnerable models appear in over 21,000 code locations across more than 5,000 open-source repositories, with 3,757 distinct flagged models (22.1\%) actively referenced in at least one project. To understand how these models are integrated in code and identify recurring usage patterns, we categorize and analyze the recurring patterns surrounding them via a customized pipeline that combines Code Property Graphs extraction with a tailored Frequent Subgraph Mining approach. Across all patterns, we find no evidence of provenance verification or model sanitization logic, with developers frequently bypassing security parameters or delegating trust to permissive framework defaults. These findings underscore that scanning model files alone is insufficient, highlighting the urgent need for context-aware security analysis and stricter integration guardrails at the application layer of the AI supply chain.

Explore related subjects

cs.CRcs.SE

Keep this discovery

Explore connections, maps & timelines

BibTeXRIS

The Anh Nguyen, Triet Huynh Minh Le, M. Ali Babar. 2026-10-09. Vulnerable Models in the AI Supply Chain: An Empirical Study of Adoption, Propagation, and Usage Patterns. https://arxiv.org/abs/2610.12893

Cite the original work for its findings. Save a collection to share your selection of sources.

KEEP EXPLORING

Related papers

arXiv2026

Mitigating Data Reconstruction Attacks in Federated Learning: Conditions, Bounds, and Theoretical Guarantees

Privacy attacks, particularly data reconstruction attacks, pose significant challenges to federated learning despite its promise in privacy-preserving collaborative training. This paper presents a comprehensive theoretical analysis of privacy leakage in federated learning through dual perspectives: linear algebra and optimization theory. By analyzing the client-update Jacobian, we identify conditions under which distinct data batches produce the same model update and derive a sufficient batch-size criterion for non-unique reconstruction. Through optimization theory, we establish a novel upper bound for privacy leakage that incorporates batch size, distortion extent, and other critical parameters. These theoretical insights elucidate the fundamental relationships between system parameters and privacy vulnerability, laying the groundwork for developing more robust privacy-preserving federated learning frameworks.

cs.CR↗
arXiv2026

A Privacy-Preserving Federated Learning Method for Heterogeneous Clients

Federated Learning (FL) with Differential Privacy (DP) can ensure the protection of healthcare data privacy against malicious user attacks. However, FL with the DP method faces an inherent trade-off: stronger privacy protection degrades predictive accuracy due to injected noise. On the other hand, Homomorphic Encryption (HE) enables computations on encrypted data and the aggregation of encrypted gradients without introducing DP-induced noise, thereby potentially increasing the predictive accuracy. However, it may increase the computation cost. To improve the predictive accuracy while considering the computational ability of heterogeneous clients, we propose a Privacy-Preserving Machine Learning (PPML)-Hybrid method. In the proposed PPML-Hybrid method, clients distributedly select either HE or DP based on their computational resources, so that HE clients contribute noise-free updates while DP clients reduce computational overhead. Meanwhile, clients with high computational resources can flexibly adopt HE or DP according to their privacy needs. Performance evaluation on omics datasets shows that our proposed method achieves comparable predictive accuracy while significantly reducing computation time relative to HE-only. Additionally, it outperforms DP-only methods under equivalent or stricter privacy budgets.

cs.CR↗
arXiv2026

Memory-Induced Tool-Drift in LLM Agents

LLM agents increasingly combine long-term memory for personalization with tool calling for taking actions. We study a failure of this combination: personality-driven preferences stored in memory (e.g., cost-consciousness, impatience, risk tolerance) silently shifting tool-call parameters in contexts where they do not apply. We call this memory-induced tool-drift and stress-test it with MEMDRIFT, an iteratively refined benchmark of 105 scenarios spanning five bias dimensions and seven professional domains. Measured as excess parameter degradation over a neutral-memory baseline, biased memories in MEMDRIFT induce drift in all eight models we evaluate, raising degradation rates to 0.38--0.83 versus 0.05--0.12 for neutral memories. The drift persists under three widely used memory frameworks. A vulnerability audit of 6,062 tools across 288 MCP servers demonstrates that real-world tools are also susceptible to this failure mode. Mechanistically, we find that biased memories act as implicit steering vectors along the same direction as explicit behavioral instructions and draw attention away from task-relevant context. Neither prompt-based defenses nor relevance filters offer a complete solution, motivating defenses at the intersection of memory management and tool calling.

cs.CR↗