Recording-Aware Evaluation of Multimodal OT Intrusion Detection

Mustafa Umut Ozbek; Khalil El-Khatib; Li Yang

Home/Archive/arXiv

arXiv · 2610.24128

Recording-Aware Evaluation of Multimodal OT Intrusion Detection

Mustafa Umut Ozbek·Khalil El-Khatib·Li Yang

View original source ↗Read source document ↗Export metadata ↓

Abstract

Machine-learning-based intrusion detection systems (IDSs) for operational technology (OT) are commonly evaluated on benchmark datasets in which normal and attack data are collected in separate recordings. In such settings, high performance may reflect recording-associated differences as well as attack behavior. We introduce a recording-aware evaluation procedure that compares attack windows with background from the same recording, uses prior data for model fitting and calibration, tests temporal alignment against randomized event schedules, and reports event coverage together with alarm burden. On the ProvICS multimodal dataset, cyber modalities strongly distinguish the benign and attack recordings before the first labeled attack. Same-recording evaluation reverses the apparent modality ranking across all studied window sizes: the physical view becomes the strongest individual modality, while cyber-only fusion shows the largest difference between cross-recording and same-recording AUC, reaching 0.54. Operational analysis further shows that permissive event-overlap metrics can overstate detection when a small number of long alarm episodes cover multiple actions. An external analysis on HAI shows smaller and reference-dependent recording sensitivity. These findings show that cross-recording benchmark performance should be interpreted alongside same-recording, forward, temporal, and operational evaluations.

Explore related subjects

cs.CR cs.LG

Keep this discovery

Explore connections, maps & timelines

BibTeX RIS

Mustafa Umut Ozbek, Khalil El-Khatib, Li Yang. 2026-10-16. Recording-Aware Evaluation of Multimodal OT Intrusion Detection. https://arxiv.org/abs/2610.24128

Cite the original work for its findings. Save a collection to share your selection of sources.

KEEP EXPLORING

Related papers

arXiv2026

Mitigating Data Reconstruction Attacks in Federated Learning: Conditions, Bounds, and Theoretical Guarantees

Privacy attacks, particularly data reconstruction attacks, pose significant challenges to federated learning despite its promise in privacy-preserving collaborative training. This paper presents a comprehensive theoretical analysis of privacy leakage in federated learning through dual perspectives: linear algebra and optimization theory. By analyzing the client-update Jacobian, we identify conditions under which distinct data batches produce the same model update and derive a sufficient batch-size criterion for non-unique reconstruction. Through optimization theory, we establish a novel upper bound for privacy leakage that incorporates batch size, distortion extent, and other critical parameters. These theoretical insights elucidate the fundamental relationships between system parameters and privacy vulnerability, laying the groundwork for developing more robust privacy-preserving federated learning frameworks.

cs.CR↗

arXiv2026

A Privacy-Preserving Federated Learning Method for Heterogeneous Clients

Federated Learning (FL) with Differential Privacy (DP) can ensure the protection of healthcare data privacy against malicious user attacks. However, FL with the DP method faces an inherent trade-off: stronger privacy protection degrades predictive accuracy due to injected noise. On the other hand, Homomorphic Encryption (HE) enables computations on encrypted data and the aggregation of encrypted gradients without introducing DP-induced noise, thereby potentially increasing the predictive accuracy. However, it may increase the computation cost. To improve the predictive accuracy while considering the computational ability of heterogeneous clients, we propose a Privacy-Preserving Machine Learning (PPML)-Hybrid method. In the proposed PPML-Hybrid method, clients distributedly select either HE or DP based on their computational resources, so that HE clients contribute noise-free updates while DP clients reduce computational overhead. Meanwhile, clients with high computational resources can flexibly adopt HE or DP according to their privacy needs. Performance evaluation on omics datasets shows that our proposed method achieves comparable predictive accuracy while significantly reducing computation time relative to HE-only. Additionally, it outperforms DP-only methods under equivalent or stricter privacy budgets.

cs.CR↗

arXiv2026

Memory-Induced Tool-Drift in LLM Agents

LLM agents increasingly combine long-term memory for personalization with tool calling for taking actions. We study a failure of this combination: personality-driven preferences stored in memory (e.g., cost-consciousness, impatience, risk tolerance) silently shifting tool-call parameters in contexts where they do not apply. We call this memory-induced tool-drift and stress-test it with MEMDRIFT, an iteratively refined benchmark of 105 scenarios spanning five bias dimensions and seven professional domains. Measured as excess parameter degradation over a neutral-memory baseline, biased memories in MEMDRIFT induce drift in all eight models we evaluate, raising degradation rates to 0.38--0.83 versus 0.05--0.12 for neutral memories. The drift persists under three widely used memory frameworks. A vulnerability audit of 6,062 tools across 288 MCP servers demonstrates that real-world tools are also susceptible to this failure mode. Mechanistically, we find that biased memories act as implicit steering vectors along the same direction as explicit behavioral instructions and draw attention away from task-relevant context. Neither prompt-based defenses nor relevance filters offer a complete solution, motivating defenses at the intersection of memory management and tool calling.

cs.CR↗