arXiv subjects

Stjepan Picek

Publications and source records attributed to Stjepan Picek.

3 recordsLinked to original sources

arXiv2026

SIR: Self-improving Red-teaming for Compute Use Agents

Computer use agents (CUAs) are vision-language models that perceive a screen and act on a real operating system through mouse, keyboard, and terminal, and they are increasingly deployed to automate everyday digital tasks. Because they can be exposed to untrusted content while operating, they are vulnerable to indirect prompt injection (IPI), in which an adversary plants instructions in content the agent will read and redirects it toward actions that violate the user's intent. Existing CUA safety benchmarks evaluate fixed injections written by hand, which may underestimate the risk posed by an adaptive adversary. We present SIR, a black box IPI attack that (i) composes stealthy injections from a small library of reusable principles stated in plain language and (ii) wraps composition in an iterative feedback loop that diagnoses the victim's failed trajectories and distills the bypasses into new, named strategies that are reapplied across tasks. Unlike prior red teaming of web agents, we target CUAs at the operating system level and score attacks with a fully deterministic oracle, using checks on filesystem, service, and permission state rather than an LLM judge. On experiment, we evaluate three frontier CUAs. Composing principles with feedback raises the attack success rate over a baseline written by hand, for example from 4% to 24% on Claude Opus 4.8 and from 0% to 28% on Gemini 3.5 Flash, while the benign task still completes. Principles discovered against one model further transfer to a different architecture with no additional feedback.

cs.CR↗

arXiv2026

BadPatches: Routing-Aware Backdoor Attacks on Vision Mixture-of-Experts

Mixture-of-Experts (MoE) architectures have gained significant traction for reducing computational costs in deep neural networks by activating only a sparse subset of parameters during inference. While this efficiency makes MoE highly attractive for scaling vision tasks, its patch-based processing mechanism inherently disrupts traditional, routing-agnostic backdoor attacks by fragmenting or discarding adversarial triggers. To expose the vulnerabilities of this architecture, we introduce BadPatches, a novel routing-aware trigger application strategy specifically designed for patch-based MoE (pMoE) models and MoE-based vision transformers. Rather than applying a global pattern across the entire image, BadPatches encapsulates triggers within targeted image patches, ensuring they are consistently routed to and processed by the active experts. Our evaluations demonstrate that BadPatches achieves a high Attack Success Rate (ASR) at lower poisoning rates than routing-agnostic triggers, reaching over 83.2% ASR with a poisoning rate of only 0.01%, and scaling to a 96.8% ASR at 0.05%, while preserving the model's clean accuracy. Furthermore, the attack remains effective in gray-box scenarios where the adversary lacks complete knowledge of the model's patch routing configuration. Finally, we evaluate fine-pruning as a potential defense mechanism, revealing that pruning alone is insufficient to mitigate the attack; successful backdoor removal strictly requires the fine-tuning stage. These findings highlight the fragility of sparse vision architectures and underscore the need for routing-aware defenses.

cs.CR↗

arXiv2026

WoE Wrote It? Watermarking Mixture-of-Experts LLMs for Black-Box Text Provenance

Large Language Model (LLM) watermarks provide a mechanism for text provenance, enabling model owners to identify machine-generated content and attribute it to a specific watermarked model. However, current LLM watermarking approaches predominantly rely on inference-time sampler methods and focus their analysis on dense models. Inference-time methods are only effective when the text is explicitly generated via the model owner's controlled API; they fail in a post-compromise scenario. An adversary who steals or leaks the model weights gains complete control over inference and can simply run an unmodified sampler, bypassing the watermark and preventing post-theft attribution. In this work, we introduce Watermarking of Experts (WoE), a novel black-box text provenance method that leverages the unique structural properties of sparse Mixture-of-Experts (MoE) models. WoE biases the vocabulary of specific experts and shifts the watermark signal embedding away from unenforceable inference wrappers. This approach ensures the watermark remains intrinsic to the model parameters, enabling defenders to attribute text generated by stolen weights, leaked checkpoints, and secondary dense models distilled from the stolen architecture without needing access to the adversary's deployment or weights. We evaluate WoE across eight MoE models, demonstrating successful watermark detection from suspect text, achieving an average true positive rate of 90.1% at a 1% false positive rate, reaching up to 94.9%, while largely preserving general model utility. Furthermore, WoE remains detectable under adversarial supervised fine-tuning, model extraction, and output-level paraphrasing, forcing malicious actors into a trade-off in which weakening the attribution signal requires additional model adaptation or text-rewriting operations, or compromises the utility of the resulting output.

cs.CR↗