arXiv Science⌕ Search

arXiv · 2609.38224

Separation of Duties for Privileged LLM Agents: A Governed Execution Architecture with Measured Security-Utility Trade-offs

Abstract

Large language model agents are increasingly granted real privileges (executing commands, modifying files, calling APIs), so an agent that errs has already acted. Existing defences concentrate on the agent's inputs, while the path from a candidate action to privileged side effects remains less directly studied. We argue that this path must be governed outside the model, and study an architecture interposing four roles (planner, policy gate, executor, auditor) between agent and operating system. Two choices are central: actions arrive as structured intents, so adjudication never parses shell syntax; and approval is a one-shot credential bound to the exact bytes that will run. We evaluate on a 313-case benchmark across eight variants, with prompt-only baselines from three hosted LLMs on 150 stratified cases, five repetitions (2,250 attempted calls; 2,249 completed). Effective attack success falls from 98.3% under direct execution to 7.7% deployed. Re-execution against the real implementation yields a similar aggregate rate (7.6% over 66 sandbox-evaluable payloads) but substantial case-level disagreement, at a corrected false-denial rate of 11.1%. A substantial final-stage reduction (from 30.8% to 7.7%) is attributable to the operating-system sandbox, and the benchmark found four implementation defects, none by design review.

Explore related subjects

Keep this discovery

Explore connections, maps & timelines

BibTeXRIS

Qishuai Jing. 2026-09-28. Separation of Duties for Privileged LLM Agents: A Governed Execution Architecture with Measured Security-Utility Trade-offs. https://arxiv.org/abs/2609.38224

Cite the original work for its findings. Save a collection to share your selection of sources.

KEEP EXPLORING

Related papers

Speed Kills: Exploring Confused Deputy Attacks Through Edge AI Accelerators

AI Accelerators (AIAs) are specialized hardware, such as Tensor Processing Units (TPUs), that enable optimal and efficient execution of AI applications and on-device inference. The growing demand for AI applications has led to the widespread adoption of AIAs on edge and embedded devices. Unlike applications, AIAs are not bound by Operating System (OS) restrictions and have limited visibility into Application Processor (AP) security mechanisms (e.g., kernel versus application memory and process isolation). This semantic gap can lead to confused deputy vulnerabilities, where an AIA can be tricked by a malicious application into performing privileged operations on its behalf. In this paper, we conduct the first in-depth study of Confused Deputy Attacks (CDAs) using AIAs. We design DeputyHunt, a Large Language Model (LLM)-assisted framework to extract CDA-relevant information for a given AIA through a combination of dynamic and static analysis. We use this information to explore the feasibility of CDAs on seven different AIAs from popular vendors, including Google, NVIDIA, Hailo, Texas Instruments, NXP, AWS, and Rockchip. Our analysis reveals that CDAs are feasible on six out of the seven AIAs, impacting over 128 System-on-Chips (SoCs) and over 100 million devices. Our findings highlight critical security risks posed by AIAs to system security. Our work has been acknowledged by the corresponding vendors and assigned CVE-2025-66425. We propose an on-demand validation defense against CDAs, and our evaluation on the Gem5-salam simulator shows that the average runtime overhead of our mechanism is significantly lower (~15% versus ~56%) than that of traditional defenses.

cs.CR↗

Do Defenses Against LLM Extraction Work Across Attacks? A Lifecycle Benchmark of Black-Box Model Extraction

Large language models (LLMs) deployed through text-only APIs face model extraction risks, as adversaries can collect their responses to train surrogates that reproduce their capabilities. While prior work has developed diverse attacks and defenses, evaluations remain fragmented across access assumptions, model configurations, query budgets, and security objectives, limiting comparability across methods. To address this gap, we introduce a unified benchmark covering six extraction attacks, ten defenses, and two adaptive attacks that paraphrase or back-translate protected responses before surrogate training. The benchmark controls model configurations, query data, budgets, and held-out evaluation conditions within each comparison while preserving attack-specific querying and training procedures. We measure surrogate capability, fidelity to the victim, output quality using Rep-4, and query-budget sensitivity; defenses use their own security metrics paired with surrogate performance. For the adaptive attacks, we jointly measure provenance-detector scores and the capability and fidelity of surrogates trained on rewritten responses. The benchmark thus provides a reproducible basis for comparing extraction methods and their interactions with defenses under text-only access. Code and artifacts are available at https://github.com/sliu11-byte/MEA-Bench.

cs.CR↗

SERA-IDS: Structured Experience Retrieval-Augmented Intrusion Detection with Small Language Models

Large language models (LLMs) offer a flexible approach to network intrusion detection, but direct classification of numerical flow records can be unreliable without traffic- specific decision boundaries. Retrieval-augmented generation (RAG) enables LLMs to leverage external experience; however, free-form textual experience requires the model to infer the numerical conditions that distinguish traffic classes. We propose SERA-IDS, a Structured Experience Retrieval-Augmented Intru- sion Detection framework that learns decision knowledge from classification errors. A Small Language Model (SLM) analyzes misclassified flows and generates structured rules containing behavioral descriptions, feature conditions, class-confusion infor- mation, tool-derived confidence, and provenance. A confidence gate admits supported rules into the Experience Library. During testing, the library is frozen, and retrieved class-diverse rules are condition-matched to the query flow and provided as evidence to the decision SLM. The same library is evaluated with three small, locally deployable SLMs, namely Llama 3.1, Phi-4:14B, and Qwen2.5:7B, without fine-tuning or paid hosted inference. On the test sets, SERA-IDS improves macro F1 on NF-BoT- IoT from 11.66%, 9.26%, and 7.83% to 86.46%, 69.50%, and 83.51%, respectively, and on NF-ToN-IoT from 3.82%, 6.17%, and 1.94% to 84.16%, 87.66%, and 83.37%. These results show that structured experience retrieval enables small, free, and locally deployable SLMs to achieve strong intrusion-detection performance, including performance exceeding that of the re- ported existing work used in our comparison.

cs.CR↗