arXiv Science⌕ Search

arXiv · 2610.38039

Reusable LLM-Based Plans with Probabilistic Guarantees for Self-Adaptive Systems

Abstract

Self-adaptive systems are designed to accommodate variants of their environment, which form a family of instances whose uncertainty is captured by probabilistic models, and must meet probabilistic requirements on each instance. Within such a family, changes in size and structure complicate the fundamental adaptation decision: whether to retain, reuse, or construct a new controller, since controllers are often synthesized for and specific to an instance. Large language models (LLMs) have recently been used to write reusable controllers, but existing approaches largely target deterministic domains, and probabilistic model checking of their output remains limited to preliminary single-instance studies. In this paper, we propose a verifier-guided framework for controller reuse with explicit support for families of environments. The framework introduces a novel combination of LLM-written controllers with probabilistic model checking to provide per-instance guarantees for probabilistic requirements. We evaluate our approach using three LLMs on three benchmarks, assessing its effectiveness and cost with respect to baseline implementations, the distribution of adaptation decisions, and the impact of individual framework components. On grid worlds, LLM-written rule lists are verified on about 90% of held-out instances, against at most 77% for the baselines, and verifying a stored rule list is 15 to 103 times faster than per-instance synthesis. Reuse serves most episodes at a small fraction of the cost of generation or per-instance synthesis, and model-checking feedback increases verified coverage over independent restarts.

Explore related subjects

Keep this discovery

Explore connections, maps & timelines

BibTeXRIS

Asher Tempest Arya, Rajan Maghera, Nicholas Wu Jiang, Federico Mora, Aren A. Babikian, Marsha Chechik. 2026-10-27. Reusable LLM-Based Plans with Probabilistic Guarantees for Self-Adaptive Systems. https://arxiv.org/abs/2610.38039

Cite the original work for its findings. Save a collection to share your selection of sources.

KEEP EXPLORING

Related papers

Understanding Developer Information Needs for Software with Embedded Natural Language Prompts

Developers use foundation models (FMs) to create novel forms of intelligent software by writing natural language prompts embedded within a software's implementation---referred to as embedded prompts---that are also passed as a string within an FM's API. Developing software with embedded prompts requires developers to engage in iterative embedded prompt changes to obtain the desired behavior. While the information needs of traditional software have been investigated, the information needs of modern developers for embedded prompt changes are underexplored, despite these needs shaping how developers plan and justify embedded prompt changes. In this work, we perform a mixed-method exploratory study on the information needs of developers making changes to embedded prompts. We create, refine, validate, and apply a list of 50 information needs that developers have while making a change to an embedded prompt by interviewing 16 developers, observing 8 developers make prompt changes, surveying 50 developers, and evaluating 72 research and commercial tools. We find that embedded prompts are not well-supported by existing tools: 16 of the 50 information needs, including four of the most important ones, are unaddressed. We conclude with important design opportunities that should be addressed by future developer tools for embedded prompts.

cs.SE↗

Enhancing Neural Code Representation with Additional Context

Automated program comprehension underpins many software engineering tasks, from code summarisation to clone detection. While recent deep learning models have achieved strong results, they typically rely on source code alone, overlooking contextual signals such as version history or structural relationships. We conduct an empirical study of how enriching code representations with such signals affects model performance on four tasks: code clone detection, code classification, code summarisation, and vulnerability detection, using three datasets (SeSaMe, CodeSearchNet-Java, Vul4J). We evaluate five foundational code models (CodeBERT, GraphCodeBERT, CodeT5, PLBART, ASTNN) and seven code-specialised LLMs (CodeLlama, Qwen2.5-Coder, DeepSeek-Coder), comparing prompt-based context injection against fine-tuned use of context. Context improves classification for all five foundational models and, after fine-tuning, for most LLMs. For summarisation, the effect depends on the model family: version history improves all four foundational models on every metric, whereas prompt-level context helps two of the seven LLMs and hurts the other five. Combining context types reaches up to +21.48% macro-F1 in code classification and +15.04% F1 in clone detection. A human evaluation of CodeT5 summaries on 100 Java snippets ranks context-augmented summaries significantly higher for Accuracy and Content Adequacy. An exploratory vulnerability detection evaluation provides initial evidence of transfer to security-oriented tasks.

cs.SE↗

Cloud-OpsBench: A Reproducible Benchmark for Agentic Root Cause Analysis in Cloud Systems

LLM agents are explored for automating root cause analysis (RCA) in cloud-native systems, creating a need to evaluate both diagnostic correctness and the quality of the supporting investigation. Existing static benchmarks offer repeatable inputs but limited system-facing interaction, while live testbeds expose tools but hinder controlled comparison because incident evidence varies across runs; both paradigms focus primarily on final answers. To address these limitations, we present Cloud-OpsBench, an evaluation infrastructure for interactive and evidence-grounded cloud RCA. It comprises 754 runtime-verified cases across 57 fault types on two microservice workloads spanning application services and Kubernetes platform layers. Each fault is captured as a state snapshot and replayed through standard diagnostic interfaces, with outcome labels and diagnostic evidence graphs that enable matched comparisons and process-level analysis. Evaluating eleven LLMs on Cloud-OpsBench reveals that strong outcome accuracy often masks incomplete investigations: while Joint RCA Accuracy (JRA) reaches up to 0.78 on OnlineBoutique and 0.68 on TrainTicket, models identify the correct root cause without achieving evidence closure (e.g., in 33.1% and 29.4% of evaluated cases for GPT-5). These results demonstrate that final-answer accuracy alone cannot establish whether an agent has traced the failure and collected the evidence needed to confirm its root cause.

cs.SE↗