arXiv Science⌕ Search

arXiv · 2610.25629

Selecting and Composing Agent Design Patterns: Industrial Case Evidence and Controlled Reference-System Experiments

Abstract

Agent pattern catalogs describe architectural alternatives, but engineers also need evidence for selecting and composing them. We extend a cognitive-function--execution-topology framework with an evidence-driven selection procedure and explicit composition contracts. The study combines two industrial case accounts, a contributor's collaboration design, seven practitioner workshops, and executable fault-injection experiments. Three local reference systems use SQLite state, shared artifact files, and a file-backed map-publication surrogate. A 2 x 2 ablation separates function-specific contracts from execution controls across 36 scenario families, with ten input variants per family. Under equal family weighting, composition correctly completes or safely refuses 11 of 12 families in each system, compared with 2, 2, and 1 for the simple baselines. These are constructed challenge-suite results, not production success rates. Recovery without consumer-side verification increases false-success cases in the map surrogate. All composed systems still fail a boundary case in which trusted evidence is wrong. An audit of a frozen six-arm language-model pilot reproduces its scores but confirms an evaluator-dependent ranking reversal. The contribution connects industrial failure, pattern selection, executable treatment, external acceptance, and revision. Selection is demonstrated rather than evaluated against an alternative procedure; catalog admission rests on separate criteria. The results support bounded composition mechanisms, not a claim that the framework improves engineers' design decisions over an equally informative conventional method.

Explore related subjects

Keep this discovery

Explore connections, maps & timelines

BibTeXRIS

Jia Huang, Bingsheng Ru, Bo Liang, Yuke Xiong, Haiyan Fan, Joey Tianyi Zhou. 2026-10-19. Selecting and Composing Agent Design Patterns: Industrial Case Evidence and Controlled Reference-System Experiments. https://arxiv.org/abs/2610.25629

Cite the original work for its findings. Save a collection to share your selection of sources.

KEEP EXPLORING

Related papers

Real Faults in Deep Learning Fault Benchmarks: How Real Are They?

As the adoption of Deep Learning (DL) systems continues to rise, an increasing number of approaches are being proposed to test these systems, localise faults within them, and repair those faults. The best attestation of effectiveness for such techniques is an evaluation that showcases their capability to detect, localise and fix real faults. To facilitate these evaluations, the research community has collected multiple benchmarks of real faults in DL systems. In this work, we perform a manual analysis of 490 faults from five different benchmarks and identify that 305 of them are eligible for our study. Our investigation focuses specifically on how well the bugs correspond to the sources they were extracted from, which fault types are represented, and whether the bugs are reproducible. Our findings indicate that only 15.4% of the faults satisfy our realism conditions. Our attempts to reproduce these faults were successful only in 54% of cases.

cs.SE↗

LLM-as-a-Judge for Contest-Style Human-AI Co-Creation: A Reliability-Aware Evaluation Framework for Coding

Large language models now judge open-ended outputs and co-create code with programmers, yet judgments of human-AI co-creation must be reliable, comparable across models, and interpretable over multi-turn interaction. This work presents a rubric-driven LLM-as-a-Judge framework for contest-style human-AI co-creation in coding, evaluated on 15 participants, 13 problems, 184 trajectories, and 517 attempts from an AI-permitted track run in parallel with a programming contest. Four judges are evaluated under grouped cross-validation with cluster-bootstrap 95% intervals and probed by injecting single defects into accepted programs against behavior-preserving controls. Three findings concern the judges as instruments and likely extend beyond this setting. First, the judge input is a first-order design variable: supplying the problem statement in place of an identifier rebuilds the judges' ranking (rank correlation 0.2617 [0.123, 0.391] for the three judges whose model was unchanged) and raises their ROC-AUC by 0.130 [0.061, 0.202] to 0.295 [0.214, 0.381]. Second, schema compliance does not imply scale validity: the judges emit as few as 8 distinct probability values, leaving as few as 6 of 99 thresholds effective. Third, accuracy and defect sensitivity are separable: Claude and Gemini are indistinguishable on accuracy (ROC-AUC 0.6985 [0.633, 0.763] against 0.7691 [0.719, 0.817]) yet separate on defect detection (0.8380 [0.806, 0.871] against 0.7512 [0.719, 0.783]). For co-creation, Success@Turn rises from 0.6033 [0.533, 0.674] at the first attempt to 0.8370 [0.783, 0.886] by the tenth, 44 of the 73 trajectories unsolved at first (60.3%) succeed later, and revision size is unrelated to judged improvement (Spearman 0.0007 [-0.085, +0.090]).

cs.SE↗

Step-Aligned Script Generation with Verification and Recovery for Web Automation Testing

Automated generation of end-to-end web tests remains costly: developers must identify stable selectors, encode multi-step workflows, attach assertions, and maintain scripts as interfaces evolve. Recent LLM-based web agents can complete natural-language tasks in a live browser session, but task completion is not the same as producing a test-ware artifact - code that is executable, replay-able from a reset state, inspectable by developers, and tied to an oracle. We present TESSARA, an adaptive test-script generation framework that converts natural-language test intent into Playwright scripts consisting of executable interaction code, goal-state assertions, and step-level provenance linking each step to its generated code fragment, selector evidence, execution outcome, and repair history. TESSARA grounds each planned step via Step-Aligned Scripting (SAS), enabling fine-grained failure localization, selective regeneration of failing suffixes, and reversibility-aware backtracking that avoids undoing likely commit points (e.g., form submissions). A three-level verification pipeline (syntactic, execution, and semantic/oracle), detects and repairs malformed code, interaction failures, and incorrect landing states. We evaluate TESSARA on three public web benchmarks, measuring script executability, oracle satisfaction, replay stability, and repair efficiency, and show substantial improvements over baselines and underlying VLMs used directly as script generators.

cs.SE↗