subject

cs.SE

cs.SE: explore 370 source-linked works published from 2025 to 2026, with original documents and citations.

This collection is a preview while coverage and quality are evaluated.

Coverage and selection

Includes records with this source-supplied label or an explicit phrase match in their metadata. Matches indicate a mention, not proof that a paper uses a method or tests a material. Source versions are consolidated by DOI.

Sources: arxiv. Collection updated 2026-09-15. Counts describe this index, not the complete source archives.

arXiv2026

You can contribute if you... An Empirical Framework of AI Contribution Policies in OSS

Artificial intelligence is reshaping open source software (OSS) contribution by lowering the cost of producing code, documentation, issue reports, and review interactions. This creates opportunities for broader participation, but also disrupts how maintainers assess contributor effort, competence, and accountability. In response, OSS projects are beginning to regulate AI-mediated contribution through contribution guidelines and other project documentation. This paper presents an empirical study of these emerging policies. We analyze project policies on AI-mediated contributions by evaluating their underlying rationales, rules, and expectations. Our analysis shows that these policies seek to protect scarce maintainer attention, preserve accountability, sustain meaningful review interactions, address legal and quality concerns, and maintain pathways for newcomer learning. Based on these findings, we introduce the AI Contribution Governance Framework, which organizes recurring concerns and governance mechanisms across projects. The framework helps OSS communities develop AI contribution policies and provides researchers with a vocabulary for studying how AI is changing collaborative software production.

cs.SE↗

arXiv2026

Service Health Engineering for Distributed Systems

Distributed systems support many critical business workflows, but service health is often judged through component dashboards rather than through end-to-end user outcomes. This article presents service health engineering as a practical reliability discipline that connects telemetry, workflow completion, dependency behavior, operational readiness, and recovery validation. Using a document approval workflow as a running example, it describes how service promises, service-level indicators and objectives, watchdogs, incident measures, resiliency testing, and weekly service-health reviews can reveal silent failures and stranded asynchronous work. It also presents a human-reviewed, AI-assisted reporting architecture for assembling service-health evidence without making AI an autonomous decision-maker. The approach brings established reliability practices together around whether user journeys complete as promised.

cs.SE↗

arXiv2026

VEX-Bench: Benchmarking LLM Agents for Assessing Exploitability of Software Supply Chain Vulnerabilities

The software supply chain has become an increasingly exposed attack surface because of its reliance on intricate yet fragile dependencies. Existing defenses such as GitHub Dependabot often raise many false alerts because their coarse-grained matching cannot determine whether a vulnerable dependency is actually exploitable. Security analysts typically spend substantial time assessing vulnerability exploitability case by case. Recent LLM agents have emerged as promising candidates for this task given their advanced capabilities in coding and cybersecurity, yet no existing benchmark evaluates them on it. Prior benchmarks target zero-day settings, where agents detect and exploit previously unknown vulnerabilities. In contrast, software supply chain security focuses on how known vulnerabilities in upstream dependencies affect downstream projects. This requires agents to reason across repositories and determine whether an upstream vulnerability is exploitable in the downstream project. To address this gap, we introduce VEX-Bench, the first benchmark for evaluating LLM agents' ability to assess the exploitability of software supply chain vulnerabilities. It contains 75 real-world cases mined from GitHub and labeled by security experts, covering Python, Java, and Go. We evaluate nine models across three agent harnesses. While GPT-5.5 and Claude Opus 4.6 reach approximately 80% F1 on binary vulnerability-status classification, only GPT-5.5 surpasses 70% macro-F1 on fine-grained justification classification. This gap highlights the challenge of moving beyond binary exploitability assessment to identifying fine-grained exploitability reasons. Code and data: https://github.com/steven1518/vex-bench

cs.CR↗

arXiv2026

Scores Alone Do Not Prove Discovery: The Discovery Certification Protocol for Auditing AI Research Agents

AI research agents combine prior knowledge, public sources, and experimental feedback to produce useful results. The Discovery Certification Protocol (DCP) turns claims about these results into executable recovery and feedback tests. Gate 1 validates useful improvement on sealed evaluation. Gate 2 gives matched agents the registered starting information and observed Web content while withholding the target research history. Every valid method reaching the numerical target supplies a recovery witness and triggers the Core veto. DCP Core requires adequate controls, zero observed recoveries, and a finite-sample bound on recovery in one fresh registered episode. Optional Gate 3 measures the average effect of truthful feedback relative to a specified neutral policy from a shared checkpoint. DCP Evidence adds this effect after independent null calibration and a registered effect margin. Two controlled audits exercise the complete protocol in SQLite optimization and virtual catalyst control under different models. Each produced zero recoveries in 96 episodes, with an upper bound of 0.0468. Each paired study yielded 30 truthful recoveries and zero neutral recoveries, with passing 60-pair null studies. Additional cases exercise Core, recovered, and audit-incomplete decisions. A deterministic, LLM-free verifier reproduces the decisions from frozen evidence. DCP provides a common evidence language for useful outcomes, alternative routes, and feedback effects across AI research.

cs.MA↗

arXiv2026

Cost-Aware Post-Hoc Deferral Under Calibration and Shift: An Environmental AI Case Study

Choosing a deferral policy for a frozen classifier requires more than ranking uncertain cases: confidence may be miscalibrated, errors have unequal costs, reviewers can err, and deployment data can leave calibration support. We study these interactions through EcoTrust, a post-hoc framework that compares automatic action with review using a six-group error-risk estimator, class-asymmetric costs, reviewer accuracy, and an optional support gate. On a Columbia River thermal-stress testbed, the learned estimator improves error-ranking area under the receiver operating characteristic curve from 0.869 to 0.889, but Chow's confidence rule has lower in-distribution cost (0.416 versus 0.567 per day). Across 12 off-the-shelf backends, learned risk and a calibration-matched, class-aware confidence estimator each beat raw Chow on six; a paired year-block bootstrap does not resolve their mean cost difference. In transfer to ten river stations, the gate flags every case and becomes an always-review fallback, attaining the lowest cost on eight stations only when review is perfect and unconstrained. These results characterize decision boundaries on one controlled task: richer risk signals do not reliably improve on calibrated confidence, and detected extrapolation does not imply transferable case-level ranking.

cs.SE↗

arXiv2026

DREAMS: Modelling Support for Research into Engineering and Artistic Design

Design Research Methodology (DRM) supports systematic design research through representations such as Reference Models and Impact Models. However, the practical construction and maintenance of these models often remains manual, requiring repeated redrawing, layout adjustment, and separate handling of assumptions, references, and supporting evidence. This can make DRM modelling time-consuming, visually cluttered, and difficult to revise as models increase in complexity. This paper presents DREAMS, an early-stage prototype modelling environment developed to support the creation and maintenance of DRM Reference Models and Impact Models. The tool enables users to construct typed causal models using DRM-relevant elements, define signed causal relationships, and attach assumptions, experiential inputs, and references directly to causal links. It also provides layout support and search functions to improve readability, modifiability, and retrieval of supporting information. A preliminary comparative evaluation with four DRM users was conducted against manual modelling practice. The results indicate reductions in model creation time, revision time, repositioning effort, edge crossings, and evidence retrieval time when using DREAMS. These findings are interpreted as early evidence of practical potential rather than full validation. The contribution of the paper lies in identifying requirements for DRM-aligned modelling support, presenting the design and implementation of DREAMS, and demonstrating its potential to reduce modelling effort and improve traceability in DRM-based research.

cs.SE↗

arXiv2026

Agentic Electronic Design Automation: A Handoff Perspective

Electronic design automation (EDA) is multi-stage and handoff-heavy, relying on transfers among humans, design artifacts, and multiple tools. LLM-based agents now participate in these transfers, yet the resulting research landscape is highly fragmented and lacks a unified perspective. This survey adopts the primary input--output pair as its organizing lens: for each agent system, we analyze its primary, supporting, and intermediate handoff objects and classify it according to the provenance of its primary input and the consumer boundary of its primary output. Intra-stage systems operate on objects within a single EDA stage, inter-stage systems transform EDA-native artifacts into forms usable by downstream stages, and extra-stage systems translate human intent into EDA artifacts. Based on this taxonomy, we survey 115 representative systems and examine them along multiple dimensions, including research trends, benchmarks, and model mechanisms. Finally, we outline an agentic EDA protocol roadmap and discuss open problems for future research.

cs.SE↗

arXiv2026

CUADebug: Diagnosing and Repairing Computer-Use Agent Failures

Computer-use agents (CUAs) interact with graphical interfaces through screenshots and low-level mouse and keyboard actions, yet the causal error may precede the terminal failure. We present CUADebug, a framework for localizing root causes in CUA trajectories and guiding re-execution. CUADebug includes a five-category, 30-subtype taxonomy; CUAErrorBench, a benchmark of 204 failed OSWorld trajectories with human root-cause annotations; and CUADebugger, a ReAct-style agent for root-cause analysis (RCA). CUADebugger iteratively selects trajectory steps, inspects paired before/after screenshots and action traces, and submits a structured diagnosis containing the causal step, taxonomy label, grounded evidence, and correction. CUADebugger performs RCA without per-trajectory human intervention; human annotations are used to evaluate RCA predictions and, in controlled re-rollout comparisons, to fix restart points. Task reasoning and control is the largest annotated failure category (110/204). CUADebugger improves L2 and Tag+Step Exact across three debugger backbones on the Claude-agent split; with Gemini 2.5 Pro, Tag+Step Exact rises from 11.1% to 19.4%. Single re-execution improves failure recovery from 13.89% to 29.86% (overall 61.77% to 68.14%); controlled continual re-execution improves it from 12.50% to 25.69% (overall 61.22% to 66.48%). Project page: cuadebug.github.io.

cs.SE↗

arXiv2026

From Reading Code to Reading Spec: A Verified Layer for LLM-Driven Codebase Maintenance

The rapid growth of LLM-generated code increases software complexity and the maintenance burden on engineers. While LLMs offer a potential automated alternative, this structural complexity hinders their ability to manage codebases directly. We introduce the Provable Representation Of Original Functionality (PROOF), which manages codebases indirectly via structured specifications. To enable full-lifecycle codebase management strictly through these specifications, PROOF abstracts codebase topology into a hierarchical natural-language representation. To establish absolute trust, the system proves semantic equivalence by reconstructing source code exclusively from this specification. This verified foundation drives maintenance requests, executing code modifications while synchronously updating itself to prevent semantic drift. Experiments on real-world repositories confirm the effectiveness of these specifications.

cs.SE↗

arXiv2026

A Unified Policy Architecture (UPA): The Governance Kernel for Enterprise AI Operating Systems

Enterprise AI is evolving into an Enterprise Operating System where autonomous AI agents can plan, reason, use memory, invoke tools, execute workflows, and collaborate with other agents. This shift creates a new governance challenge: existing authorization, security, guardrails, and compliance mechanisms are fragmented and are not designed to govern autonomous AI as a unified system. This paper introduces the Unified Policy Architecture (UPA), a governance architecture for Enterprise AI Operating Systems. UPA provides a unified policy model for governing AI and agents, tools, workflows, memory, enterprise resources, and agent-to-agent interactions and enterprise business rules. It extends policy control beyond authorisation to include runtime obligations, human approvals, compliance, audit evidence, and governance evaluation. We present UPA's governance model, declarative policy language foundations, policy evaluation semantics, extensible plugins, industry policy packs, and an evaluation framework for enterprise governance. We also identify extensions for multi-agent coordination, provenance-aware policies, and stateful runtime governance. UPA provides a foundation for building secure, accountable, and governable Enterprise Operating Systems for autonomous AI.

cs.AI↗

arXiv2026

An Autonomy Aware Metamodel for Human AI Collaboration in Software Engineering

Artificial Intelligence (AI) is shifting software engineering from tool-supported processes towards AI-first collaboration, where authority is dynamically distributed across human and artificial actors. However, existing method engineering approaches assume static, human-centric control and provide limited support explicitly capturing evolving autonomy. This paper presents a vision for autonomy-aware method engineering by proposing a metamodel that treats autonomy not as a fixed property of an actor, but as a derived, situation-dependent authority assignment determined by task, context, and collaboration pattern. The metamodel formalizes autonomy through four authority dimensions: task execution, task decomposition, task initiation, and collaboration reconfiguration. Through an analytical instantiation with a multi-agent requirements analysis tool, we illustrate how the metamodel supports dynamic authority assignment. This work provides a conceptual foundation for governance-aware, adaptable, and AI-first software engineering methods.

cs.SE↗

arXiv2026

CAROL: Context-Aware Online Learning for Fuzzer Scheduling

Ensemble fuzzing runs multiple fuzzers on a target while a scheduler allocates CPU time among them. Existing schedulers base these decisions on compact summaries of past performance and rules fixed before a campaign. Our measurements reveal two limitations. First, past-reward summaries do not reliably capture performance evolution: after accounting for estimation noise, agreement between consecutive-window rankings is statistically indistinguishable from within-window self-agreement. Second, predictive signals vary across targets: on eight of nine targets, a weighting learned from the other eight predicts reward worse than one learned on the current target. We introduce CAROL, an online scheduler that uses each fuzzer's current context. Already available to the dispatch loop, this context describes reward trends, waiting and plateau time, reached code, and estimation uncertainty. CAROL uses context in two ways: a domain-guided method detects whether a fuzzer is rising or rotting and applies a phase-specific learning rule, while a learned method predicts reward from 15 context signals and uses predictive uncertainty for online selection. Across nine Magma targets, CAROL triggers more unique bugs than each of three ensemble-scheduling baselines whenever their results differ, and fewer on none. Compared with the strongest baseline for each target, CAROL gains 11.8% and surpasses an oracle that retrospectively selects the best single fuzzer per target. Removing context eliminates the gain, and the additional bugs are concentrated among those the baselines trigger rarely or never. Run unchanged on five widely used C++ programs, CAROL finds 120 previously unknown crashing defects, deduplicated by site, fault, and entry point; all were reported to maintainers through the projects' stated disclosure channels.

cs.SE↗

arXiv2026

Shortcutting the Fix: Identifying and Categorizing Agentic Exploits in Software Engineering Benchmarks

While autonomous software engineering (SWE) agents achieve high benchmark resolution rates, these scores can mask exploitative behaviors---such as leveraging local Git histories, accessing upstream repositories, or recalling memorized solutions---rather than demonstrating genuine problem solving. We systematize and audit these exploits across five open large language models on SWE-bench Multilingual and DeepSWE using a turn-level LLM-as-a-judge protocol. Under standard prompts, exploitation rates reach 45.1\%--82.4\% on SWE-bench Multilingual and 44.2\%--66.1\% on DeepSWE. Appending a targeted instruction enforcing solution originality drastically cuts these exploitation rates---down to 4.0\%--10.7\% and 1.5\%--7.1\%, respectively, while maintaining strong core task performance. Our findings demonstrate the critical need for exploit-aware evaluation frameworks that measure true repository-level problem solving over benchmark gaming.

cs.SE↗

arXiv2026

AURA-Eval: Evaluation Framework for Acting Under Risk Awareness in LLM Agent Trajectories

LLM agents operate in workflows where unsafe actions can have real consequences. Existing safety evaluations often reduce behavior to a single score, obscuring risk recognition, pre-action detection, and safe task completion when a safe solution exists. We introduce AURA-Eval, a framework combining controlled augmentation with granular diagnosis of behavior in tool-use trajectories. Its pipeline identifies safety-critical decision points, generates controlled variations, and constructs counterparts differing in whether a request has a safe fulfillment path. Using 157 sourced trajectories, we generate 1,249 evaluation items and evaluate 20 frontier and open-weight models. We developed rubrics to classify risk detection, action strategy, and scenario-specific action safety. Our results show that LLM agents engage in unsafe behavior more often when no safe fulfillment path exists. In these cases, frontier proprietary models more often recognize risk and exhibit safer behavior by proposing alternatives, while evaluated open-weight models more often directly execute unsafe requests. Increasing impact or reducing opportunities for oversight before execution also exposes greater vulnerability across models.

cs.CR↗

arXiv2026

The Double Measurement Confound in Agent Benchmarks: De-Scaffolding, Ground-Truth Scoring, and Reliability Beyond the Mean

Agent benchmarks are increasingly used to compare large language models (LLMs) and guide deployment decisions, yet benchmark scores are meaningful only if they measure model capability rather than properties of the evaluation pipeline. We identify a double measurement confound: execution-critical decisions are performed by a fixed scaffold instead of the model, while the scorer evaluates outputs using criteria that may not reflect task correctness. We unify these issues within a measurement-theoretic framework that characterizes when benchmark scores can be interpreted as evidence of model capability, and instantiate it with an audit-and-repair protocol that (i) transfers execution-critical decisions from the scaffold to the model, (ii) replaces shape-based evaluation with seeded ground-truth scoring, and (iii) reports reliability beyond the mean through worst-case and tail-risk metrics. Experiments on ComtradeBench show that the joint intervention transforms a nearly flat leaderboard into a reliability spectrum that distinguishes both average performance and robustness across seeds. Applying the audit to existing benchmarks further shows that scorer validity is benchmark-specific, whereas scaffold ownership is an uncontrolled axis wherever we probed it. Our results suggest that benchmark scores should be interpreted together with their scaffolding level, scoring criterion, and reliability profile, providing a practical framework for more valid evaluation of LLM agents.

cs.SE↗

arXiv2026

The Flaw of Averages: Measuring Benchmark-Level Distributional Robustness

Benchmarks are central to measuring progress in language models, but aggregate scores can obscure substantial variation across subdomains, making models appear broadly competent despite concentrated strengths and weaknesses. We study this issue as benchmark-level distributional robustness: whether aggregate scores faithfully reflect performance across benchmark subdomains. We operationalize this notion with benchmark Harmony, an entropy-based measure of how uniformly model performance is distributed across subdomains. Measuring Harmony on 19 language model benchmarks across five model families, we find substantial variation in benchmark-level distributional robustness. Low-Harmony benchmarks are more likely to yield aggregate scores that overstate broad competence, whereas high-Harmony benchmarks provide more representative summaries of model capability. Rebalancing benchmarks by pruning overrepresented subdomains to increase Harmony substantially shifts aggregate scores for low-Harmony benchmarks, but leaves high-Harmony benchmarks comparatively stable. For example, while BoolQ remains comparatively stable as Harmony increases, PubMedQA, which evaluates performance in a medically consequential domain, exhibits substantial, often statistically significant, shifts in aggregate accuracy. Together, these findings show that aggregate scores can misrepresent broad competence when performance is unevenly distributed. We therefore recommend reporting benchmark Harmony alongside aggregate accuracy as a diagnostic of benchmark representativeness when interpreting claims about broad model competence.

cs.CL↗

arXiv2026

Detecting Multiple Semantic Concerns in Tangled Code Commits

Code commits in a version control system (e.g., Git) should be atomic, i.e., focused on a single goal, such as adding a feature or fixing a bug. In practice, however, developers often bundle multiple concerns into tangled commits, obscuring intent and complicating maintenance. Recent studies have used Conventional Commits Specification (CCS) and Language Models (LMs) to capture commit intent, demonstrating that Small Language Models (SLMs) can approach the performance of Large Language Models (LLMs) while maintaining efficiency and privacy within local infrastructure. However, they do not address tangled commits involving multiple concerns, leaving the feasibility of using LMs for multi-concern detection unresolved. In this paper, we frame multi-concern detection in tangled commits as a multi-label classification problem and construct a controlled dataset of artificially tangled commits based on real-world data. We then present an empirical study using SLMs to detect multiple semantic concerns in tangled commits, examining the effects of fine-tuning, concern count, commit-message inclusion, and header-preserving truncation under practical token-budget limits. Our results show that a fine-tuned 27B-parameter SLM outperforms a state-of-the-art LLM across all concern counts. In particular, including commit messages improves detection accuracy by up to 31% (in terms of Hamming Loss) with negligible latency overhead, establishing them as important semantic cues.

cs.SE↗

arXiv2026

FIKA: Expanding Dependency Reachability with Executability Guarantees

Automated third-party library analysis tools help developers by addressing key dependency management challenges, such as automating version updates, detecting vulnerabilities, and detecting breaking updates. Dependency reachability analysis aims at improving the precision of dependency management, by reducing the space of dependency issues to the ones that actually matter. Most tools for dependency reachability analysis are static and fundamentally limited by the absence of execution. In this paper, we propose FIKA, a pipeline for providing guarantees of executability for third-party library call sites. FIKA leverages a Large Language Model (LLM) guided by static analysis to generate code that is executed, and whose execution trace provides guarantees that a third-party library call site is actually executable. We apply our approach to a dataset of eight Java projects to empirically evaluate the effectiveness of FIKA. On average, 54% of these call sites are covered by the existing test suites, and therefore, have evidence for their executability. FIKA improves this coverage by 20%, generating successful reachability scenarios for 609 out of 1465 previously non-covered call sites. In six out of eight projects, FIKA provides strong guarantees that more than 75% of call sites are executable. We further demonstrate that FIKA improves over Semgrep, a state-of-the-art static vulnerability reachability analysis tool, and outperforms Jazzer, a coverage-guided directed fuzzer. We show that FIKA can help prioritize the vulnerability updates with stronger guarantees of executability.

cs.SE↗

Compare source metadata on this page

Work	Published	Source identifier	Source
You can contribute if you... An Empirical Framework of AI Contribution Policies in OSS	2026-09-07	2609.07919	arxiv
Service Health Engineering for Distributed Systems	2026-09-07	2609.08020	arxiv
VEX-Bench: Benchmarking LLM Agents for Assessing Exploitability of Software Supply Chain Vulnerabilities	2026-09-07	2609.08040	arxiv
Scores Alone Do Not Prove Discovery: The Discovery Certification Protocol for Auditing AI Research Agents	2026-09-07	2609.09219	arxiv
Cost-Aware Post-Hoc Deferral Under Calibration and Shift: An Environmental AI Case Study	2026-09-07	2609.09235	arxiv
DREAMS: Modelling Support for Research into Engineering and Artistic Design	2026-09-06	2605.10382	arxiv
Agentic Electronic Design Automation: A Handoff Perspective	2026-09-06	2606.19795	arxiv
CUADebug: Diagnosing and Repairing Computer-Use Agent Failures	2026-09-06	2608.02643	arxiv
From Reading Code to Reading Spec: A Verified Layer for LLM-Driven Codebase Maintenance	2026-09-06	2609.06383	arxiv
A Unified Policy Architecture (UPA): The Governance Kernel for Enterprise AI Operating Systems	2026-09-06	2609.06543	arxiv
An Autonomy Aware Metamodel for Human AI Collaboration in Software Engineering	2026-09-06	2609.06720	arxiv
CAROL: Context-Aware Online Learning for Fuzzer Scheduling	2026-09-06	2609.06772	arxiv
Shortcutting the Fix: Identifying and Categorizing Agentic Exploits in Software Engineering Benchmarks	2026-09-06	2609.06780	arxiv
AURA-Eval: Evaluation Framework for Acting Under Risk Awareness in LLM Agent Trajectories	2026-09-06	2609.06783	arxiv
The Double Measurement Confound in Agent Benchmarks: De-Scaffolding, Ground-Truth Scoring, and Reliability Beyond the Mean	2026-09-06	2609.09218	arxiv
The Flaw of Averages: Measuring Benchmark-Level Distributional Robustness	2026-09-05	2509.25671	arxiv
Detecting Multiple Semantic Concerns in Tangled Code Commits	2026-09-05	2601.21298	arxiv
FIKA: Expanding Dependency Reachability with Executability Guarantees	2026-09-05	2604.20015	arxiv

These are bibliographic comparisons, not experimental rankings. Follow the original document for methods and conditions.