arXiv Science⌕ Search

arXiv · 2611.21626

X-LocVul: An Evidence-Propagation Audit of Cascaded C/C++ Vulnerability Detection, Localization, and Explanation

Abstract

Developer-facing vulnerability analysis should link a function-level warning to a source location and a supported explanation. Cascaded systems can propagate weak evidence despite plausible components. We present X-LocVul, a measurement-first audit of a modular C/C++ detector-localizer-explainer pipeline combining UniXCoder, CodeT5, and Qwen2.5-Coder. We test whether evidence passed across stages is useful, merely source-resident, or misleading. Under duplication and temporal-leakage controls, UniXCoder has higher point estimates than CodeBERT, but true-CWE supervision is statistically indistinguishable from a shuffled-CWE control on PR-AUC, MCC, and vulnerable-class F1. Raw generated statements fail exact source matching in 90.83% of cases. Fuzzy projection reaches 83.79% Top-1 accuracy and exceeds semantic projection by 1.59 percentage points, although its advantage over raw ranking is not established. In a randomized, condition-label-masked two-rater evaluation of 50 paired cases, a predicted line improves evidence support, whereas a predicted CWE improves no endpoint. Oracle evidence improves evidence support (Holm-adjusted p=0.00453) and CWE consistency (p=3.49 x 10^-7), but not root-cause quality after correction (p=0.250); unsupported claims remain common. A chronological stress test retains ranking signal (ROC-AUC 0.782; PR-AUC 0.110), while the transferred threshold yields zero recall and F1. The contribution is not a state-of-the-art claim, but an auditable account of evidence construction, transfer, and consumption. It separates source residency from localization correctness, auxiliary labels from semantic supervision, and predicted evidence from useful evidence. Public E4 scores regenerate all 15 tests; missing manifests, final prompts, and profiler logs bound stronger reproducibility claims.

Explore related subjects

Keep this discovery

Explore connections, maps & timelines

BibTeXRIS

Hoang-Phuong-Linh Le, Quyet-Thang Huynh. 2026-09-13. X-LocVul: An Evidence-Propagation Audit of Cascaded C/C++ Vulnerability Detection, Localization, and Explanation. https://arxiv.science/abs/2611.21626

Citations retain the original authors and DOI and include this site’s record URL. If this discovery service contributed to your research, you can also cite arXiv Science by Indiciae: BibTeX · RIS.

KEEP EXPLORING

Related papers

AgentLens: Revealing The Lucky Pass Problem in SWE-Agent Evaluation

Evaluation of software engineering (SWE) agents is dominated by a binary signal: whether the final patch passes the tests. This outcome-only view treats a principled solution and a chaotic trial-and-error process as equivalent. We show that this equivalence is empirically false. We evaluate 2,614 OpenHands trajectories from eight model backends on 60 SWE-bench Verified tasks. Of these, 47 have enough passing trajectories to construct task-level process references, yielding a 1,815-trajectory evaluation subset. Among passing trajectories in this subset, 10.7% exhibit behavior we call a Lucky Pass: regression cycles, blind retries, missing verification, or temporally disordered exploration, implementation, and verification. We introduce AgentLens, a framework for process-level assessment of SWE-agent trajectories, and release AgentLens-Bench, a dataset of 1,815 trajectories annotated with quality scores, waste signals, divergence points, and 47 task-level Prefix Tree Acceptor (PTA) references. AgentLens builds PTA references by merging multiple passing solutions for the same task, and uses a context-sensitive intent labeler to assign actions to Exploration, Implementation, Verification, or Orchestration based on trajectory history rather than tool identity alone. On AgentLens-Bench, the quality score separates passing trajectories into Lucky, Solid, and Ideal tiers and further decomposes Lucky Passes into five recurring mechanisms. Across the eight model backends, Lucky rates range from 0.5% to 23.2%, and some models move by as many as five rank positions when ranked by quality score instead of pass rate. We publicly release AgentLens-Bench and the AgentLens SDK at https://github.com/agentlens-framework/agentlens, with a project page at https://agentlens-framework.github.io/agentlens/.

cs.SE↗

Where Is the Cost of Third-Party API Routers in Agentic Software Development?

Third-party API routers have become a common layer that unifies access across increasingly diverse LLM providers. In coding-agent workflows, high-autonomy operation is widely adopted because it reduces interaction overhead. As a result, a third-party API router, which sits between the agent and the upstream provider, inevitably occupies the trusted path. It can inspect and modify every request and response, yet no mechanism verifies alignment between the provider's output and the repository-level actions ultimately executed by the agent. Consequently, client-side permission mechanisms may become ineffective in practice. Whether this control gap produces real, hard-to-detect effects on software development tasks remains empirically unmeasured. In this paper, we conduct an empirical study of router-side injection in coding agents, examining four intervention levels of increasing subtlety: Response Substitution (L1), Response Append (L2), LLM-Polished Injection (L3), and LLM-Polished with Distribution Alignment Injection (L4). Moreover, we develop SIDEL, a framework for trace recording, replay, injection, and defense evaluation, with a curated dataset of 400 samples. We evaluate four representative coding agents, and further evaluate whitelist-based execution control and LLM review. Router-side intervention substantially alters repository-level actions and remains difficult for existing client-side safeguards to detect. Without additional mitigations, all evaluated agents achieved a defense success rate of 0 percent across all injection levels. Client-side mitigations and reactive reviews improve resistance but do not fully restore end-to-end control, motivating provider-side output-integrity guarantees. Our code is available at https://github.com/Riyasushin/SIDEL.

cs.SE↗

A Grey Literature Review of Graduate Students' Research Practices and Skills in the LLM Era

Context: Graduate students in computing now use Large Language Models (LLMs) throughout their research, from literature reviews and coding to data analysis and writing. Yet there is little shared understanding of which skills competent use of these tools requires, so graduate programs struggle to teach, mentor, or assess it, and unverified outputs can propagate into the scientific record. Objective: We investigate which LLM-based practices graduate students employ in their research and which skills they need to benefit from these practices. Method: We conducted a grey literature review of Reddit discussions. We retrieved 1,383 posts from five research-oriented subreddits, screened them against explicit inclusion and exclusion criteria, and analyzed the 54 included posts with thematic analysis, in which two authors independently coded each excerpt. Results: We identified seven categories of practices and nine categories of skills. Commenters locate the problem in accepting generated output without inspection rather than in using LLMs, and warn that delegating the effortful parts of a task, such as debugging, reading, or validating sources, delivers the output without building the skill the task was meant to develop. Traditional skills, such as programming and critical thinking, are what allow students to verify what an LLM produces. Conclusions: Our categories give graduate programs a concrete basis to teach students to work with LLMs without eroding the skills that research training should build, and they add research-specific skills, such as data sharing and scientific reproducibility, to existing AI competency frameworks.

cs.SE↗