arXiv Science⌕ Search
Home/Archive/arXiv

arXiv · 2610.27694

Enhancing Bug-Exposing Test Generation via Step-Level Credit Assignment and Self-Distillation from Valuable Failures

Wenjie Gan·Zhaoxi Zhang·Jiahui Liang·Weikang Li·Wanyuan Wang·Li Zhang·Jian Dong
View original source ↗Read source document ↗Export metadata ↓

Abstract

Large Language Models have shown strong code generation capabilities, yet their outputs often contain subtle bugs. Automatically repairing such code requires informative tests: inputs that expose faulty behavior, paired with correct expected outputs. Recent RL-based test generation methods apply the same outcome reward to all reasoning steps, which can discourage useful reasoning and reinforce flawed steps. They also fail to effectively learn from trajectories that find bug-triggering inputs but predict incorrect outputs, leaving their verified bug-triggering progress insufficiently exploited. To address these limitations, we propose SCALF, a process-supervised RL approach that combines Step-level Credit Assignment with targeted Learning from valuable Failed trajectories. SCALF derives step-specific advantages from process rewards, allowing each reasoning stage to receive its own learning signal. It further routes trajectories that expose bugs but predict incorrect outputs to on-policy self-distillation, using execution feedback to learn from their valuable bug-triggering progress. Across three benchmarks, SCALF outperforms strong prompting and post-training baselines by up to 2.97, 5.26, and 3.53 percentage points in joint success rate, output correctness rate, and bug-triggering rate, respectively. Our code and datasets are available at https://anonymous.4open.science/r/SCALF-F4D1.

Explore related subjects

cs.SE

Keep this discovery

Explore connections, maps & timelines

BibTeXRIS

Wenjie Gan, Zhaoxi Zhang, Jiahui Liang, Weikang Li, Wanyuan Wang, Li Zhang, Jian Dong. 2026-10-20. Enhancing Bug-Exposing Test Generation via Step-Level Credit Assignment and Self-Distillation from Valuable Failures. https://arxiv.org/abs/2610.27694

Cite the original work for its findings. Save a collection to share your selection of sources.

KEEP EXPLORING

Related papers

arXiv2026

Mitigating Risks in Software Development through Effective Requirements Engineering

Problems that originate in requirements are common and expensive in software projects, and they are a recurring cause of late delivery, budget overruns and cancellation. This paper is a narrative overview of requirements engineering (RE) as a means of reducing these risks. It outlines the core RE activities (elicitation, analysis and negotiation, specification, validation and change management) and the kinds of statements a requirements document contains, including the distinction between requirements, domain properties, assumptions and definitions. It reviews the evidence on the cost of requirements defects, together with the main criticisms of that evidence, and compares how the waterfall, spiral, iterative (Rational Unified Process) and agile lifecycles treat requirements. It then discusses stakeholder identification, elicitation techniques and common obstacles to elicitation, and describes practical ways of turning vague or conflicting stakeholder statements, including security and performance expectations, into precise, verifiable and prioritized requirements. The paper is intended as an introduction for practitioners and students who work with requirements but are not RE specialists.

cs.SE↗
arXiv2026

WAFFLE: Finetuning Multi-Modal Models for Automated Front-End Development

Web development involves turning UI designs into functional webpages, which can be difficult for both beginners and experienced developers due to the complexity of HTML's hierarchical structures and styles. While Large Language Models (LLMs) have shown promise in generating source code, two major challenges persist in UI-to-HTML code generation: (1) effectively representing HTML's hierarchical structure for LLMs, and (2) bridging the gap between the visual nature of UI designs and the text-based format of HTML code. To tackle these challenges, we introduce Waffle, a new fine-tuning strategy that uses a structure-aware attention mechanism to improve LLMs' understanding of HTML's structure and a contrastive fine-tuning approach to align LLMs' understanding of UI images and HTML code. Models fine-tuned with Waffle show up to 9.00 pp (percentage point) higher HTML match, 0.0982 higher CW-SSIM, 32.99 higher CLIP, and 27.12 pp higher LLEM on our new benchmark WebSight-Test and an existing benchmark Design2Code, outperforming current fine-tuning methods.

cs.SE↗
arXiv2026

Can Language Models Replace Programmers for Coding? REPOCOD Says 'Not Yet'

Recently, a number of repository-level code generation benchmarks-such as CoderEval, DevEval, RepoEval, RepoBench, and LongCodeArena-have emerged to evaluate the capabilities of large language models (LLMs) beyond standalone benchmarks like HumanEval and MBPP. Thus, a natural question is, would LLMs have similar performance in real world coding tasks as their performance in these benchmarks? Unfortunately, one cannot answer this question, since these benchmarks consist of short completions, synthetic examples, or focus on limited scale repositories, failing to represent real-world coding tasks. To address these challenges, we create REPOCOD, a Python code-generation benchmark containing complex tasks with realistic dependencies in real-world large projects and appropriate metrics for evaluating source code. It includes 980 whole-function generation tasks from 11 popular projects, 50.8% of which require repository-level context. REPOCOD includes 314 developer-written test cases per instance for better evaluation. We evaluate ten LLMs on REPOCOD and find that none achieves more than 30% pass@1 on REPOCOD, indicating the necessity of building stronger LLMs that can help developers in real-world software development. In addition, we found that retrieval-augmented generation achieves better results than using target function dependencies as context.

cs.SE↗