arXiv subjects

Zibin Zheng

Publications and source records attributed to Zibin Zheng.

At least 19 recordsLinked to original sources

arXiv2026

DSL-Bench: Evaluating and Explaining LLM Code Generation Across Domain-Specific Languages

Large language models (LLMs) are increasingly used in software-engineering workflows, but code-generation benchmarks largely focus on general-purpose languages. Domain-specific languages (DSLs) have distinct syntax, semantics, and validation requirements, making results from individual DSL benchmarks difficult to compare. We present DSL-Bench, an executable benchmark and analysis framework for cross-DSL code generation. Starting from 13,045 DSL-related papers, LLM-assisted open card sorting identifies 18 representative application domains. For each domain, we select an evaluable DSL and provide a syntax checker and task-level semantic oracle. Evaluating 7 LLMs on 18 DSLs, we observe mean syntax and semantic pass rates of 31.37% and 11.64%, respectively, revealing a substantial gap between syntax acceptance and semantic correctness. A variance decomposition attributes 76.99% of semantic pass-rate variation to differences among DSLs and 7.86% to differences among models. Among four DSL-level factors, Knowledge Demand has the strongest negative association with semantic correctness (r = -0.508). We evaluate prompt engineering across all 18 DSLs and study five improvement methods on five targeted DSLs. Prompt engineering yields uneven gains; RAG-Refinement, the strongest targeted method, improves semantic correctness by 16.29 percentage points on average over matched baselines. We further derive capability profiles and DSL-Profiler summaries for interpreting data-scarce DSLs. These results highlight toolchain-specific semantic bottlenecks and the need to evaluate DSL code generation beyond syntax acceptance.

cs.SE↗

arXiv2026

EvoGroup: A Self-Evolving Harness for Group Agents in Shared Contexts

With the rise of Group Agent such as Claude Tag, agents increasingly serve as digital workers for long-term team collaboration. These agents operate over shared contexts that continuously accumulate through multi-user interactions. However, existing agent memory systems are mainly designed for individual users and struggle to capture group-level knowledge in shared contexts, including group relationships and collaboratively formed knowledge. Manual memory strategies further limit adaptation to diverse and evolving shared contexts. To this end, we propose \textbf{EvoGroup (EvoG)}, a self-evolving harness for group agents that starts from a minimal six-tool cold start and adapts through interaction with shared context. EvoG uses confidence and task outcomes to select informative trajectories, while also eliciting self-reflections on uncertainty in low-confidence cases. The analysis agent then examines selected trajectories through adaptive progressive disclosure, followed by bucketed analysis to synthesize high-level findings that guide the evolve agent in revising the Harness. Empirically, pass@1 on held-out split of EverMemBench improves from 71.07\% to 89.58\%, surpassing both self-evolving memory and human-designed harnesses, including Claude Code (85.83\%). To evaluate generalization, we freeze the evolved harness and evaluate it on the unseen GroupMemBench, where, with GPT-5.5, it demonstrates effective generalization and outperforms both memory baselines and human-designed harnesses.

cs.CL↗

arXiv2026

Who Swept My ETH? Uncovering EIP-7702-Based Ether Redirection Attacks on Ethereum

EIP-7702 introduces a new transaction type that allows Externally Owned Accounts (EOAs) on major EVM-compatible chains to delegate execution to arbitrary contract codes temporarily. While enhancing functionality, this mechanism facilitates a novel threat: the \underline{E}IP-7702-Based \underline{E}ther \underline{R}edirection \underline{A}ttack (EERA). In such an attack, a victim is delegated to a malicious sweeper contract, which automatically redirects incoming ETH transfers to an address controlled by the attacker. As a nascent attack paradigm arising from protocol evolution, the underlying mechanisms and security implications of EERA remain systematically unexplored. To bridge this gap, we provide the first formal analysis of the EERA attack model and propose SweeperHunter, a framework that detects these attacks through transaction analysis. By evaluating transactions from the first ten months following the introduction of EIP-7702 on Ethereum mainnet, we have identified 4,668,209 malicious delegations, 20,443 attack transactions and 508 ETH sweeper contracts, forming the first EERA dataset. Finally, we analyze this attack from both attacker and victim perspectives, derive key insights, and propose corresponding mitigation strategies. We reveal the prevalence of EERA, as approximately 70\% of the observed delegations point toward ETH sweeper contracts. Furthermore, we observe a notable lack of effective responses from victims; specifically, over 600,000 accounts failed to revoke or update their malicious delegations promptly, thereby remaining in a state of vulnerability. Our research is expected to facilitate understanding and mitigation of this emerging threat posed by EIP-7702.

cs.CR↗

arXiv2026

Rimuru: Unified Memory Management for Modern LLM Serving via Elastic Expert Offloading

Modern LLMs strain device memory through large model weights and attention states, particularly in resource-constrained scenarios. Existing systems separately provision memory for model weights and attention states, imposing fixed fixed boundaries that cannot adapt to the changing memory demands of workloads. Under heavy workloads, insufficient state capacity limits request concurrency and reduces throughput; under light workloads, idle memory in one region cannot be reallocated to another, wasting device memory. Meanwhile, modern LLM architectures introduce heterogeneous tensors with diverse layouts and lifetimes, further complicating memory management. These problems motivate unified memory management for heterogeneous tensors. However, a shared pool alone does not make memory directly reusable across tensors with different allocation granularities and execution layouts. This paper presents Rimuru, a modern LLM serving system that unifies memory management through elastic expert offloading. Rimuru establishes a shared page pool to enables elastic memory sharing across heterogeneous tensors. To accommodate different allocation granularities, Rimuru introduces memory views that allow each tensor type to interpret the shared memory space using its own page size and page table. Furthermore, Rimuru combines huge-page alignment with a multi-level page allocator that splits and merges pages, enabling cross-type capacity reuse. A page-table-aware fused MoE kernel directly accesses non-contiguous expert weights, avoiding copy overhead. Building on these mechanisms, Rimuru employs a workload-aware adjuster that dynamically reallocates pages across tensor types according to runtime demands. Implemented on top of vLLM, Rimuru remains compatible with modern LLMs. Evaluations show that Rimuru achieves up to 19.03x baseline throughput and reduces TTFT by 98.1% on average.

cs.OS↗

arXiv2026

From Conversation to Evaluation: Benchmarking LLMs on Development Knowledge via SimpleDevQA

The Development Knowledge Question Answering (Dev Knowledge QA) task aims to provide accurate natural language answers to knowledge-seeking questions during software development. To investigate the importance of Dev Knowledge QA in AI-assisted software development and the extent to which it has been explored, we conduct a preliminary analysis of real user-LLM dialogues from WildChat. Our findings indicate that Dev Knowledge QA plays a significant role in real-world software development scenarios, and these raw dialogues cannot be directly used to construct a Dev Knowledge QA benchmark. Existing Dev Knowledge QA benchmarks are limited in development knowledge scope and often not built from real user queries. To bridge this gap, we design a three-phase pipeline that transforms real-world dialogue into simple development knowledge-seeking QA pairs. Through this pipeline, we introduce SimpleDevQA, a multilingual Dev Knowledge QA benchmark inspired by real user dialogues. This dataset covers three languages (English, Chinese, and Russian), and focuses on questions with unique, short, and verifiable answers, making evaluation more accurate and simple. Extensive experiments with 21 mainstream LLMs show that closed-source models generally perform best on SimpleDevQA. We also find that RAG-based knowledge injection improves accuracy, and that Dev Knowledge QA performance correlates with both model confidence and code-generation capability. To facilitate the replication study, we have released our data and code at: https://github.com/DeepSoftwareAnalytics/SimpleDevQA.

cs.SE↗

arXiv2026

Bridging Generation and Training: A Systematic Review of Quality Issues in LLMs for Code

Large language models (LLMs) frequently generate defective code, ranging from logical bugs to security vulnerabilities. Training-data quality is one contributor to these outcomes alongside model architecture, optimization and alignment, prompting and inference context, and decoding. However, the literature connecting specific training data quality issues to observable generated code quality issues has not been synthesized systematically. We review 114 included papers and establish taxonomies of eight generated-code quality dimensions and two broad groups of training-data issues. We then synthesize 12 recurring relationship families between training-data issues and generated-code outcomes, tracing each to its supporting papers, evidence basis, and limitations. We also synthesize detection and mitigation techniques across the data, model, and generation lifecycle. The findings position data curation as one layer of code-generation quality assurance coordinated with model- and inference-level controls, and identify provenance and controlled attribution as open research needs. Our repository is available at https://github.com/SYSUSELab/From-Data-to-Code.

cs.SE↗

arXiv2026

HarnessSecurity-Bench: Do Security Mechanisms Really Protect Coding Agent Harnesses?

Coding agent harnesses mediate tool use and authorize actions, yet their security mechanisms and runtime effects remain incompletely characterized. We present HarnessSecurity, the first systematic empirical study and benchmark of open- and closed-source coding agent harnesses. First, we derive a ten-mechanism taxonomy and then assess 400 harness-mechanism cells using independent ratings by researchers and large language model (LLM) judges. We find that about half of confirmed mechanism implementations are opt-in, while closed-source harnesses exhibit substantial evidence gaps. Second, we introduce HarnessSecurity-Bench, a benchmark of 23 tasks across five attack surfaces without sacrificing legitimate task requirements. Using separate deterministic oracles to measure task utility and attack effects with security setting comparisons, we evaluate nine mechanisms across six leading harnesses: Claude Code, Codex CLI, Gemini CLI, gptme, Qwen Code, and GitHub Copilot. Under a controlled LLM baseline GLM-5.2, we conduct 2,500 trials, recording 81,155 tool calls and over 2.2 billion tokens. Enabling auto-approve increases utility and raises attack success from 29.2% to 95.6%. Network isolation and read-only mode reduce attack effects with substantial utility losses, while command allowlisting and command denylisting reduce attack effects with a small utility loss and a utility gain, respectively. Task-level cases show that restrictions on a shared capability can obstruct both legitimate and malicious operations, and that allowed tools or commands can leave unauthorized operations reachable through alternative execution paths. Harness providers should make security settings verifiable, test alternative execution paths to protected operations, and assess attack effects alongside task utility and execution costs.

cs.CR↗

arXiv2026

SkillGATE: Gate-Aware Monte Carlo Tree Search for Skill Retrieval

Skill Retrieval (SR) aims to identify the most relevant skills from external skill libraries, and becomes increasingly challenging as libraries grow in scale and diversity. Existing methods either rank skills independently or rely on predefined graph propagation and hierarchical routing, making them vulnerable to semantic distractors, local trapping, and early routing errors. We formulate SR as an adaptive information-foraging process that coordinates region-level navigation with skill-level selection according to the utility and uncertainty observed during search. Based on this formulation, we propose SkillGATE, a graph-guided hierarchical retrieval framework with Gate-Aware Monte Carlo Tree Search (MCTS). SkillGATE constructs a graph-preserving hierarchical index and performs adaptive retrieval through selection, expansion, simulation, and backpropagation. G-PUCT guides action selection, expansion explores new regions, simulation evaluates candidate skills, and backpropagation updates search statistics. Experiments on six SR benchmarks show that SkillGATE consistently improves diverse retrieval and reranking backbones, achieving a 16.3\% improvement in overall R@1 over the strongest retriever-based baseline. Our code is available at https://github.com/Edwinbe/SkillGATE-v1/.

cs.AI↗

arXiv2026

SmartMemory: Detecting On-chain-off-chain Communication Inconsistency for Smart Contract via Memory-based Agent

Smart contracts underpin decentralized finance, where growing demand for on-chain/off-chain communication(OFC) has driven diverse applications such as cross-chain bridges, real-world asset tokenization, and fiat-backed stablecoins. TheOFC-related security incidents in these applications are increasingly frequent, but prior studies address separate vulnerability categories within OFC applications rather than providing a unified view, causing vulnerabilities outside known patterns to be missed.In this paper, we identify OFC inconsistency (OFCI) as a root cause of OFC vulnerabilities, which arises from business-logic flaw and ultimately breaks the equivalence between the on-chain and off-chain asset representations to induce inconsistency.Automatically detecting OFCIs faces two challenges including (1)locating heterogeneous business logic, and (2) transferring existing vulnerability knowledge to identify unseen OFCI instances. To this end, we propose SmartMemory, the first framework to leverage a memory-based agent for OFCI detection. To address heterogeneity, SmartMemory maps diverse implementations ofOFC contracts into a canonical business-semantic representation to locate the business logic for OFCI inspection. For knowledge reuse, SmartMemory integrates a memory-based agent to distill vulnerability knowledge from features into patterns and detection rules, enabling knowledge transfer across cases to identify unseenOFCIs. Lastly, SmartMemory performs taint analysis to verify the reachability, type, and impact of each candidate OFCI. We construct the first real-world OFCI dataset comprising 48 DApps with 81 OFCIs for evaluation, on which SmartMemory achieves80.68% precision and 87.65% recall. In addition, through an analysis of 325 real-world OFC applications, SmartMemory detects 36 previously unknown OFCIs, all of which have been confirmed and fixed by corresponding parties.

cs.SE↗

arXiv2026

Your Mailbox Is Mine: Prompt Injection Attacks Against Real-World LLM Email Agents

Large Language Model (LLM) email agents have emerged as pivotal autonomous assistants, serving as a critical root of trust for digital identity by managing sensitive communications and authentication workflows. Despite their importance, the prompt injection (PI) resilience of the real-world LLM email agent ecosystem remains poorly understood. Existing assessments largely rely on simulated environments or fragmented production case studies, while existing PI attacks mainly follow an instruction-takeover strategy that directly competes with the system prompt and the user's task. Our evaluation shows that such attacks are insufficient in email-agent settings: existing template-based prompt injection attack baselines achieve Attack Success Rates (ASRs) of only 4.58%-8.13%. In this paper, we propose Email-Specific Prompt Injection (ESPI), a new attack paradigm that manipulates how email agents interpret mailbox operational context. By combining Email Protocol State Masquerade and Camouflage Logic Chain, ESPI reframes attacker-desired mailbox operations as necessary remediation steps under forged email-operational states. We further develop ESPInspector, an automated black-box attack pipeline for analyzing and evaluating real-world email agents. Across 480 controlled attack trials, ESPI achieves 73.54% ASR, substantially outperforming all baselines. Further, ESPI successfully hijacks all 63 evaluated applications across 870 black-box instances, requiring only 1.84-1.89 attempts on average for the first successful hijack. Our responsible disclosure receives formal risk acknowledgment from 22 vendors and the assignment of 16 CVE IDs. Our research provides the first holistic map of security gaps in the real-world email agent ecosystems and highlights the urgent necessity for robust, state-verified security enhancement solutions.

cs.CR↗

arXiv2026

AutoSQL: Extracting SQL Templates from Imperative ORM Code in Large-Scale Repositories

Suboptimal SQL queries can significantly degrade the performance of cloud systems, motivating the extraction and auditing of SQL statements before deployment. However, Go ORM frameworks construct SQL imperatively through scattered method-call sequences, making it difficult to statically recover the resulting SQL templates. We present AutoSQL, a system that reconstructs SQL templates from Go ORM code. AutoSQL constructs a Code Index, a directed graph that captures structural dependencies between functions, types, and global variables as navigable edges. It then traces upstream call chains from ORM invocation sites to identify database-interacting functions as entry points. For each entry point, an LLM agent traverses the Code Index to collect code slices that influence SQL generation, switching to pattern-based search when the graph cannot resolve a retrieval goal. We call this strategy Hybrid Context Retrieval. Once sufficient context is collected, the agent synthesizes SQL templates. Evaluation on a benchmark of 579 test-covered entry points and 1,186 runtime-traced SQL statements from five large-scale Go repositories shows that AutoSQL achieves 68.04% to 72.18% recall, exceeding the static reachability baseline by 11.80% to 15.94% and outperforming existing methods by 8.52% to 21.50%.

cs.SE↗

arXiv2026

BridgeShield: Risk-Aware Graph Modeling for Cross-Chain Bridge Attack Detection

Cross-chain bridges enable asset and state transfers across heterogeneous blockchains, but their complex cross-domain interactions introduce new attack surfaces that are difficult to monitor using traditional single-chain analysis methods. Existing approaches often focus on isolated on-chain behaviors and fail to capture the multi-stage execution semantics of cross-chain transactions. This paper presents BridgeShield, a graph-based framework for detecting cross-chain bridge attacks through risk-aware modeling of cross-chain execution behaviors. BridgeShield reconstructs cross-chain behavior graphs from execution traces and event logs, capturing interactions across the source chain, off-chain relay components, and the destination chain. To highlight attack-relevant structures, the framework employs differential meta-path selection to identify execution patterns that exhibit structural deviations between normal and attack transactions, and hierarchical risk propagation to aggregate distributed risk signals across interaction stages. Experiments on real-world bridge incidents show that BridgeShield achieves an F1-score of 92.6% in cross-chain attack detection and consistently outperforms existing rule-based and graph-based baselines. In addition, the model remains effective in detecting previously unseen attack incidents, indicating that BridgeShield captures structural risk patterns rather than memorizing historical attack templates.

cs.CR↗

arXiv2026

BlockEmulator: An Emulator Enabling to Test Blockchain Sharding Protocols

Numerous blockchain simulators have been proposed to allow researchers to simulate mainstream blockchains. However, we have not yet found a testbed that lets researchers develop and evaluate new consensus algorithms or protocols for blockchain sharding systems. To fill this gap, we developed BlockEmulator as an experimental platform, particularly for emulating blockchain sharding mechanisms. BlockEmulator adopts a lightweight blockchain architecture so developers can focus only on implementing their new protocols or mechanisms. Using BlockEmulator's layered modules and useful programming interfaces, researchers can implement a new protocol with minimal effort. In two steps, we test BlockEmulator's functionality. First, we prove the correctness of BlockEmulator's emulation results by comparing theoretical analysis with observed experimental results. Second, other experiments show that BlockEmulator can measure a range of metrics, including throughput, transaction confirmation latency, cross-shard transaction ratio, the queuing status of transaction pools, workload distribution across blockchain shards, etc. We have made BlockEmulator open-source on GitHub.

cs.CR↗

arXiv2026

Bridging the Opacity: Evidence-Backed Cross-Chain Transaction Correspondence Reconstruction Across Heterogeneous Blockchains

Cross-chain bridges enable interoperability, but they also break the transaction trails needed to trace illicit funds. Third-party investigators typically cannot access the source-to-destination mappings maintained by bridge backends, and our survey of 131 bridges finds that only 16.79% provide complete public tracking. Existing approaches depend on official APIs, EVM-specific assumptions, or fragile temporal heuristics, limiting their ability to trace transfers across heterogeneous ledgers. We present XSplicer, an evidence-driven system for reconstructing cross-chain transaction correspondence (xTCR) without privileged access to bridge backends. XSplicer derives unified semantic specifications from public protocol documentation and transaction examples, translates them into lightweight parsers and verifiers, and links source and destination transactions by prioritizing hard evidence and using soft clues only when necessary. We evaluate XSplicer on seven bridge protocols spanning EVM, Bitcoin, and Solana. XSplicer achieves 92.5% global recovery rate and up to 98.61% on individual protocols. Under adversarial noise, its hard-evidence verifier retains the correct match in 100% of tested cases, while soft-clue matching degrades as ambiguity increases. In two real-world case studies, XSplicer recovers more than 1,900 historical transaction pairs after Multichain ceased operations and identifies 754 illicit cross-chain transfers worth 105.6 million USD in the Bybit laundering incident. These results show that public protocol invariants can support practical cross-chain forensics without privileged bridge mappings.

cs.CR↗

arXiv2026

SkillScope: Toward Fine-Grained Least-Privilege Enforcement for Agent Skills

Agent Skills have become a practical way to extend LLM agents by packaging metadata, natural-language instructions, and executable resources into reusable capability bundles. However, this growing Skill ecosystem introduces a new compliance risk: a Skill may perform high-impact actions that fall outside the scope permitted by the user's current request, thereby violating least privilege. Existing skill detection approaches are insufficient for this problem because it is inherently task-conditioned: the same action may be legitimate under one user prompt but over-privileged under another. In this paper, we present SkillScope, a framework for fine-grained least-privilege enforcement in Agent Skills. SkillScope adopts a graph-based analysis approach that models instruction-level procedures and code-level operations as fine-grained action nodes. It extracts potential over-privilege candidates, validates them under graph-instantiated user tasks through runtime analysis, and constrains validated over-privileged actions via control-flow privilege constraining. We evaluate SkillScope through effectiveness experiments and large-scale real-world measurement. SkillScope achieves a 94.53% skill-level F1 score for over-privilege detection. In the wild, SkillScope validates 6,590 of 68,312 valid real-world Skills as exhibiting over-privileged behaviors, showing that least-privilege violations are prevalent in current Skill ecosystems. In the privilege-constraining evaluation, SkillScope reduces triggered over-privileged action-in-task instances by 88.56% while preserving legitimate task completion.

cs.CR↗

arXiv2026

CoMerge: Conflict-Driven Preference Optimization for Multi-Task Model Merging

Model merging provides an efficient paradigm for constructing multi-task large language models (LLMs) without full model retraining, yet it remains challenged by parameter interference. While existing methods aim to preserve the capabilities of individual expert models and mitigate interference, they generally do not directly learn from the potentially degraded behaviors exposed by naive merging. In this paper, we propose a conflict-driven preference optimization framework for model merging (CoMerge), which reformulates model merging as a preference optimization problem. The approach utilizes a self-supervised, conflict-driven strategy that leverages the defects of naive merging methods (e.g., task arithmetic) as hard negative samples to construct preference pairs without external annotations. By applying preference optimization to refine lightweight, tensor-wise merging coefficients, CoMerge enables the model to mitigate parameter-space conflicts while preserving task-specific capabilities. Extensive experiments show that CoMerge achieves an average normalized performance of 0.9968 on MergeBench, outperforming all evaluated data-free and data-driven model-merging baselines. Furthermore, on Llama-3.1-8B-Instruct, CoMerge yields marked improvements on conflict-sensitive tasks such as instruction following and safety, while remaining highly competitive with full-parameter fine-tuning despite optimizing only 1,445 scalar coefficients.

cs.AI↗

arXiv2026

SALA: Semantic-Aware Logical Alignment for Complex Reasoning in In-Context Learning

Effective in-context learning (ICL) for complex reasoning relies on selecting the right demonstrations. Traditional retrieval methods based on surface similarity fail to capture the underlying problem-solving logic. Recent logic-based methods address this by matching predefined reasoning steps, but the rigid rules and exact-match criteria is improper to handle flexible or diverse reasoning processes. To address the problem, we propose SALA, a Semantic-Aware Logical Alignment framework. Instead of relying on a fixed inventory, SALA automatically learns task-specific reasoning operations. It then embeds these operations into a continuous semantic space and uses dynamic time warping (DTW) to align the reasoning sequences. This approach allows for soft, flexible matching of reasoning logic while remaining highly interpretable. Experiments across four reasoning benchmarks and three LLMs demonstrate that SALA outperforms existing demonstration selection methods. Further analysis confirms the roles of the operation induction and the logical semantic alignment.

cs.AI↗

arXiv2026

Adaptive Critical Token-Aware Retrieval for Repository-Level Code Generation

The repository-level code generation task requires synthesizing code that satisfies task requirements while remaining consistent with the target repository context. Since real-world repositories often exceed the input length limits of LLMs, existing approaches commonly adopt retrieval-augmented generation (RAG) to provide repository-specific context. Despite improving repository-context retrieval, existing methods typically provide context as task-level support, without explicitly identifying the critical tokens that require fine-grained repository context during generation. During the autoregressive generation process of LLMs, errors often concentrate at a small number of decisive positions: once such tokens are generated incorrectly, subsequent code may follow an incorrect semantic path and eventually lead to functional failure. We refer to these positions as "critical tokens". In this paper, we propose ACToR, an adaptive critical token-aware retrieval framework for repository-level code generation. ACToR identifies critical tokens during generation and triggers targeted retrieval on demand to provide repository context at these decisive positions. In addition, we design a position-aware weighting method for dense retrievers to prioritize context that is more informative for generation. We evaluate ACToR on two representative repository-level benchmarks, RepoExec and CoderEval. Experimental results show that ACToR consistently outperforms state-of-the-art methods, achieving relative improvements of 8.4% on RepoExec and 15.4% on CoderEval. Beyond performance gains, we systematically quantify the impact of critical tokens, revealing their central role in major generation failures and highlighting the necessity of targeted retrieval strategies. We provide the code and data at https://github.com/DeepSoftwareAnalytics/ACToR.

cs.SE↗