Speculative Forking: Turning Idle Wait into Parallel Exploration for Repository Environment Setup

Yuling Shi; Mingchu Zhong; Zhensu Sun; Weihan Peng; Zhi Chen; Shilin He; Chengcheng Wan; Kai Cai; Hongyu Zhang; David Lo; Xiaodong Gu

arXiv · 2610.33129

Speculative Forking: Turning Idle Wait into Parallel Exploration for Repository Environment Setup

Yuling Shi·Mingchu Zhong·Zhensu Sun·Weihan Peng·Zhi Chen·Shilin He·Chengcheng Wan·Kai Cai·Hongyu Zhang·David Lo·Xiaodong Gu

View original source ↗Read source document ↗Export metadata ↓

Abstract

Before developers can use a software project, they must first get it to run. This often requires installing the right language runtime, system packages, build tools, and test commands. Recent LLM-based agents automate environment setup by inspecting a repository, writing setup scripts, running them, and repairing the resulting errors. However, these agents still follow a sequential trial-and-error loop: they try one setup route, wait for slow commands to finish, observe the failure, and only then decide what to try next. Because environment commands often take long to run, this sequential workflow makes setup slow and limits exploration to one route at a time. We present SpecBuild, an LLM-based environment setup system designed to be both faster and more successful through speculative forking. SpecBuild explores candidate setup branches in parallel, returns the first committed environment, and discards the rest. It applies this design at two granularities: strategy-level forking explores alternative setup routes, such as different base images and dependency policies, while command-level forking predicts failures of running commands and prepares repair branches before those failures are observed. We evaluate SpecBuild on Multi-Docker-Eval, covering 334 tasks from 40 repositories and 9 programming languages. Across three backbone models, SpecBuild improves the success rate by 6.4 to 12.0 percentage points over the strongest baseline, while reducing average setup time by 22.4% to 41.6%. These results show that LLM-based environment setup need not trade speed for reliability: parallel speculative exploration can make agents both faster and more successful.

Explore related subjects

cs.SE cs.CL cs.LG

Keep this discovery

Explore connections, maps & timelines

BibTeX RIS

Yuling Shi, Mingchu Zhong, Zhensu Sun, Weihan Peng, Zhi Chen, Shilin He, Chengcheng Wan, Kai Cai, Hongyu Zhang, David Lo, Xiaodong Gu. 2026-10-23. Speculative Forking: Turning Idle Wait into Parallel Exploration for Repository Environment Setup. https://arxiv.org/abs/2610.33129

Cite the original work for its findings. Save a collection to share your selection of sources.

KEEP EXPLORING

Related papers

arXiv2026

Chain-of-Thought in Neural Code Generation: From and For Lightweight Language Models

Large Language Models (LLMs) have demonstrated remarkable potential in code generation. The integration of Chain of Thought (CoT) reasoning can further boost their performance. However, current CoT methods often require manual writing or LLMs with over 100 billion parameters to generate, impeding their applicability in resource-constrained scenarios. In this study, we investigate lightweight Language Models (lLMs), which are defined to have fewer than 10 billion parameters. Empirically, we find that most lLMs cannot generate high-quality CoTs when prompted by the few-shot method, but can take advantage of high-quality CoTs generated elsewhere to improve their performance in code generation. Based on these findings, we design a novel approach COTTON which can leverage lLMs to automatically generate CoTs for code generation. We synthesize new datasets and conduct extensive experiments on various benchmarks. The results show that the CoTs generated by COTTON outperform the baselines in terms of automated and human evaluation metrics. In particular, the CoTs generated by COTTON boost various lLMs to achieve higher performance gains than those generated by LLMs such as ChatGLM (130B), and are competitive with those generated by gpt-3.5-turbo (175B). Our study also showcases the potential of lLMs in software engineering applications.

cs.SE↗

arXiv2026

From Legacy Fortran to Portable Kokkos: An Autonomous Agentic AI Workflow

Scientific applications still rely heavily on legacy Fortran codebases built for homogeneous, CPU-based systems. As High-Performance Computing (HPC) shifts toward GPU-accelerated architectures, many accelerators lack native Fortran bindings, creating a need to translate and optimize legacy code for portability. Kokkos offers performance portability through a single-source C++ abstraction, but manual porting from Fortran demands significant expertise and time. While large language models (LLMs) show promise in source-to-source code generation, their use in autonomous workflows for translating and optimizing parallel code is largely unexplored. This paper presents an agentic AI workflow in which specialized LLM agents collaborate to translate, validate, compile, run, test, debug, and optimize Fortran kernels into portable Kokkos C++ programs without human intervention, bounded by configurable retry and optimization thresholds. We evaluate the pipeline over five runs per configuration on an NVIDIA A100, characterizing its variability in success rates, token cost, and optimization behavior, on benchmark kernels including a real-world multi-file kernel from a production climate model. Proprietary models such as GPT-5 and o4-mini-high completed the full workflow most reliably, producing optimized code that exceeded the Fortran baselines for only a few U.S. dollars, whereas open-weights models such as gpt-oss-120b and Qwen3.5-397B-A17B handled simpler kernels but struggled with complex Kokkos abstractions such as parallel reductions and hierarchical parallelism. This work demonstrates, at the kernel level, the feasibility of agentic AI for Fortran-to-Kokkos transformation and, more broadly, for structured software-engineering tasks such as automated code modernization and refactoring.

cs.SE↗

arXiv2026

ScratchWorld: Evaluating If World Models Compute Executable Consequences

Evaluations of world models commonly score a predicted future state by its overlap with the ground truth. Yet in interactive worlds, this rewards copying: a model that merely preserves persistent state appears accurate without predicting any consequence of an action. We introduce ScratchWorld, an execution-grounded diagnostic benchmark that treats Scratch projects as executable worlds and derives every label from replay on a pinned Scratch VM, including next-state transitions, hidden variables, causal traces, and counterfactual outcomes. Its primary metric, value-aware changed-field F1 ($F_1^{\mathrm{VA}}$), credits a prediction only when it names a changed field and computes its executed value. Model-free anchors make the confound explicit: a no-change prior attains 91.4% full-state field accuracy on synthetic long-horizon items and 0% changed-field credit on non-empty one-step items, whereas VM replay attains 100% wherever the corresponding metric is defined. Across seven frontier LLMs prompted as world models, state-only $F_1^{\mathrm{VA}}$ reaches at most 30.0% on controlled synthetic mechanisms and declines steeply as projects approach real-world complexity. Revealing the full program raises every model's value-aware score by 22.3-51.7 points and substantially reorders the models, yet the best reaches only 55.4%, an execution gap that persists even with the rules visible. Auxiliary diagnostics complete the error profile: causal-event accuracy remains within 2.5 points of the 50% no-event baseline, with no correct exact chain; counterfactual prediction trades changed-outcome accuracy against invariance; hidden-state accuracy degrades roughly twice as fast as full-state overlap over long horizons. In a word, ScratchWorld separates what a model knows about the rules from what it can actually execute.

cs.SE↗