arXiv · 2611.10453
BunshinRL: Whole-Workspace Forking for Agentic Reinforcement Learning Rollouts
Abstract
Agentic reinforcement learning (RL) rollouts interleave model reasoning with tool calls in persistent workspaces. Scaling these rollouts incurs substantial memory costs for concurrent environments, while slow tool calls delay entire rollout batches. We present BunshinRL, an agentic RL rollout system that exploits shared environment state and recurring tool calls through three techniques: memory sharing across environments, tool-prefix caching, and speculative tool execution. These techniques build on BunshinFork, a novel live whole-workspace forking mechanism for containers. BunshinFork reconstructs process trees, shares memory copy-on-write, and preserves file, communication, and GUI state, enabling cached-state reuse and rollback after incorrect speculation. We implement BunshinRL and BunshinFork through Linux kernel module and CRIU modifications. BunshinRL reduces peak memory by up to 43%, total memory consumption by up to 51%, and end-to-end rollout latency by 23%.
Explore related subjects
Keep this discovery
Explore connections, maps & timelines
Yutong Huang, Vikranth Srivatsa, Alex Asch, Yiying Zhang. 2026-11-11. BunshinRL: Whole-Workspace Forking for Agentic Reinforcement Learning Rollouts. https://arxiv.science/abs/2611.10453
Citations retain the original authors and DOI and include this site’s record URL. If this discovery service contributed to your research, you can also cite arXiv Science by Indiciae: BibTeX · RIS.