arXiv · 2611.10809
Gradient Estimation for LLM-Embedded Simulation Optimization
Abstract
Large language models (LLMs) are increasingly embedded in operational simulators to represent context-dependent human decisions, which opens new possibilities for simulation optimization. However, the LLM acts as a black box, so derivatives through its decisions are unavailable for gradient-based policy search. Finite difference estimation can avoid this requirement, but the required paired simulations that sample LLM actions independently often disagree, and such action switches persist under small perturbations, which inflates the gradient variance to order $c^{-2}$ for perturbation radius $c$. To control this variance, we develop a behavioral common random number framework. It first applies maximal coupling to the LLM's action distributions, preserving each simulation's behavioral law while making action switches as rare as possible, which reduces the variance bound to $O(c^{-1})$. It then applies first-switch conditioning, which averages over whether a switch occurs, together with an impact-aware extension that prioritizes influential switches. Under suitable stability conditions, both keep the variance bounded as $c$ shrinks. We further establish improved complexity bounds for projected stochastic approximation. Experiments on synthesized and LLM-embedded simulators confirm the theory and show substantial performance gains.
Explore related subjects
Keep this discovery
Explore connections, maps & timelines
Ruiyang Jin, Siyang Gao. 2026-11-07. Gradient Estimation for LLM-Embedded Simulation Optimization. https://arxiv.science/abs/2611.10809
Citations retain the original authors and DOI and include this site’s record URL. If this discovery service contributed to your research, you can also cite arXiv Science by Indiciae: BibTeX · RIS.