arXiv Science⌕ Search

arXiv · 2611.18934

Short Windows, Unmodelled Costs, Unknown Cutoffs: A Pre-Registered Evidence Audit of 91 LLM Trading Papers

Abstract

Papers on large-language-model (LLM) trading agents and alpha mining report Sharpe ratios that would be remarkable in any other part of empirical finance. We audit the evidence behind those numbers. Before coding a single paper we pre-registered a population of 91 papers from a curated corpus of LLM trading research, a ten-field coding rubric, an analysis script and five directional hypotheses, and committed all of them to version control. Every code is backed by a verbatim quote that a script checks against the paper's text, and a second, blind coder recoded a random 20%. Of the 75 papers with a trading test, 51% deduct no trading costs and only 9% model costs from data; the median test window is 12 months, and a quarter of windows are six months or shorter. Among the 51 papers whose test window is not known to lie after the model's training cutoff, 98% neither test after the cutoff, nor evaluate live, nor use a chronologically consistent model. Only 9% report a multiplicity-adjusted result, and just five papers combine deducted costs, a leakage control and a reported trial count. Two of the five hypotheses pass (short windows; uncontrolled look-ahead). Three fail: more papers report a trial count than we predicted, though most of those counts are repeated random seeds rather than the number of strategies searched; and the predicted links between headline Sharpe and test-window length, and between headline Sharpe and costs, are not detectable in the 13 papers that state a headline Sharpe in their abstract or conclusion. We release the codes, quotes and script, and propose a six-line reporting minimum.

Explore related subjects

Keep this discovery

Explore connections, maps & timelines

BibTeXRIS

Henry Zhang. 2026-11-13. Short Windows, Unmodelled Costs, Unknown Cutoffs: A Pre-Registered Evidence Audit of 91 LLM Trading Papers. https://arxiv.science/abs/2611.18934

Citations retain the original authors and DOI and include this site’s record URL. If this discovery service contributed to your research, you can also cite arXiv Science by Indiciae: BibTeX · RIS.

KEEP EXPLORING

Related papers

Beyond Strike Remapping: Conditional-Gamma Pricing of Leveraged ETF Options under Local Volatility

Options on leveraged ETFs (LETFs) are priced and hedged off the LETF volatility surface, but path-dependent variance drag makes that surface difficult to build consistently with the underlying index smile. We study three methods for constructing it: a closed-form first-order strike remapping, a new conditional-Gamma method, and a two-dimensional joint-law PDE that makes no distributional assumption and serves as the benchmark. The conditional-Gamma method is our main contribution. It conditions on the terminal underlying return, which preserves the state dependence of variance drag that a single remapped strike discards. Three one-dimensional forward PDEs under a calibrated local-volatility model give the conditional mean and variance of integrated variance, and a Gamma law matched to these moments keeps realised variance non-negative and prices the option in closed form. A shifted-Gamma variant that also matches the conditional third moment gives its clearest gains for inverse products at longer maturities, and a Gamma-mixture extension covers the full range of conditional skewness compatible with non-negative realised variance. On S&P 500 and Nikkei 225 surfaces the conditional-Gamma smile tracks the benchmark closely at every leverage tested, including the wings where the strike remapping drifts. A full LETF smile takes 0.85-1.42 seconds against 197-224 seconds for the benchmark, a 158- to 236-fold speed-up. Against listed TQQQ quotes the conditional-Gamma smile lies inside the market bid-ask at most strikes.

q-fin.CP↗

Mollification Regularization and Asymptotic Correction for Analytical Expansion of BSDEs with Non-Smooth Terminal Conditions. I. Theory

The analytical expansion method for BSDEs of Zhang (2017) breaks down when the terminal condition psi has kinks or discontinuities. We replace psi by psi_epsilon = psi * rho_epsilon and derive a joint error decomposition whose expansion constants explicitly track the derivative blow-up as epsilon tends to 0. For symmetric mollifiers the leading weak bias cancels. The associated balanced rates are conditional on an explicitly stated uniform local-to-global Taylor-propagation assumption: under that assumption the benchmark (l, m, alpha) = (0, 2, 1) gives epsilon* ~ delta_t^{1/3} and weak rate 2/3. Without it, the rigorous additive bound gives no decay when m - 2l = 2; an unconditional positive rate is obtained only for m - 2l > 2. Asymptotic corrections cancel the weak bias through order epsilon^{2J}; their faster exponents are formal targets unless the auxiliary expansion errors and the Picard defect satisfy additional rate conditions, which we state explicitly. For the driver-free fixed-center scheme we identify an n-independent terminal-representation error and show that it depends on the expansion center and on the terminal representative; change-of-basis invariance does not imply centering independence. A Gaussian moment-compression layer combines whitening, a Stein recursion, and knapsack truncation with a jet-conditioned error certificate. This Part I contains the theory, with all proofs self-contained; numerical evidence and implementation measurements are reported separately in the companion paper (Part II).

q-fin.CP↗