arXiv Science⌕ Search
Home/Archive/arXiv

arXiv · 2611.04375

LLM-Generated Investment Research Memos Need Validation

Xukang Gong·Chenri Luo·Binyan Yu·Xiao-Yang Liu
View original source ↗Read source document ↗Export metadata ↓

Abstract

LLMs are increasingly adopted to draft long-form investment research memos, but their reliability is judged claim by claim, while readers read each memo as a whole. In this paper, we show that a low claim-level error rate does not necessarily mean that a memo is reliable: even with a refutation rate of just 1-2%, roughly four in ten memos contain at least one claim contradicted by official filings. First, we generate 1,440 memos with four recent LLM-powered agents for 30 companies over 12 quarters, decompose them into 77,272 financial claims, and verify each against the same official source bundle by retrieving evidence, matching it to the claim, and verifying the outcome. Second, we find that only 1.17-1.77% of claims are refuted, yet 36.94-41.11% of memos contain at least one refuted claim, and agents differ in severity: Codex has the lowest memo-level rate; a qualitative review further groups errors into four patterns (time, unit, context, concept).

Explore related subjects

cs.CE

Keep this discovery

Explore connections, maps & timelines

BibTeXRIS

Xukang Gong, Chenri Luo, Binyan Yu, Xiao-Yang Liu. 2026-11-03. LLM-Generated Investment Research Memos Need Validation. https://arxiv.org/abs/2611.04375

Cite the original work for its findings. Save a collection to share your selection of sources.

KEEP EXPLORING

Related papers

arXiv2026

TCBench: A Benchmark for Tropical Cyclone Track and Intensity Forecasting at the Global Scale

TCBench is a benchmark for evaluating global, short to medium-range (1-5 days) forecasts of tropical cyclone (TC) track and intensity. To allow a fair and model-agnostic comparison, TCBench builds on the IBTrACS observational dataset and formulates TC forecasting as predicting the time evolution of an existing tropical system conditioned on its initial position and intensity. TCBench includes state-of-the-art physics-based (TIGGE) and Artificial Intelligence Weather Prediction (AIWP) models (AIFS, Pangu-Weather, FourCastNetV2, GenCast, FNV3P2). If not readily available, TC tracks are consistently derived from model outputs using the TempestExtremes library. TCBench provides deterministic and probabilistic storm-following metrics. On 2023 test cases, AIWP models skillfully forecast TC tracks, while realistic intensity forecasts require additional steps such as post-processing or task-specific training. Designed for accessibility, TCBench helps AI practitioners tackle domain-relevant TC challenges and equips tropical meteorologists with data-driven tools and workflows to improve prediction and TC process understanding. By lowering barriers to reproducible, process-aware evaluation of extreme events, TCBench aims to democratize data-driven TC forecasting.

cs.CE↗
arXiv2026

Sparse stochastic digital twins can outperform exact ones

Digital twins are often built under the assumption that greater model fidelity leads to better downstream performance. We show that this is not always the case: retaining weakly dependent or auxiliary degrees of freedom can increase the dimensionality of learning and control problems enough that a sparse stochastic surrogate outperforms even an exact high-dimensional model. To identify such structure, we introduce a framework for dependence discovery directly at the level of the full conditional law. The method combines conditional generative modeling with Gaussian-process distillation and kernel mode decomposition to quantify how strongly individual variables enter the conditional distribution, including through variability, tails, multimodality, and other features invisible to the conditional mean. The resulting dependence structure can be used both to construct sparse stochastic digital twins from data and to reduce known high-dimensional models for downstream tasks. In a modified Lunar Lander problem, an eight-dimensional sparse surrogate yields better control performance than the exact 109-dimensional simulator despite lower transition fidelity. A stochastic heat-equation example shows that the framework recovers sparse structure in a 102-dimensional data-driven setting and produces a more effective surrogate than one conditioned on all candidate inputs, while economic data demonstrate its use for identifying time-varying memory structure. These results show that model fidelity and downstream utility are distinct objectives, and that dependence discovery provides a practical route to sparse stochastic digital twins.

cs.CE↗
arXiv2026

DropFENO: A Data-Free, Physics-Based Operator-Learning Framework for Droplet Equilibria on Mixed-Wettability Patterned Surfaces

We study whether the equilibrium of a drop on a chemically patterned wall, and the capillary forces that the pattern exerts on it, can be learned from the free energy alone, without solution data. We introduce DropFENO (Droplet Free-Energy Neural Operator), in which Fourier (FNO), tensor-factorised Fourier (TFNO) and wavelet (WNO) neural operators are trained in two and three dimensions on the diffuse-interface free energy of the drop, with its volume and centroid constraints imposed as penalties. The predictions are tested with the virtual-work (Hellmann--Feynman) theorem: the force exerted by the pattern on a held drop is a wall integral of the equilibrium field which, for a drop straddling a wettability step, reduces identically to the contact-line law $F=σΔ\cosθ\,L$. DropFENO reproduces this law and the energy landscape of a drop crossing a step, and reproduces the pinning of the receding contact line less accurately. The residual error lies in the soft modes of the capillary energy (small displacements of the interface and the contact line). This explains why the energy is predicted accurately while contact-line positions are not, and why on striped walls the operators return Cassie-like states instead of the edge-pinned states whose anisotropic apparent angles the reference minimiser shares with lattice Boltzmann simulations. On held-out patterned walls the median energy of the predictions lies $\NdEmedRandFnoTwo$ of the free-drop energy above the best known minimum (FNO, two dimensions), at about one hundredth of the cost of a per-instance minimisation, and a refinement by a classical solver started from the prediction returns the reference state in almost all cases.

cs.CE↗