arXiv subjects

Benjamin Moll

Publications and source records attributed to Benjamin Moll.

4 recordsLinked to original sources

arXiv2026

Recurrent Structural Policy Gradient for Partially Observable Mean Field Games

Mean Field Games (MFGs) provide a principled framework for modelling interactions in large population systems. However, algorithmic progress has been limited since model-free methods are high variance and exact methods scale poorly. Recent Hybrid Structural Methods (HSMs) reduce variance while maintaining tractability by leveraging low-dimensional individual state and action spaces and known transition dynamics to compute the exact expected return conditioned on Monte Carlo rollouts of common noise. However, HSMs have not been extended to partially observable settings. We propose Recurrent Structural Policy Gradient (RSPG), the first history-aware HSM for MFGs with public partial information. RSPG achieves an order-of-magnitude faster convergence than model-free RL methods while learning history-aware behaviour, unlike current HSMs. To facilitate research into MFGs, we also introduce MFAX, our JAX-based framework for MFGs that supports both analytic and sample-based mean-field updates. MFAX and usage examples can be found at https://clarisse-wibault.github.io/rspg/.

cs.AI↗

arXiv2025

Structural Reinforcement Learning for Heterogeneous Agent Macroeconomics

We present a new approach to formulating and solving heterogeneous agent models with aggregate risk. We replace the cross-sectional distribution with low-dimensional prices as state variables and let agents learn equilibrium price dynamics directly from simulated paths. To do so, we introduce a structural reinforcement learning (SRL) method which treats prices via simulation while exploiting agents' structural knowledge of their own individual dynamics. Our SRL method yields a general and highly efficient global solution method for heterogeneous agent models that sidesteps the Master equation and handles problems traditional methods struggle with, in particular nontrivial market-clearing conditions. We illustrate the approach in the Krusell-Smith model, the Huggett model with aggregate shocks, and a HANK model with a forward-looking Phillips curve, all of which we solve globally within minutes.

econ.TH↗

arXiv2025

The Trouble with Rational Expectations in Heterogeneous Agent Models: A Challenge for Macroeconomics

The thesis of this essay is that, in heterogeneous agent macroeconomics, the assumption of rational expectations about equilibrium prices is unrealistic and should be replaced. Rational expectations imply that decision makers forecast equilibrium prices like interest rates by forecasting cross-sectional distributions. This leads to an extreme version of the curse of dimensionality: dynamic programming problems in which the entire distribution is a state variable ("Master equation" a.k.a. "Monster equation"). Frontier computational methods struggle with these infinite-dimensional Bellman equations, making it implausible that real-world agents solve the associated decision problems. These difficulties also limit the applicability of the heterogeneous-agent approach to central questions in macroeconomics -- those involving aggregate risk and non-linearities such as financial crises. This troublesome feature of the rational expectations assumption poses a challenge: what should replace it? I outline three criteria for alternative approaches: (1) computational tractability, (2) consistency with empirical evidence, and (3) (some) immunity to the Lucas critique. I then discuss several promising directions, including temporary equilibrium approaches, incorporating survey expectations, least-squares learning, and reinforcement learning.

econ.GN↗

arXiv2025

Mean Field Games without Rational Expectations

Mean Field Game (MFG) models implicitly assume "rational expectations", meaning that the heterogeneous agents being modeled correctly know all relevant transition probabilities for the complex system they inhabit. When there is common noise, it becomes necessary to solve the "Master equation", in which the infinite-dimensional density of agents is a state variable. The rational expectations assumption and the implication that agents solve Master equations is unrealistic in many applications. We show how to instead formulate MFGs with non-rational expectations. Departing from rational expectations is particularly relevant in "MFGs with a low-dimensional coupling", i.e. MFGs in which agents' running reward function depends on the density only through low-dimensional functionals of this density. This happens, for example, in most macroeconomics MFGs in which these low-dimensional functionals have the interpretation of "equilibrium prices." In MFGs with a low-dimensional coupling, departing from rational expectations allows for completely sidestepping the Master equation and for instead solving much simpler finite-dimensional HJB equations. We introduce an adaptive learning model as a particular example of non-rational expectations and discuss its properties.

math.AP↗