arXiv · 2611.14000
DynamAuction: A Reinforcement Learning Environment for Repeated Auctions with Dynamic Value
Abstract
A buyer who repeatedly participates in auctions for related items often faces a dynamic value problem: what they win today changes what tomorrow's items are worth. Despite the empirical importance of such effects, especially in display advertising, the algorithmic literature on repeated auctions has largely focused on the learning problem, by treating each opportunity as a contextual bandit, while sidestepping the planning problem induced by the dependence of current value on past wins. We introduce DynamAuction, a synthetic environment for single-buyer sequential bidding. The reward from each win depends on the buyer's history of previous wins, summarized through a carry-over state. The environment supports first- and second-price auctions, non-homogeneous Poisson arrivals, delayed feedback, saturation, and recency effects. We also introduce a new class of Model Predictive Control (MPC) algorithms. On our benchmark scenario, these MPC policies outperform off-the-shelf deep RL baselines (PPO, SAC, TD3) as well as a tuned shading baseline. We release DynamAuction to support RL research on repeated auctions.
Explore related subjects
Keep this discovery
Explore connections, maps & timelines
Eugénie Patard, Corentin Pla, Benjamin Heymann, Patrick Loiseau. 2026-11-10. DynamAuction: A Reinforcement Learning Environment for Repeated Auctions with Dynamic Value. https://arxiv.science/abs/2611.14000
Citations retain the original authors and DOI and include this site’s record URL. If this discovery service contributed to your research, you can also cite arXiv Science by Indiciae: BibTeX · RIS.