arXiv Science⌕ Search

arXiv subjects

Julius Durmann

Publications and source records attributed to Julius Durmann.

5 recordsLinked to original sources

Simultaneous Gradient Learning in First-Price Auctions

We study equilibrium learning in discretized Bayesian games, focusing on first-price auctions as a testbed for gradient-based learning dynamics under private information. While the symmetric independent private values model admits a unique Bayesian correlated equilibrium (BCE) in the continuous limit, standard convergence guarantees for gradient-based learners do not apply: classical no-regret theory predicts convergence only to Bayesian coarse correlated equilibria (BCCE), which can remain large and far from the competitive equilibrium even under fine discretizations. To bridge this gap, we introduce the notion of Bayesian semicoarse correlated equilibrium (BSCCE), an equilibrium concept that refines BCCE by restricting deviations to probability-preserving transformations consistent with projected gradient dynamics. We provide a linear programming characterization of BSCCE with polynomially many variables and constraints and establish the inclusion hierarchy $BCE \subseteq BSCCE \subseteq BCCE$. Our main finding is that the BSCCE set contracts in many cases as the discretization of the Bayesian game is refined. Even with priors for which BCCE fails to concentrate, the diameter of the BSCCE polytope shrinks toward zero, indicating convergence to the unique continuous-game Bayes-Nash equilibrium. These results provide a principled explanation for the observed robustness of gradient-based learning in discretized Bayesian games, showing that these dynamics select a smaller equilibrium set than standard no-regret theory predicts.

cs.GT↗

Mean-based algorithms: A lower bound and regret

Mean-based algorithms are online learning algorithms that assign low probability to actions with low average rewards. Recent research shows that they converge to serially undominated actions, which serve as approximations to Nash equilibria in economic games. However, empirical studies indicate that mean-based algorithms converge more slowly in bandit-feedback settings than established no-regret alternatives. This work investigates mean-based algorithms under unknown horizons and bandit feedback. In this setting, we provide the first lower bound on the algorithm-defining sequence $γ_t$, establishing a fundamental limit on the learning speed of such algorithms. In multi-armed bandit problems, this result constrains the rate at which any algorithm can reliably identify low-reward actions while acting according to this knowledge. We also propose two mean-based algorithms: one generalizes $ε$-greedy, and the other extends mean-based Exp3 to unknown horizons. Our experiments show that mean-based algorithms, although slightly slower, can perform competitively with other bandit-feedback algorithms. We further study the relationship to regret. Depending on the choice of $γ_t$, the intersection with no-regret algorithms is non-trivial, and we show that some algorithms are both mean-based and no-regret.

cs.LG↗

Online Optimization Algorithms in Repeated Price Competition: Equilibrium Learning and Algorithmic Collusion

This paper investigates whether online learning algorithms in pricing produce competitive outcomes or tacit collusion. This issue has drawn considerable attention from competition regulators as algorithmic pricing becomes more common in digital markets. Understanding when such algorithms lead to equilibrium or supra-competitive prices is critical for buyers, sellers, and policymakers. We study the behavior of multi-armed bandit (MAB) online learning algorithms in repeated price competition. These algorithms require little information to learn, making them realistic models of automated pricing. Our analysis shows that mean-based algorithms, a special variant of online learning algorithms, converge to correlated rationalizable actions. In the Bertrand environments considered, this implies convergence to the Nash equilibrium or adjacent prices. Numerical experiments reveal that most MAB algorithms, including those that are not mean-based, also converge. We observe supra-competitive prices only in specific cases where all sellers implement the same symmetric version of certain algorithms, such as UCB. This effect diminishes as the number of competitors increases. Our results suggest that, even in a stylized repeated Bertrand competition, sustained supra-competitive prices may be less of a concern when independent agents use different online learning algorithms. Our insights are relevant for regulators and managers considering the use of algorithmic pricing algorithms.

cs.GT↗

Agentic Markets: Game Dynamics and Equilibrium in Markets with Learning Agents

Autonomous and learning agents increasingly participate in markets - setting prices, placing bids, ordering inventory. Such agents are not just aiming to optimize in an uncertain environment; they are making decisions in a game-theoretical environment where the decision of one agent influences the profit of other agents. While game theory usually predicts outcomes of strategic interaction as an equilibrium, it does not capture how repeated interaction of learning agents arrives at a certain outcome. This article surveys developments in modeling agent behavior as dynamical systems, with a focus on projected gradient and no-regret learning algorithms. In general, learning in games can lead to all types of dynamics, including convergence to equilibrium, but also cycles and chaotic behavior. It is important to understand when we can expect efficient equilibrium in automated markets and when this is not the case. Thus, we analyze when and how learning agents converge to an equilibrium of a market game, drawing on tools from variational inequalities and Lyapunov stability theory. Special attention is given to the stability of projected dynamics and the convergence to equilibrium sets as limiting outcomes. Overall, the paper provides mathematical foundations for analyzing stability and convergence in agentic markets driven by autonomous, learning agents.

cs.GT↗

Algorithmic Pricing and Algorithmic Collusion

The rise of algorithmic pricing in online retail platforms has attracted significant interest in how autonomous software agents interact under competition. This article explores the potential emergence of algorithmic collusion - supra-competitive pricing outcomes that arise without explicit agreements - as a consequence of repeated interactions between learning agents. Most of the literature focuses on oligopoly pricing environments modeled as repeated Bertrand competitions, where firms use online learning algorithms to adapt prices over time. While experimental research has demonstrated that specific reinforcement learning algorithms can learn to maintain prices above competitive equilibrium levels in simulated environments, theoretical understanding of when and why such outcomes occur remains limited. This work highlights the interdisciplinary nature of this challenge, which connects computer science concepts of online learning with game-theoretical literature on equilibrium learning. We examine implications for the Business & Information Systems Engineering (BISE) community and identify specific research opportunities to address challenges of algorithmic competition in digital marketplaces.

cs.GT↗