Machine Learning Methods for Large Population Games with Applications in Operations Research

Abstract

In this tutorial, we provide an introduction to machine learning methods for finding Nash equilibria in games with large number of agents. These types of problems are important for the operations research community because of their applicability to real life situations such as control of epidemics, optimal decisions in financial markets, electricity grid management, or traffic control for self-driving cars. We start the tutorial by introducing stochastic optimal control problems for a single agent, in discrete time and in continuous time. Then, we present the framework of dynamic games with finite number of agents. To tackle games with a very large number of agents, we discuss the paradigm of mean field games, which provides an efficient way to compute approximate Nash equilibria. Based on this approach, we discuss machine learning algorithms for such problems. First in the context of discrete time games, we introduce fixed point based methods and related methods based on reinforcement learning. Second, we discuss machine learning methods that are specific to continuous time problems, by building on optimality conditions phrased in terms of stochastic or partial differential equations. Several examples and numerical illustrations of problems arising in operations research are provided along the way.