arXiv subjects

Cade Armstrong

Publications and source records attributed to Cade Armstrong.

3 recordsLinked to original sources

arXiv2026

$Q$ Can Play That Game: Online Fitted $Q$-Iteration for Continuous-Action Zero-Sum Markov Games with Convex-Concave Function Approximation

Zero-sum Markov games arise in a wide variety of sequential decision-making problems such as adversarial learning and planning against modeled uncertainties. However, prior work on finite-sample guarantees on the learned state-action value function ($Q$-function) for zero-sum Markov games is largely restricted to finite-action settings, or to continuous-action games in which agents have linear dynamics and quadratic rewards (i.e. linear-quadratic, or LQ, games). A central challenge in extending such guarantees to more general continuous-action zero-sum Markov games is that the associated Bellman operator involves a minimax problem that need not admit a tractable saddle-point solution. To this end, we first introduce a class of neural network function approximators for the $Q$-function that is convex-concave in the players' actions, which guarantees that this minimax problem admits a pure-strategy saddle point. We then study an online variant of fitted $Q$-iteration employing this function class and establish, to the best of our knowledge, the first finite-sample guarantees for non-LQ zero-sum Markov games with continuous states and actions. Our code can be found at https://github.com/CLeARoboticsLab/QCanPlayThatGame.

cs.GT↗

arXiv2026

Cislunar Pursuit-Evasion Game on Periodic and Quasi-Periodic Orbits

Cislunar spacecraft operate in nonlinear and unstable environments that make defensive maneuver planning difficult. We formulate cislunar spacecraft pursuit and evasion as a zero-sum differential game in the circular restricted three-body problem. Each spacecraft controls its thrust and reference orbit phase, enabling motion along periodic orbits and across quasi-periodic tori while remaining near the reference. We solve the game using a constrained discrete-time differential dynamic programming method that enforces hard input constraints. We propose a shared time regularization to synchronize both spacecraft while refining the discretization near close lunar passages. Numerical results on periodic and quasi-periodic orbits show that phase control improves maneuvering flexibility while limiting departure from the reference orbit. The discrete-time method also provides a large computational advantage over a continuous-time formulation. Comparisons between quasi-halo and quasi-near-rectilinear halo orbits show that close lunar passages create larger escape opportunities but also increase the sensitivity of the encounter. These results show that reference orbit geometry is an important part of defensive cislunar mission design.

eess.SY↗

arXiv2025

Inferring Foresightedness in Dynamic Noncooperative Games

Dynamic game theory is an increasingly popular tool for modeling multi-agent, e.g. human-robot, interactions. Game-theoretic models presume that each agent wishes to minimize a private cost function that depends on others' actions. These games typically evolve over a fixed time horizon, specifying how far into the future each agent plans. In practical settings, however, decision-makers may vary in foresightedness, or how much they care about their current cost in relation to their past and future costs. We conjecture that quantifying and estimating each agent's foresightedness from online data will enable safer and more efficient interactions with other agents. To this end, we frame this inference problem as an inverse dynamic game. We consider a specific objective function parametrization that smoothly interpolates myopic and farsighted planning. Games of this form are readily transformed into parametric mixed complementarity problems; we exploit the directional differentiability of solutions to these problems with respect to their hidden parameters to solve for agents' foresightedness. We conduct three experiments: one with synthetically generated delivery robot motion, one with real-world data involving people walking, biking, and driving vehicles, and one using high-fidelity simulators. The results of these experiments demonstrate that explicitly inferring agents' foresightedness enables game-theoretic models to make 33% more accurate models for agents' behavior.

cs.RO↗