arXiv subjects

Krishnendu Chatterjee

Publications and source records attributed to Krishnendu Chatterjee.

At least 19 recordsLinked to original sources

arXiv2026

Values of Absorbing Recursive Games Express All Real Algebraic Numbers

Many classes of two-player zero-sum stochastic games have the orderfield property: if all payoffs and transition probabilities lie in a subfield of $\mathbb{R}$, so does the undiscounted value. Absorbing games fail this property, and Oliu-Barton and Vigeral [\emph{Absorbing games with irrational values}, Oper.\ Res.\ Lett.\ 51 (2023) 555--559] conjectured the precise extent of the failure: every real algebraic number $\alpha$ of degree $m\geq 1$ over $\mathbb{Q}$ is the undiscounted value of a rational $m\times m$ absorbing game. We prove this conjecture, and in fact within a special subclass of absorbing games: for every such $\alpha$, the game realizing it is \emph{strictly absorbing} and \emph{recursive}, i.e., every action pair is absorbing with positive probability and all non-absorbing stage payoffs are zero; when $\alpha>0$ it can moreover be taken \emph{positive recursive}, with positive absorbing payoffs. As a corollary, the set of undiscounted values of rational $m\times m$ absorbing games is exactly the set of real algebraic numbers of degree at most $m$

math.OC↗

arXiv2026

The Complexity of Approximating the Value in Revealing POMDPs with Long-Run Average Objectives

We study partially observable Markov decision processes (POMDPs) with long-run average objectives, where the payoff is defined as the limit inferior of the expected average rewards. We consider the computational problem of approximating the long-run average value of a POMDP. In general, the long-run average value of a POMDP is neither computable nor approximable. We therefore consider the subclass of revealing POMDPs. Informally, a POMDP is revealing when the controller observes the underlying state with positive probability at each stage of the process. Our main contributions are threefold. First, we illustrate the practical relevance of this class of POMDPs through an application in control and optimization. Second, we present an exponential-time algorithm for the value approximation problem. Third, we establish EXPTIME-hardness by a reduction from the problem of almost-sure safety in POMDPs. Together, these results show that the problem of approximating the long-run average value in revealing POMDPs is EXPTIME-complete.

math.OC↗

arXiv2026

Quantitative Analysis of $\omega$-Regular Robust MDPs

Robust Markov Decision Processes (RMDPs) generalize classical MDPs by allowing uncertainty in transition probabilities and optimizing against their worst-case realization. We consider $(s,a)$-rectangular RMDPs with \emph{linearly defined} uncertainty sets and study parity objectives, which are a canonical representation of $\omega$-regular objectives. An uncertainty set is linearly defined if it is described by linear inequalities over the transition distribution together with auxiliary variables, which capture the standard $L_1$ and $L_\infty$ balls as well as general polytopic uncertainty sets. The quantitative value is the supremum, over all agent policies, of the satisfaction probability guaranteed against the adversarial environment. Previous work studied the qualitative analysis, namely the almost-sure (resp. positive) problem that asks whether a single agent policy guarantees satisfaction with probability one (resp. positive probability) against every environment policy. In this work, we solve the exact quantitative problem. Our contributions are threefold. First, we show that both the agent and the environment admit pure memoryless optimal policies. Second, we give a polynomial-time algorithm for quantitative parity on linearly defined robust Markov chains and use it as a subroutine in a policy-iteration algorithm for RMDPs. The algorithm combines quantitative one-step improvements with qualitative almost-sure improvements. Finally, we report experiments comparing our approach with the explicit reduction to stochastic games.

cs.AI↗

arXiv2026

Noise-aware Verification and Synthesis of Quantum Programs

While most research on quantum programming considers an idealized, noise-free semantics for quantum programs, we reason about quantum programs that are executed on real, noisy hardware. We consider the error models published by quantum hardware vendors to give a hardware-dependent semantics to quantum programs. This work presents a comprehensive study of noise-aware quantum programming, ranging from logical foundations to automated verification and synthesis. We develop a noise-aware quantum Hoare logic, and use it to derive algorithmic methods for the bounded verification of quantum programs on specific hardware, and for the automatic synthesis of noise-optimal loop-free quantum programs. In this way, we synthesize hardware-dependent subroutines that commonly occur in quantum algorithms, such as parity checks, quantum state preparation, and quantum state discrimination. We evaluate our method on the hardware specifications provided by the IBM Qiskit toolkit. Besides finding different optimal subroutines for different noise models, our synthesis tool also shows that classical probabilistic branching is needed for optimality in quantum programming.

cs.PL↗

arXiv2026

Replacers and their evolutionary stability in the Moran process on graphs

Evolutionary dynamics in finite structured populations are commonly modeled by the Moran Birth-death process. A key quantity is the fixation probability of a single invader attempting to take over a population of residents. A recent work introduced a new neighborhood-aware phenotype called a replacer. A replacer never wastes their reproductive turn by always replacing an individual of the other type (if available). In this work, we study the evolutionary stability of resident replacers who are invaded by mutant replacers. We find that residents are strongly protected against such invasions, and we quantify the strength of this effect by showing three types of results. First, we show that on well-mixed populations of size $N$, the invader fixation probability is exponentially small in $N$, even when the invader has a fixed relative reproductive rate $r>1$, and the same holds for all high-degree graphs. Second, we study bounded-degree graphs. We prove that on cycles, the fixation probability of an advantageous invader decreases only by a constant factor. However, we also present graphs with maximum degree 4, where the invader fixation probability is exponentially small in $N$ whenever $r\le1.9$. Thus, high degrees are sufficient for evolutionary stability, whereas with low degrees the evolutionary stability depends on specific features of the underlying spatial structure. Third, we prove general bounds for arbitrary graphs. Namely, we show that for any graph $G$ the invader fixation probability drops below the natural baseline given by the standard Moran process with oblivious individuals on a well-mixed population, both for $r\approx 1$ and for $r\ge 2$. Together, our results establish that the evolutionary dynamics of replacers is better characterized by the phrase ``survival of the first'' rather than the classic ``survival of the fittest''.

q-bio.PE↗

arXiv2026

PAC Learning in Turn-Based Stochastic Games with Reachability Objectives: A Decentralized Private Approach via Expected Conditional Distance

Reachability is the most fundamental logical objective, yet it is notoriously difficult to learn in reinforcement learning settings: even for Markov decision processes, PAC learning of reachability is impossible without additional assumptions. This difficulty also holds in turn-based stochastic games (TBSGs), where two adversarial players interact on a finite state space. In this work, we consider turn-based stochastic games with reachability objectives. For such settings, adversarial learning, in which players are adversarial even in the learning phase, is impossible. Therefore, the goal is to consider learning, in which both players learn the unknown model together. In this spirit, previous literature on PAC learning in TBSGs considers (a)~public information shared by both players; and (b)~centralized learning, which means that players share the same learning algorithm. In this work, our contribution is two-fold. First, we relax these strong assumptions and ensure learning: (i)~with private information not shared with the other player; and (ii)~decentralized learning where the players do not share the same learning algorithm. To the best of our knowledge, this work is the first positive result for decentralized and private information learning of TBSGs with reachability objectives. Second, we introduce a game-theoretic generalization of the Expected Conditional Distance (ECD) parameter, which measures the expected length of reaching the target set. We establish a polynomial-sample complexity bound with respect to the number of states, actions, ECD parameter, and inverses of error tolerance and failure probability.

cs.LG↗

arXiv2026

Reinforcement Learning for Reachability: Guaranteeing Asymptotic Optimality

Reinforcement learning (RL) for reachability specifications is fundamental in sequential decision-making, yet theoretical guarantees remain less explored. A recent work achieves asymptotic convergence to optimal policies. However, this approach provides limited insight into convergence dynamics. In this work, we present an alternative approach that provides deeper theoretical insights into convergence. Our approach builds on PAC learning with assumptions. PAC learning guarantees near-optimal policies with high confidence in finite time but requires knowing internal MDP parameters like minimum transition probability. We argue that while these parameters are unknown in RL, they can be iteratively refined and estimated with increasing accuracy. By iteratively satisfying PAC conditions, we show that exact optimality can be achieved in the limit. Empirical evaluations on standard benchmarks validate our theoretical insights into convergence dynamics.

cs.LG↗

arXiv2026

Automated Approach for Solving Infinite-state Polynomial Reachability Games

Reachability games are two-player games played on a graph, where the objective of $\texttt{REACH}$ player is to reach the target set whereas the objective of $\texttt{SAFE}$ player is to stay away from the target set. Reachability games have important applications in artificial intelligence and reactive synthesis, and many of these applications give rise to infinite-state reachability games. In this paper, we study turn-based reachability games on infinite-state graphs defined over valuations of a finite set of real variables. We consider the problem of determining the existence of and computing a winning strategy for $\texttt{REACH}$ player. Our contributions are twofold. First, we propose ranking certificates for reachability games, a sound and complete proof rule for proving that $\texttt{REACH}$ player has a winning strategy from the specified initial state. Second, we consider polynomial reachability games, where transitions and objectives are described by polynomial constraints over real variables, and propose a fully automated algorithm for computing a winning strategy for $\texttt{REACH}$ player together with a formal correctness witness in the form of a ranking certificate. The algorithm is sound, semi-complete, and runs in sub-exponential time. Our experiments demonstrate the ability of our method to solve challenging examples from the literature that were out of the reach of existing methods. Specifically, for the classical Cinderella-Stepmother game, we are able to compute an optimal winning strategy for an arbitrary precision parameter for the first time.

cs.AI↗

arXiv2026

On the Complexity of Discounted Robust MDPs with $L_p$ Uncertainty Sets

A basic model in sequential decision making is the Markov decision process (MDP), which is extended to Robust MDPs (RMDPs) by allowing uncertainty in transition probabilities and optimizing against the worst-case transition probabilities from the uncertainty sets. The class of $(s, a)$-rectangular RMDPs with $L_p$ uncertainty sets provides a flexible and expressive model for such problems. We study this class of RMDPs with a discounted-sum cost criterion and a constant discount factor. The existence of an efficient algorithm for this class is a fundamental theoretical question in optimization and sequential decision making. Previous results only establish a strongly polynomial-time algorithm for $L_\infty$ uncertainty sets. In this work, our main results are as follows: (a)~we show that for any compact uncertainty set, the policy iteration algorithm for RMDPs is strongly polynomial with oracle access to solutions of Robust Markov chains (RMCs); (b)~we present strongly polynomial-time bounds on the policy iteration algorithm for RMCs with $L_1$ and $L_\infty$ uncertainty sets; and (c)~we establish hardness results for RMCs with $L_p$ uncertainty sets for integer $p$ satisfying $1<p<\infty$. Finally, motivated by our theoretical bounds, we present experimental results showing how fast policy iteration converges for RMDPs with $L_1$ and $L_\infty$ uncertainty sets.

cs.CC↗

arXiv2026

Multi-Environment POMDPs with Finite-Horizon Objectives

Partially Observable Markov Decision Processes (POMDPs) are systems in which one agent interacts with a stochastic environment, and receives only partial information about the current state. In a multi-environment POMDP (MEPOMDP), the initial state is unknown, and assumed to be adversarially chosen. In this work we focus on computing the optimal value and policy in MEPOMDPs with finite-horizon objectives. That problem is known to be PSPACE-complete in POMDPs. Our main results are as follows: (1) we establish that it is also PSPACE-complete in the more general setting of MEPOMDPs; (2) we present a practical algorithm and evaluate it on classical benchmarks, significantly outperforming the only previously known algorithm.

cs.AI↗

arXiv2026

Genotype specificity and spatial arrangement govern the direction and magnitude of selection in variable environments

Spatial environmental variation can either amplify or suppress the fixation of beneficial mutants in structured populations, yet the interplay of ecological factors and spatial structure in determining which outcome occurs remains theoretically unresolved. Here, we develop a unified framework for selection on lattice graphs with environmental heterogeneity, in which mutant and resident fitness depend on the local environmental state. Across three common classes of genotype-environment interactions and a wide range of spatial arrangements of environmental states, we identify two governing principles. Genotype specificity determines the direction of the effect: heterogeneity amplifies selection when it modulates resident fitness, but suppresses selection when it modulates mutant fitness, with genotype-symmetric modulation producing weaker amplification. Spatial arrangement determines the magnitude: intermixed versus clustered environments tune the strength of amplification or suppression without reversing the direction of the effect. Together, these principles reconcile disparate theoretical results and provide predictive criteria for adaptation in heterogeneous landscapes, from microbial communities to somatic evolution and cancer.

q-bio.PE↗

arXiv2026

Predicting success of cooperators across arbitrary heterogeneous environmental landscapes

Cooperation is central to the organization of complex biological and social systems. Most theoretical models assume homogeneous environments; in reality, populations inhabit spatially varying landscapes in which the payoffs of cooperation differ across space. Here, we introduce a general framework for the evolution of cooperation in complex, heterogeneous environments where the benefit of cooperation depends on local environmental quality. Cooperators in environmentally rich sites confer greater benefits than those on poor sites. We show that whether heterogeneity promotes or suppresses cooperation is determined primarily by the spatial organization of environmental states. Across arbitrary environmental landscapes, a single quantity, the spatial correlation index (SCI), predicts the fixation probability of cooperators. Under weak selection, segregated environments enhance cooperation, whereas highly intermixed, checkerboard-like landscapes suppress it. Beyond fixation probabilities, environmental organization also controls evolutionary timescales: segregated landscapes generate long-lived metastable coexistence, whereas intermixed landscapes lead to faster but less successful fixation of cooperators. Together, these results provide a unifying description of how spatial environmental heterogeneity shapes the evolution of cooperation and suggest measurable predictors of cooperative success in biological and social settings.

q-bio.PE↗

arXiv2026

SuperDP: Differential Privacy Refutation via Supermartingales

Differential privacy (DP) has established itself as one of the standards for ensuring privacy of individual data. However, reasoning about DP is a challenging and error-prone task, hence methods for formal verification and refutation of DP properties have received significant interest in recent years. In this work, we present a novel method for automated formal refutation of $\epsilon$-DP. Our method refutes $\epsilon$-DP by searching for a pair of inputs together with a non-negative function over outputs whose expected value on these two inputs differs by a significant amount. The two inputs and the non-negative function over outputs are computed simultaneously, by utilizing upper expectation supermartingales and lower expectation submartingales from probabilistic program analysis, which we leverage to introduce a sound and complete proof rule for $\epsilon$-DP refutation. To the best of our knowledge, our method is the first method for $\epsilon$-DP refutation to offer the following four desirable features: (1)~it is fully automated, (2)~it is applicable to stochastic mechanisms with sampling instructions from both discrete and continuous distributions, (3)~it provides soundness guarantees, and (4)~it provides semi-completeness guarantees. Our experiments show that our prototype tool SuperDP achieves superior performance compared to the state of the art and manages to refute $\epsilon$-DP for a number of challenging examples collected from the literature, including ones that were out of the reach of prior methods.

cs.PL↗

arXiv2026

Approximating the Uniform Value in Hidden Stochastic Games with Doeblin Condition

We study \emph{zero-sum two-player hidden stochastic games}, where players receive partial observations of the state. We focus on a central solution concept for analyzing long-duration stochastic games: the \emph{uniform value}, a limiting average payoff that both players can guarantee for sufficiently long durations. In the general case, prior work provides examples of games that do not have a uniform value. Moreover, for the subclass of games that do have a uniform value, there exists no algorithm that approximates it. Therefore, we generalize the \emph{Doeblin condition} for Markov chains (which guarantees the existence of a unique invariant measure) to hidden stochastic games. Informally, the Doeblin condition for hidden stochastic games requires that, for every way to play the game, there exists a fixed belief such that, no matter the initial belief over the state of the game, after sufficiently many stages, the posterior belief is probably close to this fixed belief. Under the Doeblin condition, we prove the existence of the uniform value, provide an algorithm to approximate it, and prove that no algorithm can compute it exactly. Then, we identify structural conditions on the transition function that ensure the Doeblin condition holds both in the blind setting, where observations are uninformative, and in the hidden setting, where observations are partially informative. When considering games with only one player, namely partially observable Markov decision processes, our results provide a novel subclass in which the uniform value exists and can be approximated, but cannot be computed exactly

math.OC↗

arXiv2026

Strongly Polynomial Time Complexity of Policy Iteration for $L_\infty$ Robust MDPs

Markov decision processes (MDPs) are a fundamental model in sequential decision making. Robust MDPs (RMDPs) extend this framework by allowing uncertainty in transition probabilities and optimizing against the worst-case realization of that uncertainty. In particular, $(s, a)$-rectangular RMDPs with $L_\infty$ uncertainty sets form a fundamental and expressive model: they subsume classical MDPs and turn-based stochastic games. We consider this model with discounted payoffs. The existence of polynomial and strongly-polynomial time algorithms is a fundamental problem for these optimization models. For MDPs, linear programming yields polynomial-time algorithms for any arbitrary discount factor, and the seminal work of Ye established strongly--polynomial time for a fixed discount factor. The generalization of such results to RMDPs has remained an important open problem. In this work, we show that a robust policy iteration algorithm runs in strongly-polynomial time for $(s, a)$-rectangular $L_\infty$ RMDPs with a constant (fixed) discount factor, resolving an important algorithmic question.

cs.AI↗

arXiv2025

Monotone Near-Zero-Sum Games: A Generalization of Convex-Concave Minimax

Zero-sum and non-zero-sum (aka general-sum) games are relevant in a wide range of applications. While general non-zero-sum games are computationally hard, researchers focus on the special class of monotone games for gradient-based algorithms. However, there is a substantial gap between the gradient complexity of monotone zero-sum and monotone general-sum games. Moreover, in many practical scenarios of games the zero-sum assumption needs to be relaxed. To address these issues, we define a new intermediate class of monotone near-zero-sum games that contains monotone zero-sum games as a special case. Then, we present a novel algorithm that transforms the near-zero-sum games into a sequence of zero-sum subproblems, improving the gradient-based complexity for the class. Finally, we demonstrate the applicability of this new class to model practical scenarios of games motivated from the literature.

cs.GT↗

arXiv2025

Uniform Value and Decidability in Ergodic Blind Stochastic Games

We study a class of two-player zero-sum stochastic games known as \textit{blind stochastic games}, where players neither observe the state nor receive any information about it during the game. A central concept for analyzing long-duration stochastic games is the \textit{uniform value}. A game has a uniform value $v$ if for every $\varepsilon>0$, Player 1 (resp., Player 2) has a strategy such that, for all sufficiently large $n$, his average payoff over $n$ stages is at least $v-\varepsilon$ (resp., at most $v+\varepsilon$). Prior work has shown that the uniform value may not exist in general blind stochastic games. To address this, we introduce a subclass called \textit{ergodic blind stochastic games}, defined by imposing an ergodicity condition on the state transitions. For this subclass, we prove the existence of the uniform value and provide an algorithm to approximate it, establishing the \textit{decidability} of the approximation problem. Notably, this decidability result is novel even in the single-player setting of Partially Observable Markov Decision Processes (POMDPs). Furthermore, we show that no algorithm can compute the uniform value exactly, emphasizing the tightness of our result. Finally, we establish that the uniform value is independent of the initial belief.

math.OC↗

arXiv2025

Revealing POMDPs: Qualitative and Quantitative Analysis for Parity Objectives

Partially observable Markov decision processes (POMDPs) are a central model for uncertainty in sequential decision making. The most basic objective is the reachability objective, where a target set must be eventually visited, and the more general parity objectives can model all omega-regular specifications. For such objectives, the computational analysis problems are the following: (a) qualitative analysis that asks whether the objective can be satisfied with probability 1 (almost-sure winning) or probability arbitrarily close to 1 (limit-sure winning); and (b) quantitative analysis that asks for the approximation of the optimal probability of satisfying the objective. For general POMDPs, almost-sure analysis for reachability objectives is EXPTIME-complete, but limit-sure and quantitative analyses for reachability objectives are undecidable; almost-sure, limit-sure, and quantitative analyses for parity objectives are all undecidable. A special class of POMDPs, called revealing POMDPs, has been studied recently in several works, and for this subclass the almost-sure analysis for parity objectives was shown to be EXPTIME-complete. In this work, we show that for revealing POMDPs the limit-sure analysis for parity objectives is EXPTIME-complete, and even the quantitative analysis for parity objectives can be achieved in EXPTIME.

cs.CC↗