arXiv Science⌕ Search
Home/Archive/arXiv

arXiv · 2601.13045

Markovian restless bandits and index policies: A review

José Niño-Mora
View original source ↗Read source document ↗Export metadata ↓

Abstract

The restless multi-armed bandit problem is a paradigmatic modeling framework for optimal dynamic priority allocation in stochastic models of wide-ranging applications that has been widely investigated and applied since its inception in a seminal paper by Whittle in the late 1980s. The problem has generated a vast and fast-growing literature from which a significant sample is thematically organized and reviewed in this paper. While the main focus is on priority-index policies due to their intuitive appeal, tractability, asymptotic optimality properties, and often strong empirical performance, other lines of work are also reviewed. Theoretical and algorithmic developments are discussed, along with diverse applications. The main goals are to highlight the remarkable breadth of work that has been carried out on the topic and to stimulate further research in the field.

Explore related subjects

math.GM

Keep this discovery

Explore connections, maps & timelines

BibTeXRIS

José Niño-Mora. 2026-01-19. Markovian restless bandits and index policies: A review. https://doi.org/10.3390/math11071639

Cite the original work for its findings. Save a collection to share your selection of sources.

KEEP EXPLORING

Related papers

arXiv2026

On the convergence of a perturbed one dimensional Mann's process

We study a perturbed version of Mann's iterative process (DMP), defined by \[ x_{n+1} = (1 - θ_n)x_n + θ_n f(x_n) + r_n, \] where $f: [0,1] \to [0,1]$ is a continuous function, $\{θ_n\} \subset [0,1]$ is a given sequence, and $\{r_n\} $ represents an error term. We prove that if the sequence $\{θ_n\} $ converges sufficiently slowly to zero and the error term $ r_n $ is suitably small at infinity, then any sequence $\{x_n\} \subset [0,1] $ generated by this process converges to a fixed point of $f$. In addition, we investigate the asymptotic behavior of the trajectories $ x(t) $ as $ t \to \infty$ for a continuous-time version of the process (DMP). We emphasize the parallels between the discrete and the continuous dynamics. Furthermore, through numerical experiments, we analyze the influence of the sequence $\{θ_n\} $ and the error terms on the stability and the convergence rate of the the discrete and the continuous processes. Notably, we observe that the (DMP) algorithm, when affected by stochastic and relatively large error terms, can outperform the bisection method in efficiently identifying the fixed point set of the function $f$.

math.GM↗
arXiv2026

Explicit formula for the discrete Laplace transform of the Möbius function, related special functions, and a criterion for the Riemann hypothesis

In this paper, we assume that all the zeros of the Riemann zeta function are simple. Under this assumption we give an explicit formula for the function $Φ(e^{-t})=\sum_{n=1}^{\infty}μ(n)e^{-nt}$, as a function of the values of $ζ(s)$ and $ζ'(s)$ at the odd integers and as a function of the zeros of $ζ(s)$. A structural feature distinguishes this formula from the classical explicit formula for the Mertens function: the poles of $Γ(s)$ collide with the trivial zeros of $ζ(s)$, producing double poles whose residues contain a logarithmic term. Using this formula, we give a criterion for the Riemann hypothesis: the bound $O(x^{-1/2})$ on the transform implies the Riemann hypothesis unconditionally, while the converse direction requires additional hypotheses on the zeros. We also introduce special entire functions related to $ζ(s)$ and show that they admit absolutely convergent closed forms as Möbius-weighted series of Bessel functions of rotated argument.

math.GM↗
arXiv2026

There exist blow-ups in the incompressible Navier-Stokes equation

In this paper, we solve the Navier-Stokes equation, one of the seven Millennium Prize Problems suggested by Clay Mathematics Institute(CMI). In our paper, the fluid is confined in a solid sphere. We prove that, for any initial velocity u0, there has been a force vector f , such that there exists no smooth solution (p,u) to the respective Navier-Stokes equation. This result also holds for the Euler equation. The paper was first published in 2021, soon after, Professor PG Lemarié-Rieusset contact me and tell that there is a possible flaw, because u may be non-integrable when it decreases not very fast. Hence, I withdraw the paper. Through carefull consideration of several years, we think we can choose a fluid confined in a solid sphere to fix it. So, we update the paper now.

math.GM↗