arXiv · 2611.10981
An Analysis of Reinforcement Learning with Function Approximation
Abstract
We address the problem of computing the optimal Q-function in Markov decision problems with infinite state-space. We analyze the convergence properties of several variations of Q-learning when combined with function approximation, extending the analysis of TD-learning in Tsitsiklis and Van Roy (1996) to stochastic control settings. We identify conditions under which such approximate methods converge with probability 1. We conclude with a brief discussion on the general applicability of our results and compare them with several related works.
Explore related subjects
Keep this discovery
Explore connections, maps & timelines
Francisco S. Melo, Sean P. Meyn, M. Isabel Ribeiro. 2026-11-07. An Analysis of Reinforcement Learning with Function Approximation. https://doi.org/10.1145/1390156.1390240. https://arxiv.science/abs/2611.10981
Citations retain the original authors and DOI and include this site’s record URL. If this discovery service contributed to your research, you can also cite arXiv Science by Indiciae: BibTeX · RIS.