Continuous-Time Mean-Variance Portfolio Selection Reinforcement Learning Problems for a Markov Regime-Switching System with No-Shorting Constraints

Abstract

This paper is the first to study the continuous-time mean-variance portfolio selection problem with no-shorting constraints in a Markov regime-switching market using reinforcement learning (RL). The main challenge stems from the interaction between random regime shifts and portfolio non-negativity constraints, which typically causes the coupled Hamilton-Jacobi-Bellman (HJB) equations to lose smoothness across different state regions. Additionally, directly imposing no-shorting constraints on randomized policies disrupts their analytically tractable Gaussian structure, complicating both exploratory modeling and reinforcement learning. To address these challenges, we propose an auxiliary exploratory approach that limits the policy to a Gaussian distribution and applies the no-shorting constraint solely to its mean. We explicitly characterize the optimal policy for this auxiliary problem and demonstrate that, by appropriately selecting the exploration variance, its optimal mean policy exactly matches the optimal policy of the original constrained mean-variance problem. Building upon this analytical framework, we develop a model-free reinforcement learning algorithm that learns the optimal portfolio policy directly from trajectory data. Numerical experiments support the theoretical results and show that the proposed method can effectively recover the optimal no-shorting portfolio in a regime-switching market.