Law-Dependent Poisson-Refresh Randomised Feedback Control for Stochastic Systems

Randa Herzallah

Home/Archive/arXiv

arXiv · 2611.06252

Law-Dependent Poisson-Refresh Randomised Feedback Control for Stochastic Systems

Randa Herzallah

View original source ↗Read source document ↗Export metadata ↓

Abstract

This paper studies a continuous-time probabilistic control problem for stochastic systems in which the feedback law assigns a probability distribution over control inputs. The control is modelled as a law-dependent Poisson-refresh process: it is held constant between refresh times and resampled at Poisson times from a feedback rule depending on the current state and state law. The closed-loop system is therefore formulated as a jump--diffusion for the joint state--control process. The objective is to design the sampling rule by comparing the realised closed-loop system with a prescribed reference description. The cost separates two local terms: a relative-entropy penalty for the instantaneous refresh kernel and a state-side drift-mismatch cost. The state-side term is evaluated at the realised control value and is motivated by a frozen-control Girsanov identity. Under global Lipschitz, moment, and uniform ellipticity assumptions, we prove well-posedness of the closed loop and a dynamic programming principle on the joint state--control law space. For linear data and admissible mean-dependent affine-Gaussian refresh rules, the full joint law need not remain Gaussian, but the first and second moments of the joint state--control process close exactly. This yields a deterministic reduced control problem on moment space, with an explicit running cost, a reduced Hamiltonian, and a verification theorem.

Explore related subjects

math.OC

Keep this discovery

Explore connections, maps & timelines

BibTeX RIS

Randa Herzallah. 2026-11-04. Law-Dependent Poisson-Refresh Randomised Feedback Control for Stochastic Systems. https://arxiv.org/abs/2611.06252

Cite the original work for its findings. Save a collection to share your selection of sources.

KEEP EXPLORING

Related papers

arXiv2026

Distributionally Robust Airport Ground Holding Problem under Wasserstein Ambiguity Sets

We develop a distributionally robust formulation of the single-airport ground holding problem (dr-SAGHP) using a $1$-Wasserstein ambiguity set with an $\ell_1$ transportation metric over the full physical capacity support. We exploit the problem's structural properties to develop an exact dynamic programming separation algorithm within delayed constraint generation (DCG-DP). We also develop K-IL, which combines Kelley's cutting-plane method with an integer L-shaped framework to solve a a restricted-support approximation, rs-dr-SAGHP. Delayed constraint generation with MILP separation (DCG-MILP) serves as the computational baseline. Computational experiments demonstrate substantial runtime reductions for DCG-DP and K-IL relative to this baseline. Airport capacity distributions are constructed using Gaussian process regression based on historical weather and operational data at Denver International Airport from 2021 to 2023. We develop a chronological expanding-window calibration procedure to select the Wasserstein radius and the results show that the calibrated dr-SAGHP approaches reduce mean operational cost by approximately $2.6\%$ and the Conditional Value at Risk $\mathrm{CVaR}_{90}$ by approximately $8\%$ relative to the stochastic SAGHP benchmark, while avoiding the excessive conservatism observed in a classical robust optimization benchmark.

math.OC↗

arXiv2026

Q-Learning under Finite Model Uncertainty

We propose a robust Q-learning algorithm for Markov decision processes under model uncertainty when each state--action pair is associated with a finite ambiguity set of candidate transition kernels. This finite-measure framework enables user-designed uncertainty models, including competing fitted models and stress scenarios. {The learner observes successor states and their deterministic rewards from every candidate simulator, without requiring explicit knowledge of the transition probabilities or the reward function.} We establish almost sure convergence of the learned Q-function to the robust optimum, and derive a time-uniform high-probability error certificate that separates an observable empirical Bellman residual from transition-kernel estimation error. We also give quantitative finite-ambiguity approximation results for Wasserstein and parametric uncertainty models on finite state spaces. Numerical examples illustrate the effect of scenario selection and compare the tabular algorithm with robust value iteration.

math.OC↗

arXiv2026

SAGE: A Set-based Adaptive Gradient Estimator

We introduce a set membership perspective on gradient estimation for black-box scalar functions, in which the gradient is not approximated as a single point estimate but characterised as a member of a set of admissible gradients consistent with available function samples and regularity assumptions. Bounds on gradient uncertainty, obtained from standard multivariate Taylor series results, are used to represent this admissible set through linear inequalities, making gradient uncertainty explicit and directly exploitable. Building on this formulation, we propose an adaptive strategy to refine the admissible gradient set to a desired precision by selectively acquiring new samples only when necessary. The resulting framework naturally accommodates function evaluations affected by noise with finite but a priori unknown bounds, yields the theoretical best attainable precision of the gradient set, and provides the optimal sampling distance from the point of interest to maximise information gain during refinement. Within this framework, we introduce the Set-Based Adaptive Gradient Estimator (SAGE), which extracts an estimate of the true gradient from the admissible set whilst preserving high sample efficiency and robustness to noise. Numerical experiments on large collections of random gradient-query points, as well as in the context of a first-order optimisation method, compared the performance of SAGE against commonly used gradient estimators from the literature. Our results show that SAGE achieved competitive gradient accuracy across noise levels without requiring any a priori noise characterisation. In a first-order optimisation setting, noise-informed competitors can outperform SAGE when such characterisation is available. Nevertheless, SAGE remained broadly competitive, making it particularly attractive in settings where noise characterisation is impractical to obtain.

math.OC↗