arXiv Science⌕ Search
Home/Archive/arXiv

arXiv · 2610.20126

New Modelling and Solution Frameworks for Multiple Objectives, Decomposability and Impulsivity in Continuous-time Markov Decision Processes

Luke Fairley·Rob Shone·Peter Jacko·Jefferson Huang
View original source ↗Read source document ↗Export metadata ↓

Abstract

Impulsive control, in the context of continuous-time Markov decision processes (CTMDPs), refers to actions that instantaneously update the state of the process, leading to theoretical complications. Previous work has overcome these complications, but at the expense of approachability and ease of implementation: those models are mathematically complex and incompatible with pre-existing solution methods. This paper presents an alternative formulation of impulsive control for CTMDPs that is easier to employ in the modelling of real-world problems and can be solved by pre-existing solution methods. This formulation is then extended to allow for multiple objectives and decomposability. A decomposition-based heuristic method is developed that leverages Dantzig-Wolfe decomposition to find approximate Pareto fronts for large multi-objective decomposable impulsive CTMDPs. The modelling framework and solution methodology are applied to a dynamic maintenance problem to demonstrate their computational efficiency, solution space coverage, and solution quality.

Explore related subjects

math.OC

Keep this discovery

Explore connections, maps & timelines

BibTeXRIS

Luke Fairley, Rob Shone, Peter Jacko, Jefferson Huang. 2026-10-14. New Modelling and Solution Frameworks for Multiple Objectives, Decomposability and Impulsivity in Continuous-time Markov Decision Processes. https://arxiv.org/abs/2610.20126

Cite the original work for its findings. Save a collection to share your selection of sources.

KEEP EXPLORING

Related papers

arXiv2026

A Homotopy Framework for Constrained Multiobjective Optimization

We develop a homotopy continuation framework for computing Karush--Kuhn--Tucker (KKT) points of constrained multiobjective optimization problems. The proposed homotopy map continuously deforms an easily solvable system into the KKT system associated with the original multiobjective problem, yielding a deterministic continuation path. Under the stated regularity and reference-point cone avoidance assumptions, we establish the existence of a smooth, bounded continuation path whose set of accumulation points as the homotopy parameter approaches zero is nonempty and consists of KKT points. The framework is implemented using two numerical continuation algorithms including an enhanced pseudo-arclength predictor--corrector strategy with adaptive step-size control and branch-restarting mechanisms to improve robustness for challenging nonconvex and degenerate problems. Numerical experiments demonstrate that the proposed method remains effective even when initialized from nonfeasible points, indicating practical robustness beyond the theoretical guarantees. Comparative studies evaluate the proposed framework against classical scalarization approaches and the evolutionary algorithms. Experiments on benchmark problems, disconnected and nonconvex multiobjective problems, analytically solvable three-objective problems, and a multiobjective IEEE 5-bus optimal power flow problem demonstrate the ability of the framework to compute KKT points in a variety of constrained settings. The results also highlight the effects of initialization and the limitations of continuation-based methods in recovering complete Pareto-front coverage. Overall, the proposed framework provides a deterministic and interpretable approach to computing Pareto-critical (KKT) points in constrained multiobjective optimization.

math.OC↗
arXiv2026

Averaged proximal reflected gradient method for monotone variational inequalities

Projected reflected gradient (PRG) method proposed by Malitsky is efficient for solving monotone variational inequality (MVI), while the existing upper bound of step size is not tight due to the inequality scaling in the theoretical analysis. In this paper, we construct an averaged variant of PRG method for more general MVI and present a novel Lyapunov function to establish convergent theory. This averaged PRG method provides an improvement of the golden ratio algorithm [Y, Malitsky, Math. Program., 184, 383-410, 2020], and the involved step size is compatible with that for the classical methods, such as Popov's extragradient and forward-reflected-backward methods. Moreover, a fully adaptive strategy without linesearch is presented to adjust step sizes, which generates closed-form and potentially much larger step sizes. Numerical experiments on the Nash-Cournot equilibrium, HpHard, and image reconstruction problems demonstrate that the proposed algorithm significantly outperforms existing state-of-the-art methods.

math.OC↗
arXiv2026

When is a better LLM worth paying for? Stochastic Optimization with Switching Mixed-Fidelity Oracles

Large language models (LLMs) are increasingly used upstream of decision-making to clean data, detect label errors, and reconstruct missing values. Yet LLM-assisted data curation creates a new operational tradeoff: higher-fidelity models and richer prompts can be substantially more expensive, while the value of this additional fidelity depends on the downstream optimization accuracy. This raises the central question of this work: when should one pay for a better LLM, given a specific downstream task at hand? We study this question through stochastic optimization with mixed-fidelity first-order oracles, which can have systematic bias and state-dependent variance. We develop two cost-aware algorithmic frameworks that allow adaptive switching between stochastic oracles. With known bias, the first meta-algorithm applies a restarted, cost-aware policy that adaptively selects oracle fidelity across optimization stages. Its cumulative cost is no larger than that of the best fixed oracle in hindsight and can be substantially smaller. With unknown bias, we develop a budgeted certification scheme that uses occasional trusted-gradient evaluations to certify the optimization progress and eliminate unsuccessful oracle sources, and show an explicit tradeoff between certification cost and delayed detection of biased sources. We provide concrete realizations of these meta-algorithms by applying biased stochastic gradient extrapolation (SGE) and recursive regularization under state-dependent noise as subroutines. Synthetic and LLM-completion experiments illustrate the qualitative switching behavior predicted by the theory: the adaptive policy skips several expensive LLM oracles and substantially reduces cumulative cost relative to fixed high-fidelity alternatives while achieving comparable accuracy.

math.OC↗