Random-subspace Ellipsoidal Trust-region Methods for Derivative-free Optimization

Abstract

Random-subspace model-based methods offer a promising approach to solving derivative-free optimization (DFO) problems in high dimensions. However, the usual ratio of actual to predicted reduction assesses model agreement at the trial point without directly measuring subspace quality. An accurate prediction in a poorly aligned subspace may therefore trigger trust-region expansion in all directions. We introduce Random-subspace Ellipsoidal TRust-region Optimization (RETRO), a framework for unconstrained DFO using ellipsoidal trust regions. Its trust-region updates combine global size changes with additional expansion or contraction within the sampled subspace, reflecting where the trial step was computed. We define an ellipsoidal subspace alignment measure that accounts for both the subspace and trust-region shape, and analyze Haar-uniform sampling and metric-guided sampling based on the current shape matrix. We establish almost-sure first-order liminf convergence and an $\mathcal{O}(ε^{-2})$ bound on the expected number of iterations required to reach an iterate with gradient norm at most $ε$. Numerical experiments on projected low-rank problems in dimension $1000$ show that using ellipsoidal shape information in subspace selection can reduce function evaluation costs relative to ball-based QARSTA, with the most pronounced improvements occurring when the solver subspace dimension matches the projection rank.