arXiv subjects

Akiko Takeda

Publications and source records attributed to Akiko Takeda.

At least 19 recordsLinked to original sources

arXiv2026

Gradient Descent with Stochastic Subspaces via Persistence of Memory

Stochastic subspace methods have gained popularity as gradient descent based techniques for large scale optimisation problems, especially in distributed settings. In this paper, we introduce the technique of "persistence of memory" to greatly extend and improve the random subspace methods. To this end, we leverage a vector that is only weakly correlated with the gradient in order to provide a guiding structure to the generative process of the random subspace along which the descent is going to take place. This guidance vector may be fixed for a large number of iterations, only to be refreshed at wide intervals (on whose size we can provide guarantees in terms of problem parameters). In important machine learning settings, such as optimisation problems embodying sparsity or a minibatch structure, we show that the guidance vector can be obtained in an effective and computationally inexpensive manner by leveraging the structured properties of the problem. En route, we establish to our knowledge the first theoretical analysis of classical SSD methods for sparse functions. In a local neighbourhood of the optimum, we demonstrate an alignment phenomenon of our gradient estimates with a low-lying eigenvector of the Hessian, allowing a once-for-all computation of the guidance vector which renders the method computationally favourable even in scenarios with unstructured objectives.

math.OC↗

arXiv2026

A Splitting Framework for Composite Semimonotone Inclusions

We introduce a general framework for composite inclusion problems with affine constraints, covering both monotone and semimonotone regimes. The central idea is a new interpretation of the constrained inclusion through an operator-vector pair that separates the implicit inclusion from the affine constraint: the former is handled through possibly preconditioned resolvent evaluations, while the latter is handled through an explicit forward step in an auxiliary variable. This yields a single abstract iteration applicable to multioperator inclusions, linearly coupled inclusions, and block-separable inclusions with affine constraints. The freedom in choosing the operator-vector pair enables the systematic construction of problem-adapted splitting algorithms, including new schemes for several important problem classes. Exploiting the orthogonal decomposition induced by the constraint subspace, we develop a unified and streamlined convergence analysis and establish weak and strong convergence guarantees under semimonotonicity assumptions. When specialized to multioperator inclusions, the framework permits general bounded linear operator coefficients, rather than only scalar coefficients, and therefore accommodates preconditioned resolvents. The resulting schemes recover several existing methods while extending them to previously uncovered regimes, and in several important cases, require weaker assumptions and admit provably larger admissible parameter ranges.

math.OC↗

arXiv2026

Adaptive Gradient-Based Methods for a Broader Class of Optimization Problems under Performative Prediction

We study optimization under performative prediction, where deploying a model affects the future data distribution. For this setting, several gradient-based approaches have been proposed. However, they typically assume specific data distributions or loss functions, which limit their practical applicability. To overcome these limitations, we propose a gradient-based optimization method with convergence guarantees under substantially weaker assumptions. Our method explicitly estimates the induced distribution shift through finite differences. It enables higher-dimensional optimization across broader classes of loss functions and data distributions. We also propose a practical variant that reduces the number of samples required. Numerical experiments demonstrate that our proposed algorithms converge faster and more consistently than existing ones.

math.OC↗

arXiv2026

A New Dynamic Pricing Model based on Convex Hull Pricing

This paper presents a new dynamic pricing model (a.k.a. real-time pricing) that reflects startup costs of generators. Dynamic pricing, which is a method to control demand by pricing electricity at hourly (or more often) intervals, has been studied by many researchers. They assume that the cost functions of suppliers are convex, although they may be nonconvex because of the startup costs of generators in practice. We provide a dynamic pricing model that takes into account such cost functions within the settings of unit commitment problems (UCPs). Our model gives convex hull price (CHP), which has not been used in the context of dynamic pricing, though it is known that the CHP minimizes the uplift payment which is disadvantageous to suppliers for a given demand. In addition, we apply an iterative algorithm based on the subgradient method to solve our model. Numerical experiments show the efficiency of our model on reducing uplift payments. The prices determined by our algorithm give sufficiently small uplift payments in a realistic computational time.

math.OC↗

arXiv2026

Parameter-Free Cubic-Regularized Newton Method: Sharp Complexity and Generalized Smoothness

We analyze a variant of the cubic-regularized Newton method for nonconvex optimization. This variant is parameter-free in that it requires no prior knowledge of problem-dependent parameters. Under the generalized smoothness condition $\|\nabla^3 f(x)\| \leq L_0 + L_1 \|\nabla f(x)\|$, we derive an oracle complexity bound for finding an $(\varepsilon, δ)$-second-order stationary point. This assumption is weaker than the generalized smoothness conditions used in existing analyses of second-order methods, while the complexity bound improves upon existing guarantees for parameter-free second-order methods. In particular, when $L_1 = 0$, the bound matches the optimal dependence on $L_0$ as well as on $\varepsilon$, $δ$, and the initial function value gap, up to additive logarithmic terms. To establish this bound, we derive Taylor-type inequalities and prove their equivalence to the generalized smoothness condition.

math.OC↗

arXiv2026

Change Point Detection in Precision Matrices with D-trace Loss

We consider the problem of estimating a time-varying sparse precision matrix, which is assumed to evolve in a piecewise constant manner. Building upon the Group Fused LASSO and LASSO penalty functions, we estimate both the precision matrix and the change points. We propose an alternative estimator to the commonly employed Gaussian likelihood loss, namely the D-trace loss. We provide the conditions for the consistency of the estimated change points and of the sparse estimators in each block. We show that the solutions to the corresponding estimation problem exist when some conditions relating to the tuning parameters of the penalty functions are satisfied. Unfortunately, these conditions are not verifiable in general, posing challenges for tuning the parameters in practice. To address this issue, we introduce a modified regularizer and develop a revised problem that always admits solutions: these solutions can be used for detecting possible unsolvability of the original problem or obtaining a solution of the original problem otherwise. An alternating direction method of multipliers (ADMM) is then proposed to solve the revised problem. The relevance of the method is illustrated through numerical experiments.

math.ST↗

arXiv2026

Initial Placement for Fruchterman--Reingold Force Model With Coordinate Newton Direction

The Fruchterman--Reingold (FR) force model is widely used in force-directed graph drawing, and multilevel approaches such as sfdp in Graphviz scale these methods effectively. A crucial step in multilevel schemes is refinement, which improves the graph layout, typically performed with the simulation-based algorithm. For this refinement, we can utilize optimization-based methods such as L-BFGS, or combine them with initial placement methods such as Simulated Annealing to achieve better layouts. However, they have several limitations, such as suffering from high per-iteration costs for large graphs or having difficulty with weighted and structurally complex graphs, leaving room for improvement. In this research, we propose a new initial placement based on stochastic coordinate descent to accelerate the optimization process. We first reformulate the problem as a discrete optimization problem using a hexagonal lattice and then iteratively update a randomly selected vertex along the coordinate Newton direction with low per-iteration costs. We demonstrate the effectiveness of our method through numerical experiments, showing that our initial placement leads to faster convergence and higher-quality layouts compared to naive optimization approaches. We also discuss applications of our method, such as drawing for Hooke--Coulomb and Eades force models.

cs.CG↗

arXiv2026

Projection onto hyperbolicity cones and beyond: a dual Frank-Wolfe approach

We discuss the problem of projecting a point onto an arbitrary hyperbolicity cone from both theoretical and numerical perspectives. While hyperbolicity cones are furnished with a generalization of the notion of eigenvalues, obtaining closed form expressions for the projection operator as in the case of semidefinite matrices is an elusive endeavour. To address that we propose a Frank-Wolfe method to handle this task and, more generally, strongly convex optimization over closed convex cones. One of our innovations is that the Frank-Wolfe method is actually applied to the dual problem and, by doing so, subproblems can be solved in closed-form using minimum eigenvalue functions and conjugate vectors. To test the validity of our proposed approach, we present numerical experiments where we check the performance of alternative approaches including interior point methods and an earlier accelerated gradient method proposed by Renegar. We also show numerical examples where the hyperbolic polynomial has millions of monomials. Finally, we also discuss the problem of projecting onto p-cones which, although not hyperbolicity cones in general, are still amenable to our techniques.

math.OC↗

arXiv2026

A General Recipe for Parameter-Free Nonconvex Optimization via Higher-Order Regularization

We develop a systematic framework for constructing parameter-free algorithms for smooth nonconvex optimization. The framework is based on higher-order regularization: each step is computed from a regularized local model whose regularization exponent exceeds the order of the model error. This design makes the resulting method robust to misspecification of the regularization parameter and yields complexity bounds without backtracking or other acceptance tests. We apply the framework to gradient descent, Newton's method, the Gauss--Newton method, stochastic gradient descent, and PAGE. Without prior knowledge of problem-dependent parameters, the resulting algorithms achieve complexity bounds with optimal or best-known dependence on the target accuracy. When the problem-dependent parameters are known up to constant factors, suitable tuning also recovers the optimal or best-known dependence on those parameters.

math.OC↗

arXiv2026

Random-Subspace Frank--Wolfe over Strongly Convex Sets

Frank--Wolfe methods avoid projections, but over curved feasible regions the full-space linear minimization oracle (LMO) can itself become the computational bottleneck. We introduce random-subspace Frank--Wolfe (RSFW), the first Frank--Wolfe framework, to our knowledge, that replaces the ambient LMO by exact LMOs over random low-dimensional affine sections of a general feasible set, while preserving feasibility in the original space. For smooth convex objectives over compact strongly convex feasible sets, we prove a dimension-explicit approximate-oracle inequality and derive the standard $O(1/k)$ open-loop rate, with high-probability and almost-sure counterparts. Under short steps and a gradient lower bound, the same geometric control yields linear convergence, and we extend the sublinear theory to finite-sum stochastic gradients. We also show that random sections can improve the local curvature model controlling short steps: for smooth objectives, the quadratic model along a sampled section is governed by the compressed Hessian, yielding computable $d\times d$ curvature constants for quadratic objectives over balls and ellipsoids. These results provide a geometric theory of oracle-side randomization in projection-free optimization.

math.OC↗

arXiv2026

A primal-dual interior point trust region method for second-order stationary points of Riemannian inequality-constrained optimization problems

We consider Riemannian inequality-constrained optimization problems. Such problems inherit the benefits of Riemannian approach developed in the unconstrained setting and naturally arise from applications in control, machine learning, and other fields. We propose a Riemannian primal-dual interior point trust region method (RIPTRM) for solving them. We prove its global convergence to an approximate Karush-Kuhn-Tucker point and a weak second-order stationary point. Under the strict complementarity condition, this result reduces to global convergence to a second-order stationary point. To the best of our knowledge, this is the first algorithm that incorporates the trust region strategy for constrained optimization on Riemannian manifolds, and has the second-order convergence property for optimization problems on Riemannian manifolds with nonlinear inequality constraints. We conduct numerical experiments in which we introduce a truncated conjugate gradient method and an eigenvalue-based subsolver for RIPTRM to approximately and exactly solve the trust region subproblems, respectively. Empirical results show that RIPTRMs consistently find solutions with high accuracy. Additionally, we observe that RIPTRM with the exact search direction shows promising performance in an instance where the Hessian of the Lagrangian has a large negative eigenvalue.

math.OC↗

arXiv2026

Local near-quadratic convergence of Riemannian interior point methods

We consider Riemannian optimization problems with inequality and equality constraints and analyze a class of Riemannian interior point methods for solving them. The algorithm of interest consists of outer and inner iterations. We show that, under standard assumptions, the algorithm achieves local superlinear convergence by solving a linear system at each outer iteration, removing the need for further computations in the inner iterations. We also provide a specific update for the barrier parameter that achieves local near-quadratic convergence of the algorithm. We apply our results to the method proposed by Obara, Okuno, and Takeda (2026) and show its local superlinear and near-quadratic convergence with an analysis of the second-order stationarity. To our knowledge, this is the first algorithm for constrained optimization on Riemannian manifolds that achieves both local convergence and global convergence to a second-order stationary point. Numerical results support the theoretical analyses of the proposed methods.

math.OC↗

arXiv2026

Randomized Subspace Nesterov Accelerated Gradient

Randomized-subspace methods reduce the cost of first-order optimization by using only low-dimensional projected-gradient information, a feature that is attractive in forward-mode automatic differentiation and communication-limited settings. While Nesterov acceleration is well understood for full-gradient and coordinate-based methods, obtaining accelerated methods for general subspace sketches that use only projected-gradient information and can improve over full-dimensional Nesterov acceleration in oracle complexity is technically nontrivial. We develop randomized-subspace Nesterov accelerated gradient methods for smooth convex and smooth strongly convex optimization under matrix smoothness and generic sketch moment assumptions. The key technical ingredient is a three-sequence formulation tailored to matrix smoothness, which recovers the corresponding classical Nesterov methods in the full-dimensional case. The resulting theory establishes accelerated oracle-complexity guarantees and makes explicit how matrix smoothness and the sketch distribution enter the complexity. It also provides a unified basis for comparing sketch families and identifying when randomized-subspace acceleration improves over full-dimensional Nesterov acceleration in oracle complexity.

math.OC↗

arXiv2026

Inexact subgradient algorithm with a non-asymptotic convergence guarantee for copositive programming problems

In this paper, we propose a subgradient algorithm with a non-asymptotic convergence guarantee to solve copositive programming problems. The subproblem to be solved at each iteration is a standard quadratic programming problem, which is NP-hard in general. However, the proposed algorithm allows this subproblem to be solved inexactly. For a prescribed accuracy $ε> 0$ for both the objective function and the constraint arising from the copositivity condition, the proposed algorithm yields an approximate solution after $O(ε^{-2})$ iterations, even when the subproblems are solved inexactly. We also discuss exact and inexact approaches for solving standard quadratic programming problems and compare their performance through numerical experiments. In addition, we apply the proposed algorithm to the problem of testing complete positivity of a matrix and derive a sufficient condition for certifying that a matrix is not completely positive. Experimental results demonstrate that we can detect the lack of complete positivity in various doubly nonnegative matrices that are not completely positive.

math.OC↗

arXiv2026

Approximate Bregman proximal gradient algorithm with variable metric Armijo--Wolfe line search

We propose a variant of the approximate Bregman proximal gradient (ABPG) algorithm for minimizing the sum of a smooth nonconvex function and a nonsmooth convex function. ABPG is known to converge globally to a stationary point even when the smooth part of the objective function does not have a globally Lipschitz continuous gradient, and its iterates can often be expressed in closed form. However, ABPG relies on an Armijo line search to guarantee global convergence, which can slow down its practical performance. To address this issue, we propose a variant of ABPG with a variable metric Armijo--Wolfe line search. Under the variable metric Armijo--Wolfe condition, we establish global subsequential convergence of the algorithm. Moreover, assuming the Kurdyka--Łojasiewicz property, we also prove that the algorithm globally converges to a stationary point. Numerical experiments on $\ell_p$-regularized least squares problems and nonnegative linear inverse problems demonstrate that the proposed algorithm outperforms existing algorithms.

math.OC↗

arXiv2026

Practical Regularized Quasi-Newton Methods with Inexact Function Values

Many practical optimization problems involve objective function values that are corrupted by unavoidable numerical errors. In smooth nonconvex optimization, quasi-Newton methods combined with line search are widely used due to their efficiency and scalability. These methods implicitly assume accurate function evaluations and thus may fail to converge in noisy settings. Developing fast and robust quasi-Newton methods for such scenarios is therefore crucial. To address this issue, we propose a noise-tolerant regularized quasi-Newton method equipped with a relaxed Armijo-type line search, designed to remain stable under inaccurate function evaluations. By combining a regularization parameter update rule inspired by Objective-Function-Free Optimization and the AdaGrad-Norm method, we establish a global convergence rate of $\mathcal{O}(1/\varepsilon^2)$ for reaching a first-order stationary point under the assumed error model. We performed extensive experiments on the CUTEst benchmark collection with artificially noisy objective function evaluations, as well as with low-precision floating-point arithmetic (64-, 32-, and 16-bit). The results demonstrate that the proposed method is substantially more robust than several existing methods, while maintaining competitive practical convergence speed and computational cost.

math.OC↗

arXiv2026

Fairness in Robust Unit Commitment Problem Considering Suppression of Renewable Energy

Power company operators make power generation plans one day in advance, in what is known as the Unit Commitment (UC) problem. UC is exposed to uncertainties, such as unknown electricity load and disturbances caused by renewable energy sources, especially PVs. In previous research, we proposed the Renewable Energy Robust Optimization Problem (RE-RP), which solves these uncertainties by considering suppression. In this paper, we propose a new model called RE-RP with fairness (RE-RPfair), which aims to achieve fair allocation among PVs allocation. This model is an expansion of the original RE-RP, and we prove its effectiveness through simulation. To measure the degree of fairness, we use the Gini Index, which is well-known in social science.

math.OC↗

arXiv2026

Fast Frank--Wolfe Algorithms with Adaptive Bregman Step-Size for Weakly Convex Functions

We propose Frank--Wolfe (FW) algorithms with an adaptive Bregman step-size strategy for smooth adaptable (also called: relatively smooth) (weakly-) convex functions. This means that the gradient of the objective function is not necessarily Lipschitz continuous, and we only require the smooth adaptable property. Compared with existing FW algorithms, our assumptions are less restrictive. We establish convergence guarantees in various settings, including convergence rates ranging from sublinear to linear, depending on the assumptions for convex and nonconvex objective functions. Assuming that the objective function is weakly convex and satisfies the local quadratic growth condition, we provide both local sublinear and local linear convergence with respect to the primal gap. We also propose a variant of the away-step FW algorithm using Bregman distances over polytopes. We establish faster global convergence (up to a linear rate) for convex optimization under the Hölder error bound condition and local linear convergence for nonconvex optimization under the local quadratic growth condition. Numerical experiments demonstrate that our proposed FW algorithms outperform existing methods.

math.OC↗