arXiv Science⌕ Search

arXiv subjects

Andersen Ang

Publications and source records attributed to Andersen Ang.

11 recordsLinked to original sources

Lean-PEP: Performance-Estimation Certificates to Formally Verified Algorithmic Bounds

Performance-estimation certificates describe worst-case bounds for optimization algorithms. Turning a certificate into a formal theorem requires exact verification of its algebra and a connection between its coordinates, constraints and objective and the intended function-level problem. Lean-PEP provides a reusable interface for this task. It checks a submitted source and exact certificate against a supported canonical finite model, and a parameterized analytic theorem transfers the resulting finite bound to every admissible algorithm trajectory. Generated computation proofs discharge the obligations that depend on each certificate. An additional capture entry checks the numerical record and reconstruction policy. The implemented two-step proximal-gradient consumer verifies a new input with normalized bound 62500/499999, and a fixed eight-configuration batch yields seven distinct payloads with accepted certificates and executed Lean proofs. PG/GD cases and one-step attainment witnesses establish the corresponding algorithmic conclusions, and a fresh installation reproduces consumption of the supplied input. These results show how new certificate data can reuse a proved analytic interface to obtain function-level bounds within the supported models.

cs.LO↗

T-ARC: Topology-Aware Randomized Clustering via Distributionally Robust Stochastic Block Models

In this work, we introduce a new clustering method, namely T-ARC (Topology-Aware Randomized Clustering), that corrects the geometric bias of K-means by embedding topological information directly into the optimization objective. Building on the assumption that the data admits an underlying hidden structure modeled via a latent graph, the idea is to uncover this information through the interplay between the standard K-means data-fidelity term and a graph-cut penalty, which discourages cluster assignments inconsistent with the connectivity structure of the data. To render this coupling tractable, the latent graph is modeled as a random realization from a Stochastic Block Model (SBM), whose scalar parameter is optimized within a Distributionally Robust Optimization (DRO) framework, yielding a closed-form proximal update. Both SBM and DRO are informed by a persistence-based similarity matrix derived from zero-dimensional persistent homology ($H_0$), which translates the multiscale connectivity structure of the data into a pairwise topological prior. The overall optimization proceeds via Block Coordinate Descent; convergence is established through a global Lyapunov functional: the deterministic blocks satisfy monotonic descent, while the stochastic graph update satisfies descent in expectation, so that the expected energy converges. Experiments on synthetic datasets with non-convex geometries and on random subsets of Fashion-MNIST show that T-ARC recovers latent topological structures where K-means fails, achieving the highest accuracy on curved and interleaved clusters while remaining competitive, and markedly more stable than K-means, on real data.

cs.LG↗

Optimization on the Oblique Manifold for Sparse Simplex Constraints via Multiplicative Updates

Low-rank optimization problems with sparse simplex constraints involve variables that must satisfy nonnegativity, sparsity, and sum-to-1 conditions, making their optimization particularly challenging due to the interplay between low-rank structures and constraints. These problems arise in various applications, including machine learning, signal processing, environmental fields, and computational biology. In this work, we propose a novel manifold optimization approach to efficiently tackle these problems. Our method leverages the geometry of oblique manifolds to reformulate the problem and introduces a new Riemannian optimization method based on Riemannian gradient descent that strictly maintains the simplex constraints. By exploiting the underlying manifold structure, our approach improves optimization efficiency. Experiments on synthetic and real datasets demonstrate the effectiveness of the proposed method compared to standard Euclidean and Riemannian methods, paving the way for broader applications.

math.OC↗

Optimal Network Pricing for Oblivious Users under Projected Decision-Dependent Distributions

Efficient large-scale network allocation requires data-driven pricing mechanisms that internalize stochastic, nonlinear user behavior. We move beyond the classic fully strategic agents to study oblivious users (agents with bounded rationality and imperfect information). Rather than assuming an infinite horizon, our regime acknowledges that real-world flows are too transient to equilibrate among users. We introduce a novel Optimal Network Pricing (ONP) problem for such users, which induces Performativity: a Decision-Dependent environment where pricing decisions endogenously shift the flow distribution. Without a closed-form distribution, the platform must learn optimal prices from sampled responses. This setting introduces a new challenge: capacity boundaries and projection operators make the optimization landscape nonsmooth, invalidating gradient-based methods. We show that a widely adopted optimality concept Performative Stability (PS) fails in ONP, collapsing to a trivial solution. We then define a new optimality concept, the Projected Performative Optimum (ΠPO) for the unique global optimum. Targeting ΠPO is algorithmically hard given the performative nonsmooth Jacobian, so we propose a novel framework combining Sample Average Approximation with Trust-Region Sequential Quadratic Programming, explicitly handling the capacity boundaries, with theoretical guarantees on probabilistic convexity, sample complexity, and computational complexity. Experiments show that our ΠPO solver significantly outperforms PS-seeking heuristics and a proposed baseline (improving social welfare by 81\% on GEANT), highlighting that properly handling capacity boundaries unlocks substantial gains in social welfare. More broadly, this work advances intelligent systems that learn under performative, capacity-constrained feedback, a core challenge in real-world AI applications.

math.NA↗

Binno: A 1st-order method for Bi-level Nonconvex Nonsmooth Optimization for Matrix Factorizations

Nonconvex and nonsmooth bi-level optimization poses critical theoretical challenges, while arising in several applications. In this work, we develop a method for nonconvex, nonsmooth bi-level optimization and introduce Binno, a first-order method that builds on proximal-gradient updates within the the proximal alternate minimization framework with descent conditions from variational analysis. Binno couples the two levels via a descent-driven averaging mechanism, extending single-level proximal schemes to the nonconvex nonsmooth bi-level setting. We show thatBinno induces a descent property for a suitable surrogate of the bi-level objective. Each iteration performs blockwise proximal-gradient updates for the upper and lower problems separately, then forms a calibrated, block-diagonal convex combination of the two iterates. A linesearch selects combination weights enforcing simultaneous descent of both objectives. We give conditions ensuring that both these weights and the descent directions induced by the associated proximal-gradient maps exist. We apply Binno to sparse low-rank factorization, where the upper level uses elementwise l1 penalties and the lower level uses nuclear norms, coupled via a Frobenius data term. We test Binno on synthetic matrices and a real traffic-video dataset, achieving lower reconstruction error and higher peak signal-to-noise ratio than standard methods. We also validate it on a regularized market-clearing problem, where it selects policy-preferred equilibria, and compare it with bi-level baseline, showing consistent improvements

math.OC↗

Sparse Hyperparametric Itakura-Saito Nonnegative Matrix Factorization via Bi-Level Optimization

The selection of penalty hyperparameters is a critical aspect in Nonnegative Matrix Factorization (NMF), since these values control the trade-off between reconstruction accuracy and adherence to desired constraints. In this work, we focus on an NMF problem involving the Itakura-Saito (IS) divergence, which is particularly effective for extracting low spectral density components from spectrograms of mixed signals, and benefits from the introduction of sparsity constraints. We propose a new algorithm called SHINBO, which introduces a bi-level optimization framework to automatically and adaptively tune the row-dependent penalty hyperparameters, enhancing the ability of IS-NMF to isolate sparse, periodic signals in noisy environments. Experimental results demonstrate that SHINBO achieves accurate spectral decompositions and demonstrates superior performance in both synthetic and real-world applications. In the latter case, SHINBO is particularly useful for noninvasive vibration-based fault detection in rolling bearings, where the desired signal components often reside in high-frequency subbands but are obscured by stronger, spectrally broader noise. By addressing the critical issue of hyperparameter selection, SHINBO improves the state-of-the-art in signal recovery for complex, noise-dominated environments.

cs.LG↗

Sum-of-norms regularized Nonnegative Matrix Factorization

When applying nonnegative matrix factorization (NMF), the rank parameter is generally unknown. This rank, called the nonnegative rank, is usually estimated heuristically since computing its exact value is NP-hard. In this work, we propose an approximation method to estimate the rank on-the-fly while solving NMF. We use the sum-of-norm (SON), a group-lasso structure that encourages pairwise similarity, to reduce the rank of a factor matrix when the initial rank is overestimated. On various datasets, SON-NMF can reveal the correct nonnegative rank of the data without prior knowledge or parameter tuning. SON-NMF is a nonconvex, nonsmooth, non-separable, and non-proximable problem, making it nontrivial to solve. First, since rank estimation in NMF is NP-hard, the proposed approach does not benefit from lower computational complexity. Using a graph-theoretic argument, we prove that the complexity of SON-NMF is essentially irreducible. Second, the per-iteration cost of algorithms for SON-NMF can be high. This motivates us to propose a first-order BCD algorithm that approximately solves SON-NMF with low per-iteration cost via the proximal average operator. SON-NMF exhibits favorable features for applications. Besides the ability to automatically estimate the rank from data, SON-NMF can handle rank-deficient data matrices and detect weak components with small energy. Furthermore, in hyperspectral imaging, SON-NMF naturally addresses the issue of spectral variability.

cs.LG↗

Chordal-NMF with Riemannian Multiplicative Update

Nonnegative Matrix Factorization (NMF) is the problem of approximating a given nonnegative matrix M through the product of two nonnegative low-rank matrices W and H. Traditionally NMF is tackled by optimizing a specific objective function evaluating the quality of the approximation. This assessment is often done based on the Frobenius norm (F-norm). In this work, we argue that the F-norm, as the ``point-to-point'' distance, may not always be appropriate. Viewing from the perspective of cone, NMF may not naturally align with F-norm. So, a ray-to-ray chordal distance is proposed as an alternative way of measuring the quality of the approximation. As this measure corresponds to the Euclidean distance on the sphere, it motivates the use of manifold optimization techniques. We apply Riemannian optimization technique to solve chordal-NMF by casting it on a manifold. Unlike works on Riemannian optimization that require the manifold to be smooth, the nonnegativity in chordal-NMF defines a non-differentiable manifold. We propose a Riemannian Multiplicative Update (RMU), and showcase the effectiveness of the chordal-NMF on synthetic and real-world datasets.

math.OC↗

Inhomogeneous graph trend filtering via a l2,0 cardinality penalty

We study estimation of piecewise smooth signals over a graph. We propose a $\ell_{2,0}$-norm penalized Graph Trend Filtering (GTF) model to estimate piecewise smooth graph signals that exhibit inhomogeneous levels of smoothness across the nodes. We prove that the proposed GTF model is simultaneously a k-means clustering on the signal over the nodes and a minimum graph cut on the edges of the graph, where the clustering and the cut share the same assignment matrix. We propose two methods to solve the proposed GTF model: a spectral decomposition method and a method based on simulated annealing. In the experiment on synthetic and real-world datasets, we show that the proposed GTF model has a better performances compared with existing approaches on the tasks of denoising, support recovery and semi-supervised classification. We also show that the proposed GTF model can be solved more efficiently than existing models for the dataset with a large edge set.

cs.LG↗

MGProx: A nonsmooth multigrid proximal gradient method with adaptive restriction for strongly convex optimization

We study the combination of proximal gradient descent with multigrid for solving a class of possibly nonsmooth strongly convex optimization problems. We propose a multigrid proximal gradient method called MGProx, which accelerates the proximal gradient method by multigrid, based on using hierarchical information of the optimization problem. MGProx applies a newly introduced adaptive restriction operator to simplify the Minkowski sum of subdifferentials of the nondifferentiable objective function across different levels. We provide a theoretical characterization of MGProx. First we show that the MGProx update operator exhibits a fixed-point property. Next, we show that the coarse correction is a descent direction for the fine variable of the original fine level problem in the general nonsmooth case. Lastly, under some assumptions we provide the convergence rate for the algorithm. In the numerical tests on the Elastic Obstacle Problem, which is an example of nonsmooth convex optimization problem where multigrid method can be applied, we show that MGProx has a faster convergence speed than competing methods.

math.OC↗

Fast Projection onto the Capped Simplex with Applications to Sparse Regression in Bioinformatics

We consider the problem of projecting a vector onto the so-called k-capped simplex, which is a hyper-cube cut by a hyperplane. For an n-dimensional input vector with bounded elements, we found that a simple algorithm based on Newton's method is able to solve the projection problem to high precision with a complexity roughly about O(n), which has a much lower computational cost compared with the existing sorting-based methods proposed in the literature. We provide a theory for partial explanation and justification of the method. We demonstrate that the proposed algorithm can produce a solution of the projection problem with high precision on large scale datasets, and the algorithm is able to significantly outperform the state-of-the-art methods in terms of runtime (about 6-8 times faster than a commercial software with respect to CPU time for input vector with 1 million variables or more). We further illustrate the effectiveness of the proposed algorithm on solving sparse regression in a bioinformatics problem. Empirical results on the GWAS dataset (with 1,500,000 single-nucleotide polymorphisms) show that, when using the proposed method to accelerate the Projected Quasi-Newton (PQN) method, the accelerated PQN algorithm is able to handle huge-scale regression problem and it is more efficient (about 3-6 times faster) than the current state-of-the-art methods.

math.OC↗