arXiv subjects

Andrew D. McRae

Publications and source records attributed to Andrew D. McRae.

16 recordsLinked to original sources

arXiv2026

Low-rank matrix recovery landscapes beyond RIP with application to rank-one measurements

We study the problem of low-rank matrix recovery from linear measurements via the global nonconvex landscape of a low-rank factored formulation of the matrix LASSO (nuclear-norm--regularized least-squares). If the landscape is benign, that is, has no bad local optima, then practical and scalable algorithms can compute good statistical estimates. Previous state-of-the-art landscape guarantees have typically assumed that the linear measurement operator has the restricted isometry property, that is, the operator is approximately an isometry over all low-rank matrices. This is an unrealistic assumption for many applications; in particular, when the individual measurement matrices are themselves low-rank, we typically have poor upper isometry constants. To overcome this, we establish new guarantees of a benign landscape under a weaker isometry condition: rather than requiring upper isometry over all low-rank matrices, we only require it over the linear low-rank tangent space to the low-rank ground truth matrix. To illustrate the utility of this result, we apply it to the problem of matrix recovery from random rank-one linear measurements; via high-probability concentration bounds on the random measurement operator, we prove a novel landscape guarantee with statistically near-optimal sample complexity and recovery error.

math.OC↗

arXiv2026

Spectral Initialization and Certification for Power System Angle Estimation

Power System State Estimation (PSSE) is commonly formulated as a nonconvex weighted least-squares (WLS) problem, making global optimality difficult both to attain and to certify. Recent work has shown that, when voltage magnitudes are known to sufficient accuracy, the remaining angle estimation subproblem can be reduced to phase synchronization and solved effectively using spectral initialization and spectral certification. This paper explains why these spectral methods succeed. We prove that their behavior is governed by the measurement error, normalized against the usual observability margin from the classical literature. Below a fixed threshold, spectral initialization recovers the true voltage angles to first-order accuracy, and the WLS estimator is unique up to a global phase. Moreover, a zero-duality-gap spectral certificate verifies recovery of this unique WLS estimate. In the noiseless observable regime, spectral initialization exactly recovers the true angles and certification is exact without local refinement. Numerical experiments on standard benchmark systems support the theory and show that the spectral methods remain effective beyond the conservative regime covered by the guarantees.

math.OC↗

arXiv2026

Sharp recovery and landscape guarantees for the nonconvex matrix LASSO

Low-rank matrix recovery can be solved to statistical optimality by convex matrix optimization under the classical assumption of restricted isometry property (RIP). However, for large problems, the convex formulation is commonly replaced by a smooth rank-constrained factored nonconvex problem for which algorithmic theory typically only guarantees convergence to second-order critical points. In this paper, we develop a sharp and statistically optimal theory for second-order critical points of the factored nonconvex matrix LASSO (nuclear-norm--regularized least-squares estimator) under RIP with particular emphasis on the overparametrized regime where the search rank $r$ exceeds the ground-truth rank $r_*$. Our recovery error bounds reveal the precise role of nuclear norm regularization, interpolating between the classical convex rate and known rates for the unregularized nonconvex problem. Complementing this positive result, we give examples showing that, contrary to popular belief, rank overparametrization does not always improve the optimization landscape even under RIP. This negative result raises questions about the fundamental statistical recovery capability of rank-constrained nonconvex approaches in comparison to convex approaches which have worse computational scaling. All of our results generalize to arbitrary convex functions with nuclear-norm regularization under restricted strong convexity and smoothness. In particular, we give sharp conditions under which second-order critical points of the nonconvex problem either (1) approximately recover low-rank approximate minima of the convex problem or (2) exactly recover a low-rank global optimum if one exists.

math.OC↗

arXiv2025

Phase retrieval via overparametrized nonconvex optimization: nonsmooth amplitude loss landscapes

We study nonconvex optimization for phase retrieval and the more general problem of semidefinite low-rank matrix sensing; in particular, we focus on the global nonconvex landscape of overparametrized versions of the nonsmooth amplitude least-squares loss as well as a smooth reformulation of this loss based on the PhaseCut approach. We first give a general, deterministic result on properties of second-order critical points for a general class of loss functions; we then specialize this result to the nonsmooth amplitude loss and, additionally, prove nearly identical results for a smooth reformulation (similar to PhaseCut) as a synchronization problem over spheres. Finally, we show the usefulness of these tools by proving high-probability landscape guarantees in two settings: (1) phase retrieval with isotropic sub-Gaussian measurements, and (2) phase retrieval in a general (possibly infinite-dimensional) Hilbert space with Gaussian measurements. In both cases, our results give state-of-the-art and statistically optimal guarantees with only a constant amount of overparametrization (in the well-studied case of isotropic sub-Gaussian measurements, such statistical guarantees had previously required greater degrees of overparametrization/relaxation); this demonstrates the potential of overparametrized nonconvex optimization as a principled and scalable algorithmic approach to phase retrieval.

math.OC↗

arXiv2025

Sensor network localization has a benign landscape after low-dimensional relaxation

We consider the sensor network localization problem, which is closely related to multidimensional scaling and Euclidean distance matrix completion. Given a ground truth configuration of $n$ points in $\mathbb{R}^\ell$, we observe a subset of the pairwise distances and aim to recover the underlying configuration (up to rigid transformations). We show with a simple counterexample that the associated optimization problem is nonconvex and may admit spurious local minimizers, even when all distances are known. Yet, inspired by numerical experiments, we argue that all second-order critical points become global minimizers when the problem is relaxed by optimizing over configurations in dimension $k > \ell$. Specifically, we show this for two settings, both when all pairwise distances are known: (1) for arbitrary ground truth points, and $k= O(\sqrt{\ell n})$, and: (2) for isotropic random ground truth points, and $k = O(\ell + \log n)$. To prove these results, we identify and exploit key properties of the linear map which sends inner products to squared distances.

math.OC↗

arXiv2025

Phase retrieval and matrix sensing via benign and overparametrized nonconvex optimization

We study a nonconvex optimization algorithmic approach to phase retrieval and the more general problem of semidefinite low-rank matrix sensing. Specifically, we analyze the nonconvex landscape of a quartic Burer-Monteiro factored least-squares optimization problem. We develop a new analysis framework, taking advantage of the semidefinite problem structure, to understand the properties of second-order critical points -- specifically, whether they (approximately) recover the ground truth matrix. We show that it can be helpful to (mildly) overparametrize the problem, that is, to optimize over matrices of higher rank than the ground truth. We then apply this framework to several well-studied problem instances: in addition to recovering existing state-of-the-art phase retrieval landscape guarantees (without overparametrization), we show that overparametrizing by a factor at most logarithmic in the dimension allows recovery with optimal statistical sample complexity and error for the problems of (1) phase retrieval with sub-Gaussian measurements and (2) more general semidefinite matrix sensing with rank-1 Gaussian measurements. Previously, such statistical results had been shown only for estimators based on semidefinite programming. More generally, our analysis is partially based on the powerful method of convex dual certificates, suggesting that it could be applied to a much wider class of problems.

math.OC↗

arXiv2025

Benign landscapes for synchronization on spheres via normalized Laplacian matrices

We study the nonconvex optimization landscapes of synchronization problems on spheres. First, we present new results for the statistical problem of synchronization over the two-element group $\mathbf{Z}_2$. We consider the nonconvex least-squares problem with $\mathbf{Z}_2 = \{\pm 1\}$ relaxed to the unit sphere in $\mathbf{R}^r$ for $r \geq 2$; for several popular models, including graph clustering under the binary stochastic block model, we show that, for any $r \geq 2$, every second-order critical point recovers the ground truth in the asymptotic regimes where exact recovery is information-theoretically possible. Such statistical optimality via spherical relaxations had previously only been shown for (potentially arbitrarily) larger relaxation dimension $r$. Second, we consider the global synchronization of networks of coupled oscillators under the (homogeneous) Kuramoto model. We prove new and optimal asymptotic results for random signed networks on an Erd\H{o}s--R\'enyi graph, and we give new and simple proofs for several existing state-of-the-art results. Our key tool is a deterministic landscape condition that extends a recent result of Rakoto Endor and Waldspurger. This result says that, if a certain problem-dependent Laplacian matrix has small enough condition number, the nonconvex landscape is benign. Our extension allows the condition number to include an arbitrary diagonal preconditioner, which gives tighter results for many problems. We show that, for the synchronization of Kuramoto oscillator networks on nearest-neighbor circulant graphs as studied by Wiley, Strogatz, and Girvan, this condition is optimal. We also prove a natural complex extension that may be of interest for synchronization on the special orthogonal group $\operatorname{SO}(2)$.

math.OC↗

arXiv2024

Nonconvex landscapes for $\mathbf{Z}_2$ synchronization and graph clustering are benign near exact recovery thresholds

We study the optimization landscape of a smooth nonconvex program arising from synchronization over the two-element group $\mathbf{Z}_2$, that is, recovering $z_1, \dots, z_n \in \{\pm 1\}$ from (noisy) relative measurements $R_{ij} \approx z_i z_j$. Starting from a max-cut--like combinatorial problem, for integer parameter $r \geq 2$, the nonconvex problem we study can be viewed both as a rank-$r$ Burer--Monteiro factorization of the standard max-cut semidefinite relaxation and as a relaxation of $\{ \pm 1 \}$ to the unit sphere in $\mathbf{R}^r$. First, we present deterministic, non-asymptotic conditions on the measurement graph and noise under which every second-order critical point of the nonconvex problem yields exact recovery of the ground truth. Then, via probabilistic analysis, we obtain asymptotic guarantees for three benchmark problems: (1) synchronization with a complete graph and Gaussian noise, (2) synchronization with an Erd\H{o}s--R\'enyi random graph and Bernoulli noise, and (3) graph clustering under the binary symmetric stochastic block model. In each case, we have, asymptotically as the problem size goes to infinity, a benign nonconvex landscape near a previously-established optimal threshold for exact recovery; we can approach this threshold to arbitrary precision with large enough (but finite) rank parameter $r$. In addition, our results are robust to monotone adversaries.

math.OC↗

arXiv2024

Synchronization on circles and spheres with nonlinear interactions

We consider the dynamics of $n$ points on a sphere in $\mathbb{R}^d$ ($d \geq 2$) which attract each other according to a function $\varphi$ of their inner products. When $\varphi$ is linear ($\varphi(t) = t$), the points converge to a common value (i.e., synchronize) in various connectivity scenarios: this is part of classical work on Kuramoto oscillator networks. When $\varphi$ is exponential ($\varphi(t) = e^{\beta t}$), these dynamics correspond to a limit of how idealized transformers process data, as described by Geshkovski et al. (2025). Accordingly, they ask whether synchronization occurs for exponential $\varphi$. The answer depends on the dimension $d$. In the context of consensus for multi-agent control, Markdahl et al. (2018) show that for $d \geq 3$ (spheres), if the interaction graph is connected and $\varphi$ is increasing and convex, then the system synchronizes. We give a separate proof of this result. What is the situation on circles ($d=2$)? First, we show that $\varphi$ being increasing and convex is no longer sufficient (even for complete graphs). Then we identify a new condition under which we do have synchronization on the circle (namely, if the Taylor coefficients of $\varphi'$ are decreasing). As a corollary, this provide synchronization for exponential $\varphi$ with $\beta \in (0, 1]$. The proofs are based on nonconvex landscape analysis.

math.OC↗

arXiv2024

Low solution rank of the matrix LASSO under RIP with consequences for rank-constrained algorithms

We show that solutions to the popular convex matrix LASSO problem (nuclear-norm--penalized linear least-squares) have low rank under similar assumptions as required by classical low-rank matrix sensing error bounds. Although the purpose of the nuclear norm penalty is to promote low solution rank, a proof has not yet (to our knowledge) been provided outside very specific circumstances. Furthermore, we show that this result has significant theoretical consequences for nonconvex rank-constrained optimization approaches. Specifically, we show that if (a) the ground truth matrix has low rank, (b) the (linear) measurement operator has the matrix restricted isometry property (RIP), and (c) the measurement error is small enough relative to the nuclear norm penalty, then the (unique) LASSO solution has rank (approximately) bounded by that of the ground truth. From this, we show (a) that a low-rank--projected proximal gradient descent algorithm will converge linearly to the LASSO solution from any initialization, and (b) that the nonconvex landscape of the low-rank Burer-Monteiro--factored problem formulation is benign in the sense that all second-order critical points are globally optimal and yield the LASSO solution.

math.OC↗

arXiv2023

Perceptual adjustment queries and an inverted measurement paradigm for low-rank metric learning

We introduce a new type of query mechanism for collecting human feedback, called the perceptual adjustment query ( PAQ). Being both informative and cognitively lightweight, the PAQ adopts an inverted measurement scheme, and combines advantages from both cardinal and ordinal queries. We showcase the PAQ in the metric learning problem, where we collect PAQ measurements to learn an unknown Mahalanobis distance. This gives rise to a high-dimensional, low-rank matrix estimation problem to which standard matrix estimators cannot be applied. Consequently, we develop a two-stage estimator for metric learning from PAQs, and provide sample complexity guarantees for this estimator. We present numerical simulations demonstrating the performance of the estimator and its notable properties.

stat.ML↗

arXiv2023

Benign landscapes of low-dimensional relaxations for orthogonal synchronization on general graphs

Orthogonal group synchronization is the problem of estimating $n$ elements $Z_1, \ldots, Z_n$ from the $r \times r$ orthogonal group given some relative measurements $R_{ij} \approx Z_i^{}Z_j^{-1}$. The least-squares formulation is nonconvex. To avoid its local minima, a Shor-type convex relaxation squares the dimension of the optimization problem from $O(n)$ to $O(n^2)$. Alternatively, Burer--Monteiro-type nonconvex relaxations have generic landscape guarantees at dimension $O(n^{3/2})$. For smaller relaxations, the problem structure matters. It has been observed in the robotics literature that, for SLAM problems, it seems sufficient to increase the dimension by a small constant multiple over the original. We partially explain this. This also has implications for Kuramoto oscillators. Specifically, we minimize the least-squares cost function in terms of estimators $Y_1, \ldots, Y_n$. For $p \geq r$, each $Y_i$ is relaxed to the Stiefel manifold $\mathrm{St}(r, p)$ of $r \times p$ matrices with orthonormal rows. The available measurements implicitly define a (connected) graph $G$ on $n$ vertices. In the noiseless case, we show that, for all connected graphs $G$, second-order critical points are globally optimal as soon as $p \geq r+2$. (This implies that Kuramoto oscillators on $\mathrm{St}(r, p)$ synchronize for all $p \geq r + 2$.) This result is the best possible for general graphs; the previous best known result requires $2p \geq 3(r + 1)$. For $p > r + 2$, our result is robust to modest amounts of noise (depending on $p$ and $G$). Our proof uses a novel randomized choice of tangent direction to prove (near-)optimality of second-order critical points. Finally, we partially extend our noiseless landscape results to the complex case (unitary group); we show that there are no spurious local minima when $2p \geq 3r$.

math.OC↗

arXiv2023

New Equivalences Between Interpolation and SVMs: Kernels and Structured Features

The support vector machine (SVM) is a supervised learning algorithm that finds a maximum-margin linear classifier, often after mapping the data to a high-dimensional feature space via the kernel trick. Recent work has demonstrated that in certain sufficiently overparameterized settings, the SVM decision function coincides exactly with the minimum-norm label interpolant. This phenomenon of support vector proliferation (SVP) is especially interesting because it allows us to understand SVM performance by leveraging recent analyses of harmless interpolation in linear and kernel models. However, previous work on SVP has made restrictive assumptions on the data/feature distribution and spectrum. In this paper, we present a new and flexible analysis framework for proving SVP in an arbitrary reproducing kernel Hilbert space with a flexible class of generative models for the labels. We present conditions for SVP for features in the families of general bounded orthonormal systems (e.g. Fourier features) and independent sub-Gaussian features. In both cases, we show that SVP occurs in many interesting settings not covered by prior work, and we leverage these results to prove novel generalization results for kernel SVM classification.

stat.ML↗

arXiv2021

Harmless interpolation in regression and classification with structured features

Overparametrized neural networks tend to perfectly fit noisy training data yet generalize well on test data. Inspired by this empirical observation, recent work has sought to understand this phenomenon of benign overfitting or harmless interpolation in the much simpler linear model. Previous theoretical work critically assumes that either the data features are statistically independent or the input data is high-dimensional; this precludes general nonparametric settings with structured feature maps. In this paper, we present a general and flexible framework for upper bounding regression and classification risk in a reproducing kernel Hilbert space. A key contribution is that our framework describes precise sufficient conditions on the data Gram matrix under which harmless interpolation occurs. Our results recover prior independent-features results (with a much simpler analysis), but they furthermore show that harmless interpolation can occur in more general settings such as features that are a bounded orthonormal system. Furthermore, our results show an asymptotic separation between classification and regression performance in a manner that was previously only shown for Gaussian features.

stat.ML↗

arXiv2021

Optimal convex lifted sparse phase retrieval and PCA with an atomic matrix norm regularizer

We present novel analysis and algorithms for solving sparse phase retrieval and sparse principal component analysis (PCA) with convex lifted matrix formulations. The key innovation is a new mixed atomic matrix norm that, when used as regularization, promotes low-rank matrices with sparse factors. We show that convex programs with this atomic norm as a regularizer provide near-optimal sample complexity and error rate guarantees for sparse phase retrieval and sparse PCA. While we do not know how to solve the convex programs exactly with an efficient algorithm, for the phase retrieval case we carefully analyze the program and its dual and thereby derive a practical heuristic algorithm. We show empirically that this practical algorithm performs similarly to existing state-of-the-art algorithms.

math.ST↗

arXiv2019

Low-rank matrix completion and denoising under Poisson noise

This paper considers the problem of estimating a low-rank matrix from the observation of all or a subset of its entries in the presence of Poisson noise. When we observe all entries, this is a problem of matrix denoising; when we observe only a subset of the entries, this is a problem of matrix completion. In both cases, we exploit an assumption that the underlying matrix is low-rank. Specifically, we analyze several estimators, including a constrained nuclear-norm minimization program, nuclear-norm regularized least squares, and a nonconvex constrained low-rank optimization problem. We show that for all three estimators, with high probability, we have an upper error bound (in the Frobenius norm error metric) that depends on the matrix rank, the fraction of the elements observed, and maximal row and column sums of the true matrix. We furthermore show that the above results are minimax optimal (within a universal constant) in classes of matrices with low rank and bounded row and column sums. We also extend these results to handle the case of matrix multinomial denoising and completion.

stat.ML↗