arXiv subjects

Hengchao Chen

Publications and source records attributed to Hengchao Chen.

12 recordsLinked to original sources

arXiv2026

Mixture of Geodesic Factor Analyzers on Riemannian Homogeneous Spaces

This paper introduces Mixtures of Geodesic Factor Analyzers (MGFA) on Riemannian homogeneous spaces. MGFA uses a geodesic factor model within each mixture component, providing greater expressiveness than mixtures of Riemannian radial distributions and enabling clustering of manifold-valued data with anisotropic subpopulations. We establish root-$n$ consistency for the MGFA maximum likelihood estimator (MLE), thereby filling a theoretical gap for mixtures of Riemannian radial distributions as a special case. We also propose an iterative estimation algorithm and implement it on spheres, shape spaces, and hyperbolic spaces. Numerical experiments show that MGFA substantially outperforms competing methods in well-specified regimes while remaining robust under model misspecification. Finally, case studies on corpus callosum and left hippocampus shape datasets demonstrate MGFA's effectiveness for both 2D contour and 3D shape analysis.

stat.ML↗

arXiv2026

Distributional Matching for Vector Quantization: A Unified Theoretical and Empirical Framework

The effectiveness of modern visual representation learning and autoregressive models critically depends on vector quantization (VQ), which discretizes continuous feature representations using a learnable codebook. Despite its widespread use, existing VQ methods often suffer from training instability and codebook collapse, arising from gradient mismatch induced by the straight-through estimator and the under-utilization of code vectors. In this work, we show that both issues can be traced to a fundamental mismatch between the distributions of feature vectors and code vectors, leading to inefficient representation and information loss. Building on this observation, we propose a distributional matching framework for vector quantization. We introduce principled criteria for desirable VQ behavior and demonstrate through theoretical analysis and empirical evaluation that aligning feature and code vector distributions provides a unifying mechanism for mitigating training instability and codebook collapse. We instantiate this framework using a Wasserstein-based objective with an efficient closed-form under a mild Gaussian approximation, and further show that a nonparametric alternative based on maximum mean discrepancy yields comparable performance. Extensive experiments on visual tokenization benchmarks support the effectiveness and robustness of the proposed approach.

cs.CV↗

arXiv2025

Enhancing Vector Quantization with Distributional Matching: A Theoretical and Empirical Study

The success of autoregressive models largely depends on the effectiveness of vector quantization, a technique that discretizes continuous features by mapping them to the nearest code vectors within a learnable codebook. Two critical issues in existing vector quantization methods are training instability and codebook collapse. Training instability arises from the gradient discrepancy introduced by the straight-through estimator, especially in the presence of significant quantization errors, while codebook collapse occurs when only a small subset of code vectors are utilized during training. A closer examination of these issues reveals that they are primarily driven by a mismatch between the distributions of the features and code vectors, leading to unrepresentative code vectors and significant data information loss during compression. To address this, we employ the Wasserstein distance to align these two distributions, achieving near 100\% codebook utilization and significantly reducing the quantization error. Both empirical and theoretical analyses validate the effectiveness of the proposed approach.

cs.CV↗

arXiv2024

Decentralized Online Riemannian Optimization with Dynamic Environments

This paper develops the first decentralized online Riemannian optimization algorithm on Hadamard manifolds. Our algorithm, the decentralized projected Riemannian gradient descent, iteratively performs local updates using projected Riemannian gradient descent and a consensus step via weighted Frechet mean. Theoretically, we establish linear variance reduction for the consensus step. Building on this, we prove a dynamic regret bound of order ${\cal O}(\sqrt{T(1+P_T)}/\sqrt{(1-\sigma_2(W))})$, where $T$ is the time horizon, $P_T$ represents the path variation measuring nonstationarity, and $\sigma_2(W)$ measures the network connectivity. The weighted Frechet mean in our algorithm incurs a minimization problem, which can be computationally expensive. To further alleviate this cost, we propose a simplified consensus step with a closed-form, replacing the weighted Frechet mean. We then establish linear variance reduction for this alternative and prove that the decentralized algorithm, even with this simple consensus step, achieves the same dynamic regret bound. Finally, we validate our approach with experiments on nonstationary decentralized Frechet mean computation over hyperbolic spaces and the space of symmetric positive definite matrices, demonstrating the effectiveness of our methods.

math.OC↗

arXiv2024

Riemannian radial distributions on Riemannian symmetric spaces: Optimal rates of convergence for parameter estimation

Manifold data analysis is challenging due to the lack of parametric distributions on manifolds. To address this, we introduce a series of Riemannian radial distributions on Riemannian symmetric spaces. By utilizing the symmetry, we show that for many Riemannian radial distributions, the Riemannian $L^p$ center of mass is uniquely given by the location parameter, and the maximum likelihood estimator (MLE) of this parameter is given by an M-estimator. Therefore, these parametric distributions provide a promising tool for statistical modeling and algorithmic design. In addition, our paper develops a novel theory for parameter estimation and minimax optimality by integrating statistics, Riemannian geometry, and Lie theory. We demonstrate that the MLE achieves a convergence rate of root-$n$ up to logarithmic terms, where the rate is quantified by both the hellinger distance between distributions and geodesic distance between parameters. Then we derive a root-$n$ minimax lower bound for the parameter estimation rate, demonstrating the optimality of the MLE. Our minimax analysis is limited to the case of simply connected Riemannian symmetric spaces for technical reasons, but is still applicable to numerous applications. Finally, we extend our studies to Riemannian radial distributions with an unknown temperature parameter, and establish the convergence rate of the MLE. We also derive the model complexity of von Mises-Fisher distributions on spheres and discuss the effects of geometry in statistical estimation.

math.ST↗

arXiv2024

Quotient geometry of bounded or fixed rank correlation matrices

This paper studies the quotient geometry of bounded or fixed-rank correlation matrices. We establish a bijection between the set of bounded-rank correlation matrices and a quotient set of a spherical product manifold by an orthogonal group. We show that it forms an orbit space, whose stratification is determined by the rank of the matrices, and the principal stratum has a compatible Riemannian quotient manifold structure. We show that any minimizing geodesic in the orbit space has constant rank on the interior of the segment. We also develop efficient Riemannian optimization algorithms for computing the distance and weighted the Frechet mean in the orbit space. Moreover, we examine geometric properties of the quotient manifold, including horizontal and vertical spaces, Riemannian metric, injectivity radius, exponential and logarithmic map, curvature, gradient and Hessian. Finally, we apply our approach to a functional connectivity study using the Autism Brain Imaging Data Exchange.

math.MG↗

arXiv2023

Ridge Estimation with Nonlinear Transformations

Ridge estimation is an important manifold learning technique. The goal of this paper is to examine the effects of nonlinear transformations on the ridge sets. The main result proves the inclusion relationship between ridges: $\cR(f\circ p)\subseteq \cR(p)$, provided that the transformation $f$ is strictly increasing and concave on the range of the function $p$. Additionally, given an underlying true manifold $\cM$, we show that the Hausdorff distance between $\cR(f\circ p)$ and its projection onto $\cM$ is smaller than the Hausdorff distance between $\cR(p)$ and the corresponding projection. This motivates us to apply an increasing and concave transformation before the ridge estimation. In specific, we show that the power transformations $f^{q}(y)=y^q/q,-\infty<q\leq 1$ are increasing and concave on $\RR_+$, and thus we can use such power transformations when $p$ is strictly positive. Numerical experiments demonstrate the advantages of the proposed methods.

cs.LG↗

arXiv2023

Gradient descent in matrix factorization: Understanding large initialization

Gradient Descent (GD) has been proven effective in solving various matrix factorization problems. However, its optimization behavior with large initial values remains less understood. To address this gap, this paper presents a novel theoretical framework for examining the convergence trajectory of GD with a large initialization. The framework is grounded in signal-to-noise ratio concepts and inductive arguments. The results uncover an implicit incremental learning phenomenon in GD and offer a deeper understanding of its performance in large initialization scenarios.

math.OC↗

arXiv2023

Bounded Projection Matrix Approximation with Applications to Community Detection

Community detection is an important problem in unsupervised learning. This paper proposes to solve a projection matrix approximation problem with an additional entrywise bounded constraint. Algorithmically, we introduce a new differentiable convex penalty and derive an alternating direction method of multipliers (ADMM) algorithm. Theoretically, we establish the convergence properties of the proposed algorithm. Numerical experiments demonstrate the superiority of our algorithm over its competitors, such as the semi-definite relaxation method and spectral clustering.

cs.SI↗

arXiv2023

Statistical Analysis of Karcher Means for Random Restricted PSD Matrices

Non-asymptotic statistical analysis is often missing for modern geometry-aware machine learning algorithms due to the possibly intricate non-linear manifold structure. This paper studies an intrinsic mean model on the manifold of restricted positive semi-definite matrices and provides a non-asymptotic statistical analysis of the Karcher mean. We also consider a general extrinsic signal-plus-noise model, under which a deterministic error bound of the Karcher mean is provided. As an application, we show that the distributed principal component analysis algorithm, LRC-dPCA, achieves the same performance as the full sample PCA algorithm. Numerical experiments lend strong support to our theories.

stat.ML↗

arXiv2023

Quadratic Matrix Factorization with Applications to Manifold Learning

Matrix factorization is a popular framework for modeling low-rank data matrices. Motivated by manifold learning problems, this paper proposes a quadratic matrix factorization (QMF) framework to learn the curved manifold on which the dataset lies. Unlike local linear methods such as the local principal component analysis, QMF can better exploit the curved structure of the underlying manifold. Algorithmically, we propose an alternating minimization algorithm to optimize QMF and establish its theoretical convergence properties. Moreover, to avoid possible over-fitting, we then propose a regularized QMF algorithm and discuss how to tune its regularization parameter. Finally, we elaborate how to apply the regularized QMF to manifold learning problems. Experiments on a synthetic manifold learning dataset and two real datasets, including the MNIST handwritten dataset and a cryogenic electron microscopy dataset, demonstrate the superiority of the proposed method over its competitors.

cs.LG↗

arXiv2022

Distributed Sparse Multicategory Discriminant Analysis

This paper proposes a convex formulation for sparse multicategory linear discriminant analysis and then extend it to the distributed setting when data are stored across multiple sites. The key observation is that for the purpose of classification it suffices to recover the discriminant subspace which is invariant to orthogonal transformations. Theoretically, we establish statistical properties ensuring that the distributed sparse multicategory linear discriminant analysis performs as good as the centralized version after {a few rounds} of communications. Numerical studies lend strong support to our methodology and theory.

math.ST↗