Volume Sampling and Spectral Equalization in Randomized Alternating Projections

Abstract

We study a family of randomized alternating projection methods that alternate among subspaces spanned by subsets of \(n\) vectors from a prescribed set. We derive an explicit characterization of performance bounds when these subspaces are sampled with probabilities proportional to the volumes subtended by the vectors in their corresponding subsets. For each \(n\), the bound is obtained through an explicit nonlinear transformation of the spectrum of an associated matrix. Our analysis reveals an explicit spectral equalization mechanism that drives the spectrum toward improved conditioning as $n$ increases, establishing an unexpected connection between volume sampling and the Cayley-Hamilton theorem. Furthermore, we introduce a computationally efficient uniform sampling scheme that achieves comparable theoretical guarantees through a specific relaxation strategy. Empirical results show that this relaxation method achieves convergence rates comparable to, and often better than, volume sampling, while avoiding the infeasibility of volume sampling at large scale. Besides the randomized Kaczmarz, these results also directly apply to randomized Gauss-Seidel and coordinate descent methods.