Concentration Inequalities and Moment Bounds for Sample Covariance Operators

Abstract

Let $X,X_1,\dots, X_n,\dots$ be i.i.d. centered Gaussian random variables in a separable Banach space $E$ with covariance operator $Σ:$ $$ Σ:E^{\ast}\mapsto E,\ \ Σu = {\mathbb E}\langle X,u\rangle, u\in E^{\ast}. $$ The sample covariance operator $\hat Σ:E^{\ast}\mapsto E$ is defined as $$ \hat Σu := n^{-1}\sum_{j=1}^n \langle X_j,u\rangle X_j, u\in E^{\ast}. $$ The goal of the paper is to obtain concentration inequalities and expectation bounds for the operator norm $\|\hat Σ-Σ\|$ of the deviation of the sample covariance operator from the true covariance operator. In particular, it is shown that $$ {\mathbb E}\|\hat Σ-Σ\|\asymp \|Σ\|\biggl(\sqrt{\frac{{\bf r}(Σ)}{n}}\bigvee \frac{{\bf r}(Σ)}{n}\biggr), $$ where $$ {\bf r}(Σ):=\frac{\Bigl({\mathbb E}\|X\|\Bigr)^2}{\|Σ\|}. $$ Moreover, under the assumption that ${\bf r}(Σ)\lesssim n,$ it is proved that, for all $t\geq 1,$ with probability at least $1-e^{-t}$ \begin{align*} \Bigl|\|\hatΣ- Σ\|-{\mathbb E}\|\hatΣ- Σ\|\Bigr| \lesssim \|Σ\|\biggl(\sqrt{\frac{t}{n}}\bigvee \frac{t}{n}\biggr). \end{align*}