arXiv subjects

Xinsheng Zhang

Publications and source records attributed to Xinsheng Zhang.

27 records · Page 2Linked to original sources

arXiv2020

Large-dimensional Factor Analysis without Moment Constraints

Large-dimensional factor model has drawn much attention in the big-data era, in order to reduce the dimensionality and extract underlying features using a few latent common factors. Conventional methods for estimating the factor model typically requires finite fourth moment of the data, which ignores the effect of heavy-tailedness and thus may result in unrobust or even inconsistent estimation of the factor space and common components. In this paper, we propose to recover the factor space by performing principal component analysis to the spatial Kendall's tau matrix instead of the sample covariance matrix. In a second step, we estimate the factor scores by the ordinary least square (OLS) regression. Theoretically, we show that under the elliptical distribution framework the factor loadings and scores as well as the common components can be estimated consistently without any moment constraint. The convergence rates of the estimated factor loadings, scores and common components are provided. The finite sample performance of the proposed procedure is assessed through thorough simulations. An analysis of a financial data set of asset returns shows the superiority of the proposed method over the classical PCA method.

stat.ME↗

arXiv2020

Robust Covariance Estimation for High-dimensional Compositional Data with Application to Microbial Communities Analysis

Microbial communities analysis is drawing growing attention due to the rapid development of high-throughput sequencing techniques nowadays. The observed data has the following typical characteristics: it is high-dimensional, compositional (lying in a simplex) and even would be leptokurtic and highly skewed due to the existence of overly abundant taxa, which makes the conventional correlation analysis infeasible to study the co-occurrence and co-exclusion relationship between microbial taxa. In this article, we address the challenges of covariance estimation for this kind of data. Assuming the basis covariance matrix lying in a well-recognized class of sparse covariance matrices, we adopt a proxy matrix known as centered log-ratio covariance matrix in the literature, which is approximately indistinguishable from the real basis covariance matrix as the dimensionality tends to infinity. We construct a Median-of-Means (MOM) estimator for the centered log-ratio covariance matrix and propose a thresholding procedure that is adaptive to the variability of individual entries. By imposing a much weaker finite fourth moment condition compared with the sub-Gaussianity condition in the literature, we derive the optimal rate of convergence under the spectral norm. In addition, we also provide theoretical guarantee on support recovery. The adaptive thresholding procedure of the MOM estimator is easy to implement and gains robustness when outliers or heavy-tailedness exist. Thorough simulation studies are conducted to show the advantages of the proposed procedure over some state-of-the-arts methods. At last, we apply the proposed method to analyze a microbiome dataset in human gut. The R script for implementing the method is available at https://github.com/heyongstat/RCEC.

stat.ME↗

arXiv2020

Network-Assisted Estimation for Large-dimensional Factor Model with Guaranteed Convergence Rate Improvement

Network structure is growing popular for capturing the intrinsic relationship between large-scale variables. In the paper we propose to improve the estimation accuracy for large-dimensional factor model when a network structure between individuals is observed. To fully excavate the prior network information, we construct two different penalties to regularize the factor loadings and shrink the idiosyncratic errors. Closed-form solutions are provided for the penalized optimization problems. Theoretical results demonstrate that the modified estimators achieve faster convergence rates and lower asymptotic mean squared errors when the underlying network structure among individuals is correct. An interesting finding is that even if the priori network is totally misleading, the proposed estimators perform no worse than conventional state-of-art methods. Furthermore, to facilitate the practical application, we propose a data-driven approach to select the tuning parameters, which is computationally efficient. We also provide an empirical criterion to determine the number of common factors. Simulation studies and application to the S&P100 weekly return dataset convincingly illustrate the superiority and adaptivity of the new approach.

stat.ME↗

arXiv2019

Robust Factor Number Specification for Large-dimensional Elliptical Factor Model

The accurate specification of the number of factors is critical to the validity of factor models and the topic almost occupies the central position in factor analysis. Plenty of estimators are available under the restrictive condition that the fourth moments of the factors and idiosyncratic errors are bounded. In this paper we propose efficient and robust estimators for the factor number via considering a more general static Elliptical Factor Model (EFM) framework. We innovatively propose to exploit the multivariate Kendall's tau matrix, which captures the correlation structure of elliptical random vectors. Theoretically we show that the proposed estimators are consistent without exerting any moment condition when both cross-sections N and time dimensions T go to infinity. Simulation study shows that the new estimators perform much better in heavy-tailed data setting while performing comparably with the state-of-the-art methods in the light-tailed Gaussian setting. At last, a real macroeconomic data example is given to illustrate its empirical advantages and usefulness.

stat.ME↗

arXiv2019

D$^2$-City: A Large-Scale Dashcam Video Dataset of Diverse Traffic Scenarios

Driving datasets accelerate the development of intelligent driving and related computer vision technologies, while substantial and detailed annotations serve as fuels and powers to boost the efficacy of such datasets to improve learning-based models. We propose D$^2$-City, a large-scale comprehensive collection of dashcam videos collected by vehicles on DiDi's platform. D$^2$-City contains more than 10000 video clips which deeply reflect the diversity and complexity of real-world traffic scenarios in China. We also provide bounding boxes and tracking annotations of 12 classes of objects in all frames of 1000 videos and detection annotations on keyframes for the remainder of the videos. Compared with existing datasets, D$^2$-City features data in varying weather, road, and traffic conditions and a huge amount of elaborate detection and tracking annotations. By bringing a diverse set of challenging cases to the community, we expect the D$^2$-City dataset will advance the perception and related areas of intelligent driving.

cs.LG↗

arXiv2018

High-dimensional Two-sample Precision Matrices Test: An Adaptive Approach through Multiplier Bootstrap

Precision matrix, which is the inverse of covariance matrix, plays an important role in statistics, as it captures the partial correlation between variables. Testing the equality of two precision matrices in high dimensional setting is a very challenging but meaningful problem, especially in the differential network modelling. To our best knowledge, existing test is only powerful for sparse alternative patterns where two precision matrices differ in a small number of elements. In this paper we propose a data-adaptive test which is powerful against either dense or sparse alternatives. Multiplier bootstrap approach is utilized to approximate the limiting distribution of the test statistic. Theoretical properties including asymptotic size and power of the test are investigated. Simulation study verifies that the data-adaptive test performs well under various alternative scenarios. The practical usefulness of the test is illustrated by applying it to a gene expression data set associated with lung cancer.

stat.ME↗

arXiv2018

Doubly Robust Sure Screening for Elliptical Copula Regression Model

Regression analysis has always been a hot research topic in statistics. We propose a very flexible semi-parametric regression model called Elliptical Copula Regression (ECR) model, which covers a large class of linear and nonlinear regression models such as additive regression model,single index model. Besides, ECR model can capture the heavy-tail characteristic and tail dependence between variables, thus it could be widely applied in many areas such as econometrics and finance. In this paper we mainly focus on the feature screening problem for ECR model in ultra-high dimensional setting. We propose a doubly robust sure screening procedure for ECR model, in which two types of correlation coefficient are involved: Kendall tau correlation and Canonical correlation. Theoretical analysis shows that the procedure enjoys sure screening property, i.e., with probability tending to 1, the screening procedure selects out all important variables and substantially reduces the dimensionality to a moderate size against the sample size. Thorough numerical studies are conducted to illustrate its advantage over existing sure independence screening methods and thus it can be used as a safe replacement of the existing procedures in practice. At last, the proposed procedure is applied on a gene-expression real data set to show its empirical usefulness.

stat.ME↗

arXiv2018

A Unified Framework for Testing High Dimensional Parameters: A Data-Adaptive Approach

High dimensional hypothesis test deals with models in which the number of parameters is significantly larger than the sample size. Existing literature develops a variety of individual tests. Some of them are sensitive to the dense and small disturbance, and others are sensitive to the sparse and large disturbance. Hence, the powers of these tests depend on the assumption of the alternative scenario. This paper provides a unified framework for developing new tests which are adaptive to a large variety of alternative scenarios in high dimensions. In particular, our framework includes arbitrary hypotheses which can be tested using high dimensional $U$-statistic based vectors. Under this framework, we first develop a broad family of tests based on a novel variant of the $L_p$-norm with $p\in \{1,\dots,\infty\}$. We then combine these tests to construct a data-adaptive test that is simultaneously powerful under various alternative scenarios. To obtain the asymptotic distributions of these tests, we utilize the multiplier bootstrap for $U$-statistics. In addition, we consider the computational aspect of the bootstrap method and propose a novel low-cost scheme. We prove the optimality of the proposed tests. Thorough numerical results on simulated and real datasets are provided to support our theory.

math.ST↗

arXiv2018

An Extreme-Value Approach for Testing the Equality of Large U-Statistic Based Correlation Matrices

There has been an increasing interest in testing the equality of large Pearson's correlation matrices. However, in many applications it is more important to test the equality of large rank-based correlation matrices since they are more robust to outliers and nonlinearity. Unlike the Pearson's case, testing the equality of large rank-based statistics has not been well explored and requires us to develop new methods and theory. In this paper, we provide a framework for testing the equality of two large U-statistic based correlation matrices, which include the rank-based correlation matrices as special cases. Our approach exploits extreme value statistics and the Jackknife estimator for uncertainty assessment and is valid under a fully nonparametric model. Theoretically, we develop a theory for testing the equality of U-statistic based correlation matrices. We then apply this theory to study the problem of testing large Kendall's tau correlation matrices and demonstrate its optimality. For proving this optimality, a novel construction of least favourable distributions is developed for the correlation matrix comparison.

math.ST↗