Order determination for large matrices with spiked structure

Abstract

Motivated by dimension reduction in regression analysis and signal detection, we investigate order determination for large dimensional matrices with spiked structures in which the dimensions of the matrices are proportional to the sample sizes. Because the asymptotic behaviors of the estimated eigenvalues differ completely from those in fixed dimension scenarios, we then discuss the largest possible order, say q, we can identify and introduce criteria for different settings of q. When q is assumed to be fixed, we propose a "valley-cliff" criterion with two versions - one based on the original differences of eigenvalues and the other based on the transformed differences - to reduce the effect of ridge selection in the criterion. This generic method is very easy to implement and computationally inexpensive, and it can be applied to various matrices. As examples, we focus on spiked population models, spiked Fisher matrices and factor models with auto-covariance matrices. For the case of divergent q, we propose a scale-adjusted truncated double ridge ratio (STDRR) criterion, where a scale adjustment is implemented to deal with the bias in scale parameter for large q. Again, examples include spiked population models, spiked Fisher matrices. Numerical studies are conducted to examine the finite sample performances of the method and to compare it with existing methods. As for theoretical contributions, we investigate the limiting properties, including convergence in probability and central limit theorems, for spiked eigenvalues of spiked Fisher matrices with divergent q. Keywords: Auto-covariance matrix, factor model, finite-rank perturbation, Fisher matrix, principal component analysis (PCA), phase transition, random matrix theory (RMT), ridge ratio, spiked population model.