Independence tests with random subspace of two random vectors in high dimension

Tao Qiu, Wangli Xu*, Lixing Zhu

*Corresponding author for this work

Research output: Contribution to journalJournal articlepeer-review

Abstract

Testing for independence between two random vectors is a fundamental problem in statistics. When the dimension of these two random vectors are fixed, the existing tests based on the distance covariance and Hilbert–Schmidt independence criterion with many desirable properties, including the capacity to capture linear and non-linear dependence. However, these tests may fail to capture the non-linear dependence due to the “curse of dimensionality” when the random vectors are high dimensional. To attack this problem, we propose a general framework for testing the dependence of two random vectors to randomly select two subspaces consisting of components of the vectors, respectively. To enhance the performance of this method, we repeatedly implement this procedure to construct the final test statistic. The new method can also work for non-linear dependence detection in a high-dimensional setup. Theoretically, if the replication time tends to infinity to get the final statistic, we can avoid potential power loss caused by lousy subspaces. Therefore, the two proposed tests are consistent with general alternatives. The weak limit under the null hypothesis is normal; thus, determining critical value need not resort to resampling approximation. We demonstrate the finite-sample performance of the proposed test by using Monte Carlo simulations and the analysis for a real-data example.

Original languageEnglish
Article number105160
Number of pages22
JournalJournal of Multivariate Analysis
Volume195
Early online date2 Feb 2023
DOIs
Publication statusPublished - May 2023

Scopus Subject Areas

  • Statistics and Probability
  • Numerical Analysis
  • Statistics, Probability and Uncertainty

User-Defined Keywords

  • Distance covariance
  • High dimension
  • Hilbert–Schmidt independence criterion
  • Independence test
  • Random subspace sampling
  • U-statistics

Fingerprint

Dive into the research topics of 'Independence tests with random subspace of two random vectors in high dimension'. Together they form a unique fingerprint.

Cite this