An adaptive-to-model distributed hybrid test for conditional independence

In this paper, we propose a distributed test for conditional independence. To this end, we develop a distributed groupwise least squares estimation for the groupwise central dimension reduction subspace to identify the dimension of the underlying model structure. The test is an adaptive-to-model hybrid of two simple distributed tests. The dimension identification automatically adapts the test to the underlying model so that it is an omnibus test with a tractable limiting null distribution. It can detect the local alternatives distinct from the null hypothesis at a rate as close to 1/N1/\sqrt{N} as possible, where N is the total sample size. This is the fastest possible rate in hypothesis testing. When the total sample size is fixed, the sensitivity of the test to the alternative hypothesis does not decrease as the number of machines increases. Numerical studies suggest that the power of the hybrid remains high when the number of machines increases, while fixing the total sample size and the computational time decreases rapidly. The test is conducted to examine the suitability of proxies for unobserved factors in the study of returns to schooling, as well as to analyze the conditional independence between PM_2.5 and other air pollution and meteorological variables for illustration.