A study of local linear ridge regression estimators

In the case of the random design nonparametric regression, to correct for the unbounded finite-sample variance of the local linear estimator (LLE), Seifert and Gasser (J. Amer. Statist. Assoc. 91 (1996) 267–275) apply the idea of ridge regression to the LLE, and propose the local linear ridge regression estimator (LLRRE). However, the finite sample and the asymptotic properties of the LLRRE are not discussed there. In this paper, upper bounds of the finite-sample variance and bias of the LLRRE are obtained. It is shown that if the ridge regression parameters are not properly selected, then the resulting LLRRE has some drawbacks. For example, it may have a nonzero constant asymptotic bias, may suffer from boundary effects, or may be unable to share the nice asymptotic bias quality of the LLE. On the other hand, if the ridge regression parameters are properly selected, then the resulting LLRRE does not suffer from the above problems, and has the same asymptotic mean-square error as the LLE. For this purpose, the ridge regression parameters are allowed to depend on the sample size, and converge to 0 as the sample size increases. In practice, to select both the bandwidth and the ridge regression parameters, the idea of cross-validation is applied. Simulation studies demonstrate that the LLRRE using the cross-validated bandwidth and ridge regression parameters could have smaller sample mean integrated square error than the LLE using the cross-validated bandwidth, in reasonable sample sizes.