To address Small Sample Size (S3) problem and nonlinear problem of face recognition, this paper proposes a novel rank-lifting based kernel regularized discriminant analysis method (RL-KRDA). It first proves a rank-lifting theorem using algebraic theory. Combining a new ranklifting strategy with standby three-to-one regularization technique, the complete regularized technology is developed on the within-class scatter matrix Sw. Our regularized scheme not only adjusts the projection directions but tunes their corresponding weights as well. It is also shown that the final regularized within-class scatter matrix approaches to the original one as the regularized parameters tend to zeros. The public available database, i.e. CMU PIE face database, is selected for evaluation. Comparing with some existing kernel-based LDA methods for solving S3 problem, the proposed RL-KRDA approach gives the best performance.