For face recognition task the PCA plus LDA technique is a famous two-phrase framework to deal with high dimensional space and singular cases. In this paper, we examine the theory of this framework: (1) LDA can still fail even after PCA procedure. (2) Some small principal components that might be essential for classification are thrown away after PCA step. (3) The null space of the within-class scatter matrix Sw contains discriminative information for classification. To eliminate these deficiencies of the PCA plus LDA method we thus develop a new framework by introducing an inverse Fisher criterion and adding a constrain in PCA procedure so that the singularity phenomenon will not occur. Experiment results suggest that this new approach works well.