Sparse Orthogonal Linear Discriminant Analysis

In this paper, sparse orthogonal linear discriminant analysis (OLDA) is studied. The main contributions of the present work include the following: (i) all minimum Frobenius-norm/dimension solutions of the optimization problem used for establishing OLDA are characterized explicitly; and (ii) this explicit characterization leads to two numerical algorithms for computing a sparse linear transformation for OLDA. The first is based on the gradient flow approach while the second is a sequential linear Bregman method. We experiment with real world datasets to illustrate that the sequential linear Bregman method is much better than the gradient flow approach. The sequential linear Bregman method always achieves comparable classification accuracy with the normal OLDA, satisfactory sparsity and orthogonality, and acceptable CPU times.