To effectively classify infrared spectrum (IRS) fingerprints of Chinese herbs, this paper presents a new radial basis function (RBF) network namely, Locally Gaussian Mixture Based RBF (LGM-RBF) Network. Unlike the traditional RBF network, the LGM-RBF has a mix layer between the hidden layer and the output layer. The hidden nodes with spherical Gaussian are initially grouped so that each group is corresponding to a class. The outputs of hidden nodes in a group are linearly weighted and mixed by a node in the mix layer. All outputs of the mix layer are nonlinearly weighted and then transferred to the output layer. In order to reduce the number of hidden nodes and further improve the system performance, a strategy is proposed to optimize the distribution of the training data in the feature space. The LGM-RBF features the fast learning speed and robust performance on high-dimensional data with a small sample size. Experimental results show the efficacy of the LGM-RBF to the IRS fingerprint classification of Chinese herbs.