In this paper, we describe our model designed for automatic detection of diseases based on multimedia data collected in hospitals. Specifically, a two-stage learning strategy is designed to predict the diseases. In the first stage, a dimensionality reduction method called bidirectional marginal Fisher analysis (BMFA) is proposed to project the original data to the low-dimensional space, with the key discriminant information being well preserved. In the second stage, the multi-class support vector machine (SVM) is utilized on the low-dimensional space for detection. Experimental results demonstrate the efficiency of designed model.