Plant identification is an important problem for ecologists, amateur botanists, educators, and so on. Leaf, which can be easily obtained, is usually one of the important factors of plants. In this paper, we propose a growing convolution neural network (GCNN) for plant leaf identification and report the promising results on the ImageCLEF2012 Plant Identification database. TheGCNN owns a growing structure which starts training from a simple structure of a single convolution kernel and is gradually added new convolution neurons to. Simultaneously, the growing connection weights are modified until the squared-error achieves the desired result. Moreover, we propose a progressive learning method to determine the number of learning samples, which can further improve the recognition rate. Experiments and analyses show that our proposed GCNN outperforms other state-of-the-art algorithms such as the traditional CNN and the hand-crafted features with SVM classifiers.