Compressing Deep Convolutional Neural Networks by Stacking Low-dimensional Binary Convolution Filters

Weichao Lan, Liang Lan

Research output: Chapter in book/report/conference proceedingConference contributionpeer-review

Abstract

Deep Convolutional Neural Networks (CNN) have been successfully applied to many real-life problems. However, the huge memory cost of deep CNN models poses a great challenge of deploying them on memory-constrained devices (e.g., mobile phones). One popular way to reduce the memory cost of deep CNN model is to train binary CNN where the weights in convolution filters are either 1 or −1 and therefore each weight can be efficiently stored using a single bit. However, the compression ratio of existing binary CNN models is upper bounded by around 32. To address this limitation, we propose a novel method to compress deep CNN model by stacking low-dimensional binary convolution filters. Our proposed method approximates a standard convolution filter by selecting and stacking filters from a set of low-dimensional binary convolution filters. This set of low-dimensional binary convolution filters is shared across all filters for a given convolution layer. Therefore, our method will achieve much larger compression ratio than binary CNN models. In order to train our proposed model, we have theoretically shown that our proposed model is equivalent to select and stack intermediate feature maps generated by low-dimensional binary filters. Therefore, our proposed model can be efficiently trained using the split-transform-merge strategy. We also provide detailed analysis of the memory and computation cost of our model in model inference. We compared the proposed method with other five popular model compression techniques on three benchmark datasets. Our experimental results have clearly demonstrated that our proposed method achieves much higher compression ratio of 103.2 times on VGG-16 network than existing methods while maintains comparable accuracy.
Original languageEnglish
Title of host publication35th AAAI Conference on Artificial Intelligence, AAAI 2021
PublisherAAAI press
Pages8235-8242
Number of pages8
ISBN (Print)9781577358664
Publication statusPublished - 18 May 2021

Publication series

NameProceedings of the AAAI Conference on Artificial Intelligence
Number9
Volume35
ISSN (Print)2159-5399
ISSN (Electronic)2374-3468
NameAAAI-21/ IAAI-21/ EAAI-21 Proceedings

Fingerprint

Dive into the research topics of 'Compressing Deep Convolutional Neural Networks by Stacking Low-dimensional Binary Convolution Filters'. Together they form a unique fingerprint.

Cite this