EKDF: An Ensemble Knowledge Distillation Framework for Robust Collaborative Inference on Heterogeneous Edge Devices

The integration of edge computing and deep neural networks (DNNs) holds great promise for enhancing application intelligence. Edge devices generate or collect vast amounts of data, which DNNs can leverage to make informed decisions. Nevertheless, the limited resources of edge devices pose a significant challenge for deploying DNNs. To accommodate some edge devices (e.g. smart watches), lightweight models are often required. However, the accuracy of these models may not meet user expectations. In this paper, we present EKDF, an ensemble knowledge distillation framework that crafts lightweight models for collaborative DNN inferences. More specifically, we utilize knowledge distillation to compress DNN models. On this basis, we introduce multi-teacher joint supervision and dropout in knowledge distillation to improve model performance and preserve the diversity between the generated DNN models. This process produces a range of compact models of varying computational complexity for different edge devices. The experimental results demonstrate that our proposed EKDF can greatly improve the overall predictive ability.