Learning with Complex Data: A Mutual-Information-Guided Deep Learning (MIGDL) Framework

The goal of this project is to design, analyze, and demonstrate a Mutual-Information-Guided Deep Learning (MIGDL) Framework. This novel framework aims to systematically address the fundamental challenges of modeling and prediction with complex data, i.e., learning capacity characterization, feature adaptation, and model adaptation.

The rapid development of deep learning has offered a scientifically grounded solution to various problems in data analytics. Nevertheless, the design of deep learning methods for real-world applications remains a fundamental challenge. This challenge is encapsulated in the questions of:
(i) How a deep model's learning capacity with respect to a specific learning task and dataset can be quantitatively characterized, such that the most appropriate model can be designed;
(ii) How a deep learning model's current structure and learning performance can be used to decide what must be further acquired from the data; and
(iii) How a deep learning model based on the existing structure can be adapted to enable the information in further acquired data to be most effectively extracted and utilized.

To answer these questions, we will develop a MIGDL framework to characterize the behaviors of deep learning models via mutual information. The MIGDL framework will comprise the following three important and interrelated components: (1) A Capacity characterization module (C-Module) to analytically characterize and quantitatively describe the capacity of a learning model to extract useful information from data, given a learning target; (2) A Feature adaptation module (F-Module) to determine what kind(s) of data must be further acquired to effectively complement the extracted/learned information, with respect to the learning target; and (3) A Model adaptation module (M-Module) to determine which part(s) of the existing model must be further adapted to make the optimal use of the data newly acquired by the F-Module, corresponding to the learning target.

The analytical validation of the properties and the empirical evaluation of the performance of our MIGDL framework will contribute to fundamental developments in artificial intelligence and machine learning. More importantly, this project could lead to answers to the following important open question, as stated above: How can a deep model's learning capacity with respect to a specific learning task and a dataset be quantitatively characterized, such that the most appropriate model can be designed? This project will also nurture talents in artificial intelligence, machine learning, and data analytics by creating research and practice opportunities for undergraduate and postgraduate students.