Neurodynamical Approach for Optimization Problems with Partial Orthogonality Constraints in Supervised Learning

According to the Wikipedia, “Supervised learning is the machine learning task of learning a function that maps an input to an output based on example input-output pairs.” With the emerging technologies in artificial intelligence and big data, supervised learning plays an essential role in artificial neural networks, data mining, machine learn- ing, and so on. The mathematical models behind these applications can be viewed as the optimization problems with partial orthogonality constraints. Hong Kong, as one of the world centers of finance and logistics and one of the key players in the great bay area, has a very strong need for the cutting edge technology in the artificial intelligence and big data. The neurodynamical optimization models and their corresponding solu- tion schemes studied in this proposal provide some novel and innovative techniques in supervised learning. With the demonstrated applications in graph clustering and super- vised locality preserving projection, we are fully believed that the results of this research proposal have great application potential in various areas of the artificial intelligence and big data.

As mentioned above, the optimization problems with partial orthogonality constraints are the key behind many applications in supervised learning. In the literature, only some special cases of such problems can be solved so far. Based on the success of the neu- rodynamical optimization in the neural network community and the mature techniques in optimization, in this research proposal, we plan to (a) establish some neurodynami- cal optimization models, which are in the form of some ordinary differential equations (ODE), for the optimization problems with partial orthogonality constraints; (b) develop some solution schemes for the ODE systems in (a), some theoretical issues including the convergence of these solution schemes will be also studied; (c) demonstrate that some existing solvers for some special cases of the optimization problems with partial orthogo- nality constraints can be regarded as some discrete approximate solution schemes of the neurodynamical optimization ODE systems developed in (a); and (d) conduct extensive numerical experiment on various applications in supervised learning.

In this research proposal, by completing the above tasks, we will be able to develop some new theoretically sound and numerical efficient algorithms for various problems in supervised learning. Our results will advance the state-of-the-art in both theoreti- cal and computational aspects for the optimization problems with partial orthogonality constraints in various areas of the artificial intelligence and big data.