Tucker network: Expressive power and comparison

Ye Liu, Junjun Pan*, Michael K. Ng

*Corresponding author for this work

Research output: Contribution to journalJournal articlepeer-review

1 Citation (Scopus)

Abstract

Deep neural networks have achieved great success in solving many machine learning and computer vision problems. In this paper, we propose a deep neural network called the Tucker network derived from the Tucker format and analyze its expressive power. The results demonstrate that the Tucker network has exponentially higher expressive power than the shallow network. In other words, a shallow network with an exponential width is required to realize the same score function as that computed by the Tucker network. Moreover, we discuss the expressive power between the hierarchical Tucker tensor network (HT network) and the proposed Tucker network. To generalize the Tucker network into a deep version, we combine the hierarchical Tucker format and Tucker format to propose a deep Tucker tensor decomposition. Its corresponding deep Tucker network is presented. Experiments are conducted on three datasets: MNIST, CIFAR-10 and CIFAR-100. The results experimentally validate the theoretical results and show that the Tucker network and deep Tucker network have better performance than the shallow network and HT network.

Original languageEnglish
Pages (from-to)63-83
Number of pages21
JournalNeural Networks
Volume160
Early online date24 Dec 2022
DOIs
Publication statusPublished - Mar 2023

Scopus Subject Areas

  • Cognitive Neuroscience
  • Artificial Intelligence

User-Defined Keywords

  • Deep neural network
  • Expressive power
  • Tensor decomposition

Fingerprint

Dive into the research topics of 'Tucker network: Expressive power and comparison'. Together they form a unique fingerprint.

Cite this