Learning perceptual embeddings with two related tasks for joint predictions of media interestingness and emotions

Yang LIU, Zhonglei Gu, Tobey H. Ko, Kien A. Hua

Research output: Chapter in book/report/conference proceedingConference contributionpeer-review

4 Citations (Scopus)

Abstract

Integrating media elements of various medium, multimedia is capable of expressing complex information in a neat and compact way. Early studies have linked different sensory presentation in multimedia with the perception of human-like concepts. Yet, the richness of information in multimedia makes understanding and predicting user perceptions in multimedia content a challenging task both to the machine and the human mind. This paper presents a novel multi-task feature extraction method for accurate prediction of user perceptions in multimedia content. Differentiating from the conventional feature extraction algorithms which focus on perfecting a single task, the proposed model recognizes the commonality between different perceptions (e.g., interestingness and emotional impact), and attempts to jointly optimize the performance of all the tasks through uncovered commonality features. Using both a media interestingness dataset and a media emotion dataset for user perception prediction tasks, the proposed model attempts to simultaneously characterize the individualities of each task and capture the commonalities shared by both tasks, and achieves better accuracy in predictions than other competing algorithms on real-world datasets of two related tasks: MediaEval 2017 Predicting Media Interestingness Task and MediaEval 2017 Emotional Impact of Movies Task.

Original languageEnglish
Title of host publicationICMR 2018 - Proceedings of the 2018 ACM International Conference on Multimedia Retrieval
PublisherAssociation for Computing Machinery, Inc
Pages420-427
Number of pages8
ISBN (Print)9781450350464
DOIs
Publication statusPublished - 5 Jun 2018
Event8th ACM International Conference on Multimedia Retrieval, ICMR 2018 - Yokohama, Japan
Duration: 11 Jun 201814 Jun 2018

Publication series

NameICMR 2018 - Proceedings of the 2018 ACM International Conference on Multimedia Retrieval

Conference

Conference8th ACM International Conference on Multimedia Retrieval, ICMR 2018
Country/TerritoryJapan
CityYokohama
Period11/06/1814/06/18

Scopus Subject Areas

  • Computer Graphics and Computer-Aided Design
  • Computer Networks and Communications
  • Computer Science Applications

User-Defined Keywords

  • Multi-task feature extraction
  • Multimedia emotional impact prediction
  • Multimedia interestingness prediction
  • Perceptual embedding

Fingerprint

Dive into the research topics of 'Learning perceptual embeddings with two related tasks for joint predictions of media interestingness and emotions'. Together they form a unique fingerprint.

Cite this