Online Non-Negative Multi-Modality Feature Template Learning for RGB-Assisted Infrared Tracking

Xiangyuan LAN, Mang Ye*, Rui Shao, Bineng Zhong, Deepak Kumar Jain, Huiyu Zhou

*Corresponding author for this work

Research output: Contribution to journalJournal articlepeer-review

22 Citations (Scopus)


Infrared sensors have been deployed in many video surveillance systems because of the insensibility of their imaging procedure to some extreme conditions (e.g. low illumination condition, dim environment). To reduce human labor in video monitoring and perform intelligent infrared video understanding, an important issue we need to consider is how to locate the object of interest in consecutive video frames accurately. Therefore, developing a robust object tracking algorithm for infrared videos is necessary. However, the infrared information may not be reliable (e.g. thermal crossover), and appearance modeling with only the infrared modality may not be able to achieve good results. To address these issues, with the wide deployment of RGB-infrared camera systems, this paper proposes an infrared tracking framework in which information from RGB-modality will be exploited to assist the infrared object tracking. Specifically, within the tracking framework, in order to deal with the contaminated features caused by large appearance variations, an online non-negative feature template learning model is designed. The non-negative constraint enables the model to capture the local part-based characteristic of the target appearance. To ensure more important modality contribute more in appearance representation, an adaptive modality importance weight learning scheme is also incorporated in the proposed feature learning model. To guarantee the model optimality, an iterative optimization algorithm is derived. The experimental results on various RGB-infrared videos show the effectiveness of the proposed method.

Original languageEnglish
Article number8713854
Pages (from-to)67761-67771
Number of pages11
JournalIEEE Access
Publication statusPublished - 14 May 2019

Scopus Subject Areas

  • Computer Science(all)
  • Materials Science(all)
  • Engineering(all)

User-Defined Keywords

  • computer vision
  • Optical image processing
  • sensor fusion


Dive into the research topics of 'Online Non-Negative Multi-Modality Feature Template Learning for RGB-Assisted Infrared Tracking'. Together they form a unique fingerprint.

Cite this