SVP-T: A Shape-level Variable-Position Transformer for Multivariate Time Series Classification

Rundong Zuo*, Guozhong Li, Byron Choi, Sourav S. Bhowmick, Daphne Ngar Yin Mah, Grace Lai Hung Wong

*Corresponding author for this work

Research output: Chapter in book/report/conference proceedingConference proceedingpeer-review

3 Citations (Scopus)

Abstract

Multivariate time series classification (MTSC), one of the most fundamental time series applications, has not only gained substantial research attentions but has also emerged in many real-life applications. Recently, using transformers to solve MTSC has been reported. However, current transformer-based methods take data points of individual timestamps as inputs (timestamp-level), which only capture the temporal dependencies, not the dependencies among variables. In this paper, we propose a novel method, called SVP-T. Specifically, we first propose to take time series subsequences, which can be from different variables and positions (time interval), as the inputs (shape-level). The temporal and variable dependencies are both handled by capturing the long- and short-term dependencies among shapes. Second, we propose a variable-position encoding layer (VP-layer) to utilize both the variable and position information of each shape. Third, we introduce a novel VP-based (Variable-Position) self-attention mechanism to allow the enhancing of the attention weights of overlapping shapes. We evaluate our method on all UEA MTS datasets. SVP-T achieves the best accuracy rank compared with several competitive state-of-the-art methods. Furthermore, we demonstrate the effectiveness of the VP-layer and the VP-based self-attention mechanism. Finally, we present one case study to interpret the result of SVP-T.
Original languageEnglish
Title of host publicationProceedings of the 37th AAAI Conference on Artificial Intelligence, AAAI 2023
EditorsBrian Williams, Yiling Chen, Jennifer Neville
Place of PublicationWashington, DC
PublisherAAAI press
Pages11497-11505
Number of pages9
Edition1st
ISBN (Electronic)9781577358800
DOIs
Publication statusPublished - 27 Jun 2023
Event37th AAAI Conference on Artificial Intelligence, AAAI 2023 and the 35th Conference on Innovative Applications of Artificial Intelligence, IAAI 2023 - Washington, United States
Duration: 7 Feb 202314 Feb 2023
https://ojs.aaai.org/index.php/AAAI/issue/view/553
https://aaai-23.aaai.org/

Publication series

NameProceedings of the AAAI Conference on Artificial Intelligence
Number9
Volume37
ISSN (Print)2159-5399
ISSN (Electronic)2374-3468

Conference

Conference37th AAAI Conference on Artificial Intelligence, AAAI 2023 and the 35th Conference on Innovative Applications of Artificial Intelligence, IAAI 2023
Country/TerritoryUnited States
CityWashington
Period7/02/2314/02/23
Internet address

User-Defined Keywords

  • ML: Time-Series/Data Streams
  • DMKM: Mining of Spatial
  • Temporal or Spatio-Temporal Data

Fingerprint

Dive into the research topics of 'SVP-T: A Shape-level Variable-Position Transformer for Multivariate Time Series Classification'. Together they form a unique fingerprint.

Cite this