ODE-RSSM: Learning Stochastic Recurrent State Space Model from Irregularly Sampled Data

Zhaolin Yuan, Xiaojuan Ban*, Zixuan Zhang, Xiaorui Li, Hong-Ning Dai

*Corresponding author for this work

Research output: Chapter in book/report/conference proceedingConference proceedingpeer-review

Abstract

For the complicated input-output systems with nonlinearity and stochasticity, Deep State Space Models (SSMs) are effective for identifying systems in the latent state space, which are of great significance for representation, forecasting, and planning in online scenarios. However, most SSMs are designed for discrete-time sequences and inapplicable when the observations are irregular in time. To solve the problem, we propose a novel continuous-time SSM named Ordinary Differential Equation Recurrent State Space Model (ODE-RSSM). ODE-RSSM incorporates an ordinary differential equation (ODE) network (ODE-Net) to model the continuous-time evolution of latent states between adjacent time points. Inspired from the equivalent linear transformation on integration limits, we propose an efficient reparameterization method for solving batched ODEs with non-uniform time spans in parallel for efficiently training the ODE-RSSM with irregularly sampled sequences. We also conduct extensive experiments to evaluate the proposed ODE-RSSM and the baselines on three input-output datasets, one of which is a rollout of a private industrial dataset with strong long-term delay and stochasticity. The results demonstrate that the ODE-RSSM achieves better performance than other baselines in open loop prediction even if the time spans of predicted points are uneven and the distribution of length is changeable. Code is availiable at https://github.com/yuanzhaolin/ODE-RSSM.

Original languageEnglish
Title of host publicationProceedings of the 37th AAAI Conference on Artificial Intelligence
EditorsBrian Williams, Yiling Chen, Jennifer Neville
Place of PublicationWashington, DC
PublisherAAAI press
Pages11060-11068
Number of pages9
Edition1st
ISBN (Electronic)9781577358800
DOIs
Publication statusPublished - 27 Jun 2023
Event37th AAAI Conference on Artificial Intelligence, AAAI 2023 and the 35th Conference on Innovative Applications of Artificial Intelligence, IAAI 2023 - Washington, United States
Duration: 7 Feb 202314 Feb 2023
https://ojs.aaai.org/index.php/AAAI/issue/view/553
https://aaai-23.aaai.org/

Publication series

NameProceedings of the AAAI Conference on Artificial Intelligence
PublisherAAAI Press
Number9
Volume37
ISSN (Print)2159-5399
ISSN (Electronic)2374-3468

Conference

Conference37th AAAI Conference on Artificial Intelligence, AAAI 2023 and the 35th Conference on Innovative Applications of Artificial Intelligence, IAAI 2023
Country/TerritoryUnited States
CityWashington
Period7/02/2314/02/23
Internet address

Scopus Subject Areas

  • Artificial Intelligence

User-Defined Keywords

  • ML: Deep Generative Models & Autoencoders
  • ROB: Behavior Learning & Control
  • ML: Bayesian Learning
  • ML: Probabilistic Methods
  • ML: Representation Learning
  • PRS: Planning With Markov Models (MDPs, POMDPs)
  • RU: Stochastic Models & Probabilistic Inference
  • RU: Uncertainty Representations

Fingerprint

Dive into the research topics of 'ODE-RSSM: Learning Stochastic Recurrent State Space Model from Irregularly Sampled Data'. Together they form a unique fingerprint.

Cite this