Decomposing large-scale POMDP via belief state analysis

Xin Li*, Kwok Wai CHEUNG, Jiming LIU

*Corresponding author for this work

Research output: Chapter in book/report/conference proceedingConference contributionpeer-review

3 Citations (Scopus)

Abstract

Partially observable Markov decision process (POMDP) is commonly used to model a stochastic environment with unobservable states for supporting optimal decision making. Computing the optimal policy for a large-scale POMDP is known to be intractable. Belief compression, being an approximate solution, has recently been proposed to reduce the dimension of POMDP's belief state space and shown to be effective in improving the problem tractability. In this paper, with the conjecture that temporally close belief states could be characterized by a lower intrinsic dimension, we propose a spatio-temporal brief clustering that considers both the belief states ' spatial (in the belief space) and temporal similarities, as well as incorporate it into the belief compression algorithm. The proposed clustering results in belief state clusters as sub-POMDPs of much lower dimension so as to be distributed to a set of distributed agents for collaborative problem solving. The proposed method has been tested using a synthesized navigation problem (Hallway2) and empirically shown to be able to result in policies of superior long-term rewards when compared with those based on solely belief compression. Some future research directions for extending this belief state analysis approach are also included.

Original languageEnglish
Title of host publicationProceedings - 2005 IEEE/WIC/ACM International Conference on Intelligent Agent Technology, IAT'05
Pages428-434
Number of pages7
DOIs
Publication statusPublished - 2005
Event2005 IEEE/WIC/ACM International Conference on Intelligent Agent Technology - France, France
Duration: 19 Sep 200522 Sep 2005

Publication series

NameProceedings - 2005 IEEE/WIC/ACM International Conference on Intelligent Agent Technology, IAT'05
Volume2005

Conference

Conference2005 IEEE/WIC/ACM International Conference on Intelligent Agent Technology
Country/TerritoryFrance
CityFrance
Period19/09/0522/09/05

Scopus Subject Areas

  • Engineering(all)

Fingerprint

Dive into the research topics of 'Decomposing large-scale POMDP via belief state analysis'. Together they form a unique fingerprint.

Cite this