MILP based value backups in partially observed Markov decision processes (POMDPs) with very large or continuous action and observation spaces

Cited 2 time in webofscience Cited 0 time in scopus
  • Hit : 340
  • Download : 0
Partially observed Markov decision processes (POMDPs) serve as powerful tools to model stochastic systems with partial state information. Since the exact solution methods for POMDPs are limited to problems with very small sizes of state, action and observation spaces, approximate point-based solution methods like PERSEUS have gained popularity. In this work, a mixed integer linear program (MILP) is developed for calculation of exact value updates (in PERSEUS and similar algorithms), when the POMDP has very large or continuous action space. Since the solution time of the MILP is very sensitive to the size of the observation space, the concept of post-decision belief space is introduced to generate a more efficient and flexible model. An example involving a flow network is presented to illustrate the concepts and compare the results with those of the existing techniques. (C) 2013 Elsevier Ltd. All rights reserved.
Publisher
PERGAMON-ELSEVIER SCIENCE LTD
Issue Date
2013-09
Language
English
Article Type
Article
Keywords

INFINITE-HORIZON; SENSOR PLACEMENT; WATER NETWORKS

Citation

COMPUTERS & CHEMICAL ENGINEERING, v.56, pp.101 - 113

ISSN
0098-1354
DOI
10.1016/j.compchemeng.2013.04.020
URI
http://hdl.handle.net/10203/175493
Appears in Collection
CBE-Journal Papers(저널논문)
Files in This Item
There are no files associated with this item.
This item is cited by other documents in WoS
⊙ Detail Information in WoSⓡ Click to see webofscience_button
⊙ Cited 2 items in WoS Click to see citing articles in records_button

qr_code

  • mendeley

    citeulike


rss_1.0 rss_2.0 atom_1.0