Robust Performance Evaluation of POMDP-Based Dialogue Systems

Cited 8 time in webofscience Cited 0 time in scopus
  • Hit : 1181
  • Download : 0
Partially observable Markov decision processes (POMDPs) have received significant interest in research on spoken dialogue systems, due to among many benefits its ability to naturally model the dialogue strategy selection problem under unreliable automated speech recognition. However, the POMDP approaches are essentially model-based, and as a result, the dialogue strategy computed from POMDP is still subject to the correctness of the model. In this paper, we extend some of the previous MDP user models to POMDPs, and evaluate the effects of user models on the dialogue strategy computed from POMDPs. We experimentally show that the strategies computed from POMDPs perform better than those from MDPs, and the strategies computed from poor user models fail severely when tested on different user models. This paper further investigates the evaluation methods for dialogue strategies, and proposes a method based on the bias-variance analysis for reliably estimating the dialogue performance.
Publisher
IEEE-INST ELECTRICAL ELECTRONICS ENGINEERS INC
Issue Date
2011-05
Language
English
Article Type
Article
Citation

IEEE TRANSACTIONS ON AUDIO SPEECH AND LANGUAGE PROCESSING, v.19, no.4, pp.1029 - 1040

ISSN
1558-7916
URI
http://hdl.handle.net/10203/97740
Appears in Collection
CS-Journal Papers(저널논문)AI-Journal Papers(저널논문)
Files in This Item
There are no files associated with this item.
This item is cited by other documents in WoS
⊙ Detail Information in WoSⓡ Click to see webofscience_button
⊙ Cited 8 items in WoS Click to see citing articles in records_button

qr_code

  • mendeley

    citeulike


rss_1.0 rss_2.0 atom_1.0