Hostname: page-component-78c5997874-fbnjt Total loading time: 0 Render date: 2024-11-14T16:48:45.307Z Has data issue: false hasContentIssue false
  • English
  • Français

A tractable hybrid DDN–POMDP approach to affective dialogue modeling for probabilistic frame-based dialogue systems

Published online by Cambridge University Press:  01 April 2009

TRUNG H. BUI ,
MANNES POEL ,
ANTON NIJHOLT  and
JOB ZWIERS
TRUNG H. BUI
Affiliation:
Human Media Interaction Group, Department of Computer Science, University of Twente, Postbus 217, 7500 AE, Enschede, The Netherlands e-mail: thbui@stanford.edu, m.poel@ewi.utwente.nl, a.nijholt@ewi.utwente.nl, j.zwiers@ewi.utwente.nl
MANNES POEL
Affiliation:
Human Media Interaction Group, Department of Computer Science, University of Twente, Postbus 217, 7500 AE, Enschede, The Netherlands e-mail: thbui@stanford.edu, m.poel@ewi.utwente.nl, a.nijholt@ewi.utwente.nl, j.zwiers@ewi.utwente.nl
ANTON NIJHOLT
Affiliation:
Human Media Interaction Group, Department of Computer Science, University of Twente, Postbus 217, 7500 AE, Enschede, The Netherlands e-mail: thbui@stanford.edu, m.poel@ewi.utwente.nl, a.nijholt@ewi.utwente.nl, j.zwiers@ewi.utwente.nl
JOB ZWIERS
Affiliation:
Human Media Interaction Group, Department of Computer Science, University of Twente, Postbus 217, 7500 AE, Enschede, The Netherlands e-mail: thbui@stanford.edu, m.poel@ewi.utwente.nl, a.nijholt@ewi.utwente.nl, j.zwiers@ewi.utwente.nl
Get access Rights & Permissions [Opens in a new window]

Abstract

We propose a novel approach to developing a tractable affective dialogue model for probabilistic frame-based dialogue systems. The affective dialogue model, based on Partially Observable Markov Decision Process (POMDP) and Dynamic Decision Network (DDN) techniques, is composed of two main parts: the slot-level dialogue manager and the global dialogue manager. It has two new features: (1) being able to deal with a large number of slots and (2) being able to take into account some aspects of the user's affective state in deriving the adaptive dialogue strategies. Our implemented prototype dialogue manager can handle hundreds of slots, where each individual slot might have hundreds of values. Our approach is illustrated through a route navigation example in the crisis management domain. We conducted various experiments to evaluate our approach and to compare it with approximate POMDP techniques and handcrafted policies. The experimental results showed that the DDN–POMDP policy outperforms three handcrafted policies when the user's action error is induced by stress as well as when the observation error increases. Further, performance of the one-step look-ahead DDN–POMDP policy after optimizing its internal reward is close to state-of-the-art approximate POMDP counterparts.

Type
Papers
Information
Natural Language Engineering , Volume 15 , Issue 2 , April 2009 , pp. 273 - 307
Copyright
Copyright © Cambridge University Press 2009

Access options

Get access to the full version of this content by using one of the access options below. (Log in options will check for institutional or personal access. Content may require purchase if you do not have access.)

References

André, E., Dybkjær, L., Minker, W., and Heisterkamp, P. (eds.) 2004. Affective Dialogue Systems: Tutorial and Research Workshop. Lecture Notes in Artificial Intelligence. Berlin: Springer-Verlag.CrossRefGoogle Scholar
André, E., Rehm, M., Minker, W. and Bühler, D. 2004. Endowing spoken language dialogue systems with emotional intelligence. In Affective Dialogue Systems: Tutorial and Research Workshop, pp. 178–187. Lecture Notes in Artificial Intelligence. Berlin: Springer-Verlag.CrossRefGoogle Scholar
Ball, E. 2003. A Bayesian heart: Computer recognition and simulation of emotion. In Petta, P., Trappl, R. and Payr, S. (eds.), Emotions in Humans and Artifacts, chapter 11, pp. 303–332. Cambridge, MA: MIT Press.Google Scholar
Batliner, A., Fischer, K., Huber, R., Spilker, J. and Noth, E. 2003. How to find trouble in communication. Speech Communication 40: 117–143.Google Scholar
Boutilier, C. and Poole, D. 1996. Computing optimal policies for partially observable decision processes using compact representations. In Proceedings of the 13th National Conference on Artificial Intelligence, volume 2, pp. 1168–1175. Menlo Park, CA: AAAI Press.Google Scholar
Bui, T. H. 2006. Multimodal dialogue management—state of the art. Technical Report TR-CTIT-06-01, University of Twente.Google Scholar
Bui, T. H., Rajman, M. and Melichar, M. 2004. Rapid dialogue prototyping methodology. In Text, Speech & Dialogue: 7th International Conference, pp. 579–586. Lecture Notes in Artificial Intelligence. Berlin: Springer-Verlag.CrossRefGoogle Scholar
Bui, T. H., Zwiers, J., Poel, M. and Nijholt, A. 2006. Toward affective dialogue modeling using partially observable Markov decision processes. In Proceedings of the 1st workshop on Emotion and Computing—Current Research and Future Impact, pp. 47–50, University of Bremen.Google Scholar
Bui, T. H., van Schooten, B., and Hofs, D. 2007. Practical dialogue manager development using POMDPs. In Proceedings of 8th SIGdial Workshop on Discourse and Dialogue, pp. 215–218, Antwerp, Belgium.Google Scholar
Cassandra, A. R., Kaelbling, L. P. and Littman, M. L. 1994. Acting optimally in partially observable stochastic domains. In Proceedings of the 12th National Conference on Artificial Intelligence, volume 2, pp. 1023–1028. Menlo Park, CA: AAAI Press.Google Scholar
deRosis, F. Rosis, F., Novielli, N., Carofiglio, V., Cavalluzzi, A., and deCarolis, B. Carolis, B. 2006. User modeling and adaptation in health promotion dialogs with an animated character. Journal of Biomedical Informatics 39 (5): 514–531.Google Scholar
Eckert, W., Levin, E. and Pieraccini, R. 1997. User modelling for spoken dialogue system evaluation. In Proceedings of the IEEE Workshop on Automatic Speech Recognition and Understanding, pp. 80–87, Santa Barbara, CA.Google Scholar
Fernandez, R. and Picard, R. W. 2003. Modeling drivers' speech under stress. Speech Communication 40: 145–159.CrossRefGoogle Scholar
Fitrianie, S., Poppe, R. W., Bui, T. H., Chitu, A. G., Datcu, D., Dor, R., Hofs, D. H. W., Wiggers, P., Willems, D. J. M., Poel, M., Rothkrantz, L. J. M., Vuurpijl, L. G., and Zwiers, J. 2007. A multimodal human-computer interaction framework for research into crisis management. In Proceedings of the 4th International Conference on Information Systems for Crisis Management, pp. 149–158, Delft, The Netherlands.Google Scholar
Hauskrecht, M. 2000. Value-function approximations for partially observable Markov decision processes. Journal of Artificial Intelligence Research 13: 33–94.CrossRefGoogle Scholar
Heylen, D., Nijholt, A. and op den Akker, R. 2005. Affect in tutoring dialogues. Applied Artificial Intelligence 19 (3–4): 287–311.CrossRefGoogle Scholar
Holzapfel, H., Fuegen, C., Denecke, M. and Waibel, A. 2002. Integrating emotional cues into a framework for dialogue management. In Proceedings of the IEEE 4th International Conference on Multimodal Interfaces, pp. 141–146, Pittsburgh, PA.Google Scholar
Kaelbling, L. P., Littman, M. L. and Cassandra, A. R. 1998. Planning and acting in partially observable stochastic domains. Artificial Intelligence 101 (1–2): 99–134.CrossRefGoogle Scholar
Kanazawa, K. and Dean, T. 1989. A model for projection and action. In Proceedings of the 11th International Joint Conferences on Artificial Intelligence, pp. 985–990. San Francisco, CA: Morgan Kaufmann.Google Scholar
Levin, E., Pieraccini, R. and Eckert, W. 2000. A stochastic model of human-machine interaction for learning dialog strategies. IEEE Transactions on Speech and Audio Processing 8 (1): 11–23.Google Scholar
Li, X. and Ji, Q. 2005. Active affective state detection and user assistance with dynamic Bayesian networks. IEEE Transactions on Systems, Man, and Cybernetics, Part A 35 (1): 93–105.Google Scholar
Liao, W., Zhang, W., Zhu, Z., Ji, Q. and Wayne, D. G. 2006. Toward a decision-theoretic framework for affect recognition and user assistance. International Journal of Man-Machine Studies 64 (9): 847–873.Google Scholar
Littman, M. L., Cassandra, A. R., and Kaelbling, L. P. 1995. Learning policies for partially observable environments: Scaling up. In Proceedings of the 12th International Conference on Machine Learning, pp. 362–370, Tahoe, CA.CrossRefGoogle Scholar
McTear, M. 2002. Spoken dialogue technology: Enabling the conversational user interface. ACM Computing Survey 34 (1): 90–169.Google Scholar
Paek, T. and Horvitz, E. 2000. Conversation as action under uncertainty. In Proceedings of the 16th Conference on Uncertainty in Artificial Intelligence, pp. 455–464, San Francisco, CA.Google Scholar
Picard, R. W. 1997. Affective Computing, chapter 1, page 24. MA: MIT Press.Google Scholar
Pietquin, O. 2004. A Framework for Unsupervised Learning of Dialogue Strategies. PhD thesis, Universitaires de Louvain.Google Scholar
Pineau, J., Gordon, G. and Thrun, S. 2003. Point-based value iteration: An anytime algorithm for POMDPs. In Proceedings of the 18th Intenational Joint Conference on Artificial Intelligence (IJCAI), pp. 1025–1030, Acapulco, Mexico.Google Scholar
Pineau, J. and Thrun, S. 2001. Hierarchical POMDP decomposition for a conversational robot. In ICML Workshop on Hierarchy and Memory in Reinforcement Learning, Williams College, MA.Google Scholar
Pittermann, J., Pittermann, A, Meng, H. and Minker, W. 2007. Towards an emotion-sensitive spoken dialogue system—Classification and dialogue modeling. In Proceedings of the 3rd IET International Conference on Intelligent Environments, pp. 239–246, Ulm, Germany.CrossRefGoogle Scholar
Polzin, T. S. and Waibel, A. 2000. Emotion-sensitive human-computer interfaces. In SpeechEmotion-2000, pp. 201–206, Newcastle, UK.Google Scholar
Rajman, M., Bui, T. H., Rajman, A., Seydoux, F., Trutnev, A. and Quarteroni, S. 2004. Assessing the usability of a dialogue management system designed in the framework of a rapid dialogue prototyping methodology. ACTA ACUSTICA united with ACUSTICA, the Journal of the European Acoustics Association (EAA): International Journal on Acoustics. 90 (6): 1096–1111.Google Scholar
Roy, N., Pineau, J. and Thrun, S. 2000. Spoken dialogue management using probabilistic reasoning. In Proceedings of the 38th Annual Meeting of the Association for Computational Linguistics, pp. 93–100, Hong Kong.CrossRefGoogle Scholar
Roy, N. and Thrun, S. 1999. Coastal navigation with mobile robots. In Advances in Neural Information Processing Systems 12, pp. 1043–1049, Denver, CO.Google Scholar
Russell, S. and Norvig, P. 2003. Artificial Intelligence: A Modern Approach, 2nd ed., chapter 17, page 630. New Jersey: Prentice Hall.Google Scholar
Schatzmann, J., Weilhammer, K., Stuttle, M. and Young, S. 2006. A survey of statistical user simulation techniques for reinforcement-learning of dialogue management strategies. Knowledge Engineering Review 21 (2): 97–126.Google Scholar
Spaan, M. T. J. and Vlassis, N. 2005. Perseus: Randomized point-based value iteration for POMDPs. Journal of Artificial Intelligence Research, 24: 195–220.CrossRefGoogle Scholar
Sutton, R. S. and Barto, A. W. 1998. Reinforcement Learning: An Introduction, page 57. MA: MIT Press.Google Scholar
Traum, D. and Larsson, S. 2003. The information state approach to dialogue management. In Kuppevelt, J. van and Smith, R. W. (ed.), Current and New Directions in Discourse and Dialogue, chapter 15, pp. 325–353. Amsterdam: Kluwer Academic Publishers.Google Scholar
Williams, J. D., Poupart, P. and Young, S. 2005a. Factored partially observable Markov decision processes for dialogue management. In Proceedings of the 4th Workshop on Knowledge and Reasoning in Practical Dialogue Systems, pp. 76–82, Edinburgh, UK.Google Scholar
Williams, J. D., Poupart, P. and Young, S, 2005b. Partially observable Markov decision processes with continuous observations for dialogue management. In Proceedings of the 6th SIGdial Workshop on Discourse and Dialogue, pp. 25–34, Lisbon, Portugal.Google Scholar
Williams, J. D., Poupart, P. and Young, S. 2005c. Scaling up POMDPs for dialogue management: the summary POMDP method. In Proceedings of the IEEE workshop on Automatic Speech Recognition and Understanding, pp. 250–255, San Juan, Puerto Rico.CrossRefGoogle Scholar
Williams, J. D. and Young, S. 2006. Scaling POMDPs for dialog management with composite summary point-based value iteration (CSPBVI). In AAAI Workshop on Statistical and Empirical Approaches for Spoken Dialogue Systems, pp. 37–42, Boston, MA.Google Scholar
Williams, J. D. and Young, S. 2007. Partially observable Markov decision processes for spoken dialogue systems. Computer Speech and Language 21: 393–422.Google Scholar
Young, S., Schatzmann, J., Weilhammer, K. and Ye, H. 2007. The hidden information state approach to dialogue management. In Proceedings of International Conference on Acoustics, Speech, and Signal Processing, Honolulu, HI.Google Scholar
Young, S., Williams, J. D., Schatzmann, J., Stuttle, M. and Weilhammer, K. 2005. The hidden information state approach to dialogue management. Technical Report CUED/F-INFENG/TR.544, University of Cambridge.Google Scholar
Zhang, B., Cai, Q., Mao, J. and Guo, B. 2001. Spoken dialog management as planning and acting under uncertainty. In Proceedings of the 7th European Conference on Speech Communication and Technology, pp. 2169–2172, Aalborg, Denmark.Google Scholar
Zhou, X. and Conati, C. 2003. Inferring user goals from personality and behavior in a causal model of user affect. In Proceedings of the 8th international conference on Intelligent user interfaces, pp. 211–218, New York, USA.CrossRefGoogle Scholar