Dialogue Systems Go Multimodal: The SmartKom Experience -- Facts and Figures About the SmartKom Project -- An Exemplary Interaction with SmartKom -- Multimodal Input Analysis -- The SmartKom Architecture: A Framework for Multimodal Dialogue Systems -- Modeling Domain Knowledge: Know-How and Know-What -- Speech Recognition -- Class-Based Language Model Adaptation -- The Dynamic Lexicon -- The Prosody Module -- The Sense of Vision: Gestures and Real Objects -- The Facial Expression Module -- Multiple Biometrics -- Natural Language Understanding -- The Gesture Interpretation Module -- Multimodal Dialogue Processing -- Modality Fusion -- Discourse Modeling -- Overlay: The Basic Operation for Discourse Processing -- In Context: Integrating Domain- and Situation-Specific Knowledge -- Intention Recognition -- Plan-Based Dialogue Management for Multiple Cooperating Applications -- Emotion Analysis and Emotion-Handling Subdialogues -- Problematic, Indirect, Affective, and Other Nonstandard Input Processing -- Multimodal Output Generation -- Realizing Complex User Wishes with a Function Planning Module -- Intelligent Integration of External Data and Services into SmartKom -- Multimodal Fission and Media Design -- Natural Language Generation with Fully Specified Templates -- Multimodal Speech Synthesis -- Scenarios and Applications -- Building Multimodal Dialogue Applications: System Integration in SmartKom -- SmartKom-English: From Robust Recognition to Felicitous Interaction -- SmartKom-Public -- SmartKom-Home: The Interface to Home Entertainment -- SmartKom-Mobile: Intelligent Interaction with a Mobile System -- SmartKom-Mobile Car: User Interaction with Mobile Services in a Car Environment -- Data Collection and Evaluation -- Wizard-of-Oz Recordings -- Annotation of Multimodal Data -- Multimodal Emogram, Data Collection and Presentation -- Empirical Studies for Intuitive Interaction -- Evaluation of Multimodal Dialogue Systems.