Cover -- Title Page -- Copyright -- Contents -- About the Author -- List of Contributors -- Series Editor's Foreword -- Preface -- List of Acronyms -- Chapter 1 Senses, Perception, and Natural Human-Interfaces for Interactive Displays -- 1.1 Introduction -- 1.2 Human Senses and Perception -- 1.3 Human Interface Technologies -- 1.3.1 Legacy Input Devices -- 1.3.2 Touch-based Interactions -- 1.3.3 Voice-based Interactions -- 1.3.4 Vision-based Interactions -- 1.3.5 Multimodal Interactions -- 1.4 Towards "True'' 3D Interactive Displays -- 1.5 Summary -- References -- Chapter 2 Touch Sensing -- 2.1 Introduction -- 2.2 Introduction to Touch Technologies -- 2.2.1 Touchscreens -- 2.2.2 Classifying Touch Technologies by Size and Application -- 2.2.3 Classifying Touch Technologies by Materials and Structure -- 2.2.4 Classifying Touch Technologies by the Physical Quantity Being Measured -- 2.2.5 Classifying Touch Technologies by Their Sensing Capabilities -- 2.2.6 The Future of Touch Technologies -- 2.3 History of Touch Technologies -- 2.4 Capacitive Touch Technologies -- 2.4.1 Projected Capacitive (P-Cap) -- 2.4.2 Surface Capacitive -- 2.5 Resistive Touch Technologies -- 2.5.1 Analog Resistive -- 2.5.2 Digital Multi-touch Resistive (DMR) -- 2.5.3 Analog Multi-touch Resistive (AMR) -- 2.6 Acoustic Touch Technologies -- 2.6.1 Surface Acoustic Wave (SAW) -- 2.6.2 Acoustic Pulse Recognition (APR) -- 2.6.3 Dispersive Signal Technology (DST) -- 2.7 Optical Touch Technologies -- 2.7.1 Traditional Infrared -- 2.7.2 Multi-touch Infrared -- 2.7.3 Camera-based Optical -- 2.7.4 In-glass Optical (Planar Scatter Detection - PSD) -- 2.7.5 Vision-based Optical -- 2.8 Embedded Touch Technologies -- 2.8.1 On-cell Mutual-capacitive -- 2.8.2 Hybrid In-cell/On-cell Mutual-capacitive -- 2.8.3 In-cell Mutual-capacitive.

2.8.4 In-cell Light Sensing -- 2.9 Other Touch Technologies -- 2.9.1 Force-sensing -- 2.9.2 Combinations of Touch Technologies -- 2.10 Summary -- 2.11 Appendix -- References -- Chapter 3 Voice in the User Interface -- 3.1 Introduction -- 3.2 Voice Recognition -- 3.2.1 Nature of Speech -- 3.2.2 Acoustic Model and Front-end -- 3.2.3 Aligning Speech to HMMs -- 3.2.4 Language Model -- 3.2.5 Search: Solving Crosswords at 1000 Words a Second -- 3.2.6 Training Acoustic and Language Models -- 3.2.7 Adapting Acoustic and Language Models for Speaker Dependent Recognition -- 3.2.8 Alternatives to the "Canonical'' System -- 3.2.9 Performance -- 3.3 Deep Neural Networks for Voice Recognition -- 3.4 Hardware Optimization -- 3.4.1 Lower Power Wake-up Computation -- 3.4.2 Hardware Optimization for Specific Computations -- 3.5 Signal Enhancement Techniques for Robust Voice Recognition -- 3.5.1 Robust Voice Recognition -- 3.5.2 Single-channel Noise Suppression -- 3.5.3 Multi-channel Noise Suppression -- 3.5.4 Noise Cancellation -- 3.5.5 Acoustic Echo Cancellation -- 3.5.6 Beamforming -- 3.6 Voice Biometrics -- 3.6.1 Introduction -- 3.6.2 Existing Challenges to Voice Biometrics -- 3.6.3 New Areas of Research in Voice Biometrics -- 3.7 Speech Synthesis -- 3.8 Natural Language Understanding -- 3.8.1 Mixed Initiative Conversations -- 3.8.2 Limitations of Slot and Filler Technology -- 3.9 Multi-turn Dialog Management -- 3.10 Planning and Reasoning -- 3.10.1 Technical Challenges -- 3.10.2 Semantic Analysis and Discourse Representation -- 3.10.3 Pragmatics -- 3.10.4 Dialog Management as Collaboration -- 3.10.5 Planning and Re-planning -- 3.10.6 Knowledge Representation and Reasoning -- 3.10.7 Monitoring -- 3.10.8 Suggested Readings -- 3.11 Question Answering -- 3.11.1 Question Analysis.

3.11.2 Find Relevant Information -- 3.11.3 Answers and Evidence -- 3.11.4 Presenting the Answer -- 3.12 Distributed Voice Interface Architecture -- 3.12.1 Distributed User Interfaces -- 3.12.2 Distributed Speech and Language Technology -- 3.13 Conclusion -- Acknowledgements -- References -- Chapter 4 Visual Sensing and Gesture Interactions -- 4.1 Introduction -- 4.2 Imaging Technologies: 2D and 3D -- 4.3 Interacting with Gestures -- 4.4 Summary -- References -- Chapter 5 Real-Time 3D Sensing With Structured Light Techniques -- 5.1 Introduction -- 5.2 Structured Pattern Codifications -- 5.2.1 2D Pseudo-random Codifications -- 5.2.2 Binary Structured Codifications -- 5.2.3 N-ary Codifications -- 5.2.4 Continuous Sinusoidal Phase Codifications -- 5.3 Structured Light System Calibration -- 5.4 Examples of 3D Sensing with DFP Techniques -- 5.5 Real-Time 3D Sensing Techniques -- 5.5.1 Fundamentals of Digital-light-processing (DLP) Technology -- 5.5.2 Real-Time 3D Data Acquisition -- 5.5.3 Real-Time 3D Data Processing and Visualization -- 5.5.4 Example of Real-Time 3D Sensing -- 5.6 Real-Time 3D Sensing for Human Computer Interaction Applications -- 5.6.1 Real-Time 3D Facial Expression Capture and its HCI Implications -- 5.6.2 Real-Time 3D Body Part Gesture Capture and its HCI Implications -- 5.6.3 Concluding Human Computer Interaction Implications -- 5.7 Some Recent Advancements -- 5.7.1 Real-Time 3D Sensing and Natural 2D Color Texture Capture -- 5.7.2 Superfast 3D Sensing -- 5.8 Summary -- Acknowledgements -- References -- Chapter 6 Real-Time Stereo 3D Imaging Techniques -- 6.1 Introduction -- 6.2 Background -- 6.3 Structure of Stereo Correspondence Algorithms -- 6.3.1 Matching Cost Computation -- 6.3.2 Matching Cost Aggregation -- 6.4 Categorization of Characteristics -- 6.4.1 Depth Estimation Density.

6.4.2 Optimization Strategy -- 6.5 Categorization of Implementation Platform -- 6.5.1 CPU-only Methods -- 6.5.2 GPU-accelerated Methods -- 6.5.3 Hardware Implementations (FPGAs, ASICs) -- 6.6 Conclusion -- References -- Chapter 7 Time-of-Flight 3D-Imaging Techniques -- 7.1 Introduction -- 7.2 Time-of-Flight 3D Sensing -- 7.3 Pulsed Time-of-Flight Method -- 7.4 Continuous Time-of-Flight Method -- 7.5 Calculations -- 7.6 Accuracy -- 7.7 Limitations and Improvements -- 7.7.1 TOF Challenges -- 7.7.2 Theoretical Limits -- 7.7.3 Distance Aliasing -- 7.7.4 Multi-path and Scattering -- 7.7.5 Power Budget and Optimization -- 7.8 Time-of-Flight Camera Components -- 7.9 Typical Values -- 7.9.1 Light Power Range -- 7.9.2 Background Light -- 7.10 Current State of the Art -- 7.11 Conclusion -- References -- Chapter 8 Eye Gaze Tracking -- 8.1 Introduction and Motivation -- 8.2 The Eyes -- 8.3 Eye Trackers -- 8.3.1 Types of Eye Trackers -- 8.3.2 Corneal Reflection Method -- 8.4 Objections and Obstacles -- 8.4.1 Human Aspects -- 8.4.2 Outdoor Use -- 8.4.3 Calibration -- 8.4.4 Accuracy -- 8.4.5 Midas Touch Problem -- 8.5 Eye Gaze Interaction Research -- 8.6 Gaze Pointing -- 8.6.1 Solving the Midas Touch Problem -- 8.6.2 Solving the Accuracy Issue -- 8.6.3 Comparison of Mouse and Gaze Pointing -- 8.6.4 Mouse and Gaze Coordination -- 8.6.5 Gaze Pointing Feedback -- 8.7 Gaze Gestures -- 8.7.1 The Concept of Gaze Gestures -- 8.7.2 Gesture Detection Algorithm -- 8.7.3 Human Ability to Perform Gaze Gestures -- 8.7.4 Gaze Gesture Alphabets -- 8.7.5 Gesture Separation from Natural Eye Movement -- 8.7.6 Applications for Gaze Gestures -- 8.8 Gaze as Context -- 8.8.1 Activity Recognition -- 8.8.2 Reading Detection -- 8.8.3 Attention Detection -- 8.8.4 Using Gaze Context -- 8.9 Outlook -- References.

Chapter 9 Multimodal Input for Perceptual User Interfaces -- 9.1 Introduction -- 9.2 Multimodal Interaction Types -- 9.3 Multimodal Interfaces -- 9.3.1 Touch Input -- 9.3.2 3D Gesture -- 9.3.3 Eye Tracking and Gaze -- 9.3.4 Facial Expressions -- 9.3.5 Brain-computer Input -- 9.4 Multimodal Integration Strategies -- 9.4.1 Frame-based Integration -- 9.4.2 Unification-based Integration -- 9.4.3 Procedural Integration -- 9.4.4 Symbolic/Statistical Integration -- 9.5 Usability Issues with Multimodal Interaction -- 9.6 Conclusion -- References -- Chapter 10 Multimodal Interaction in Biometrics: Technological and Usability Challenges -- 10.1 Introduction -- 10.1.1 Motivations for Identity Assurance -- 10.1.2 Biometrics -- 10.1.3 Application Characteristics of Multimodal Biometrics -- 10.1.4 2D and 3D Face Recognition -- 10.1.5 A Multimodal Case Study -- 10.1.6 Adaptation to Blind Subjects -- 10.1.7 Chapter Organization -- 10.2 Anatomy of the Mobile Biometry Platform -- 10.2.1 Face Analysis -- 10.2.2 Voice Analysis -- 10.2.3 Model Adaptation -- 10.2.4 Data Fusion -- 10.2.5 Mobile Platform Implementation -- 10.2.6 MoBio Database and Protocol -- 10.3 Case Study: Usability Study for the Visually Impaired -- 10.3.1 Impact of Head Pose Variations on Performance -- 10.3.2 User Interaction Module: Head Pose Quality Assessment -- 10.3.3 User-Interaction Module: Audio Feedback Mechanism -- 10.3.4 Usability Testing with the Visually Impaired -- 10.4 Discussions and Conclusions -- Acknowledgements -- References -- Chapter 11 Towards "True'' 3D Interactive Displays -- 11.1 Introduction -- 11.2 The Origins of Biological Vision -- 11.3 Light Field Imaging -- 11.4 Towards "True'' 3D Visual Displays -- 11.5 Interacting with Visual Content on a 3D Display -- 11.6 Summary -- References -- Index -- Supplemental Images -- EULA.