Touch-Based Human-Machine Interaction : Principles and Applications

This textbook presents a comprehensive treatment of touch technologies, explaining current mainstream and new contact/non-contact based human-machine interactivity (HMI) techniques, which are ubiquitous in modern electronic devices and allow machines to exchange information with users in an efficient and reliable manner. The book provides a detailed study of HMI working principles and practical product examples. Haptic, which has become essential for users to gain immersive experience, is also discussed. The book concludes with an overview of novel applications enabled by emerging technologies, such as advanced materials, virtual reality and machine learning, providing a roadmap for possible development trends for touch interactivities. The book can be used as a graduate text for students in display and touch interface technology courses in electrical and computer engineering, and a professional reference for researchers, practicing engineers, and product designers working in broadareas of engineering. Helps students understand the working principles of current touch technologies; Offers design considerations for prototypes and products; Provides seamless connectivity between broad subject areas involved in HMI, including material science, microelectronic circuits, mechanical engineering, and digital signal processing. Contents Chapter 1: Ambient Touch Interactivities References Chapter 2: Properties of Touch Events 2.1 Finger ́s Physical Properties 2.1.1 Physical Characteristics of Human Fingers and Touch Events 2.1.2 Characterization of Touch Events 2.2 User ́s Touch Behavior 2.2.1 Dynamic Force Touch Behavior 2.2.2 Static Force Touch Behavior 2.2.2.1 Instable Force Amplitudes Induced by Inevitable Finger Movement 2.2.2.2 Force Hysteresis Phenomenon References Chapter 3: Touch Detection Technologies 3.1 Contact-Based Touch Detection Techniques 3.1.1 Resistive-Based Techniques 3.1.1.1 Working Principle Four-Wire Structure Calculation of the Y Coordinate Calculation of the X Coordinate Five-Wire Construction Calculation of the Y Coordinate Calculation of the X Coordinate 3.1.1.2 Resistive Touch-Sensing System 3.1.1.3 Pros and Cons 3.1.2 Capacitive-Based Techniques 3.1.2.1 Surface Capacitance Touch Panel 3.1.2.2 Projected Capacitance Touch Panel Self-Capacitive Architecture Mutual-Capacitive Architecture Projected Capacitive Touch Panel Construction 3.1.2.3 Characteristics of Capacitive Touch Signals 3.1.3 Acoustic-Based Techniques 3.1.3.1 Active Acoustic Techniques Surface Acoustic Wave (SAW) Finger Reflection-Based Acoustic Touch Panel Bezel-Less Acoustic Touch Panel 3.1.3.2 Passive Acoustic Techniques Acoustic Pulse Recognition Dispersive Signal Technology 3.1.3.3 Pros and Cons 3.1.4 Optical-Based Techniques 3.1.4.1 Brief History and Current Status 3.1.4.2 Infrared Touch Sensing Working Principle Pros and Cons 3.1.4.3 Digital Waveguide Touch Working Principle Pros and Cons 3.1.4.4 Camera (CCD/CMOS)-Based Optical Touch Sensing Working Principle Pros and Cons 3.1.4.5 Frustrated Total Internal Reflection Working Principle Pros and Cons 3.1.4.6 Diffused Illumination Working Principle Pros and Cons 3.1.4.7 Laser Light Plane Working Principle Pros and Cons 3.1.4.8 LED Light Plane Working Principle Pros and Cons 3.1.4.9 Conclusions 3.1.5 Piezoelectric-Based Techniques 3.1.5.1 A Brief History of Piezoelectric Touch Panel 3.1.5.2 Working Principle 3.1.5.3 Typical Architecture 3.1.5.4 Typical Readout Circuit and System Construction 3.1.5.5 Pros and Cons 3.1.5.6 Commercial Products 3.1.5.7 Recent Research Advances 3.1.6 Piezoresistive-Based Techniques 3.1.6.1 Working Principle 3.1.6.2 Typical Architecture 3.1.6.3 Pros and Cons 3.1.7 Pyroelectric-Based Techniques 3.1.7.1 Working Principle 3.1.7.2 Relevant Studies 3.1.7.3 Pros and Cons 3.1.8 Triboelectric-Based Techniques 3.1.8.1 Working Principle Dielectric-to-Dielectric Case Metal-to-Dielectric Case 3.1.8.2 State-of-the-Art Work 3.1.8.3 Pros and Cons 3.2 Non-contact Techniques 3.2.1 Camera-Based Techniques 3.2.2 Inertial Motion Unit-Based Techniques 3.2.3 Electromyogram-Based Detection Techniques 3.2.4 Electrical Capacitance Tomography-Based Techniques 3.2.4.1 Working Principle 3.2.4.2 LBP Algorithm 3.2.4.3 Landweber Iterative Algorithm 3.2.4.4 Sensor Structure and Stimulation Method Structures of 2D ECT Touch Panels Structures of 3D ECT Touch Panels 3.2.4.5 System Construction 3.2.4.6 Pros and Cons 3.2.5 Electrical Impedance Tomography-Based Techniques References Chapter 4: Haptic Feedback 4.1 A Brief Introduction of Feedback Technology 4.1.1 Feedback Technology in HMI Systems 4.1.2 Limitations of Conventional Visual and Auditory Feedback 4.2 Haptic Feedback Technology 4.2.1 Vibration Technology 4.2.1.1 Inertial Actuator Eccentric Rotating Mass Working Principle Pros and Cons Linear Resonant Actuators Working Principle Pros and Cons Piezoelectric Actuator Working Principle Pros and Cons Surface Actuator Working Principle Pros and Cons Electroactive Polymer (EAP) Actuator Working Principle Pros and Cons Capacitive Electrosensory Interface Working Principle Pros and Cons 4.2.2 Bioelectrical Stimulation Technique 4.2.2.1 Functional Electrical Stimulation Working Principle Pros and Cons 4.2.2.2 Electrical Muscle Stimulation Working Principle Pros and Cons References Chapter 5: Performance Optimization 5.1 Capacitive TSP for 2D Detection 5.1.1 Noise Reduction 5.1.2 Noise Source 5.1.2.1 Main Noise Source in TSP 5.1.2.2 Noise from Display Panel 5.1.3 Solution for Noise Reduction 5.1.3.1 Circuitry-Based Techniques 5.1.3.2 Algorithm-Based Techniques 5.1.3.3 Pixel Level-Based Noise Reduction Techniques 5.1.3.4 Frame Level-Based Noise Reduction Techniques 5.1.4 Techniques for Fast Readout Speed and Low Power Consumption 5.1.5 Time and Frequency Domain 5.1.6 Spatial Domain 5.1.7 Conclusion 5.2 Piezoelectric TSP for 3D Force Touch Detection 5.2.1 Touch Event-Related Instable Responsivity Issues 5.2.1.1 Contact Area and Touch Direction 5.2.1.2 Touch Speed 5.2.2 Touch Panel ́s Mechanical Property-Induced Low Force Detection Accuracy 5.2.2.1 Boundary Condition 5.2.2.2 Preload Effect 5.2.2.3 Interference from Adjacent Force Touches 5.2.3 Conclusion References Chapter 6: User Experience Evaluation 6.1 The Definition of User Experience 6.1.1 The Importance of User Experience Evaluation 6.2 The Development of User Experience Evaluation 6.2.1 From Behavior to Perception 6.2.2 Immersive Assessment 6.2.3 Studies for Special Design 6.3 Evaluation Methods 6.3.1 Usability Test 6.3.1.1 Screen-Based Interaction 6.3.1.2 No Screen-Based Interaction 6.3.2 Behavioral Observation 6.3.3 Behavioral Measures 6.3.3.1 Eye Tracking 6.3.3.2 Facial Expression 6.3.3.3 Physiological Measures 6.3.4 Physiological Signals 6.3.4.1 Electromyography 6.3.4.2 Electrodermal Activity and Galvanic Skin Response 6.3.4.3 Cardiovascular Measures 6.3.4.4 Electroencephalography 6.3.5 Affective Index 6.3.5.1 Cognitive Load 6.3.5.2 Attention 6.3.5.3 Stress 6.3.5.4 Engagement 6.3.6 Survey and Questionnaires 6.3.6.1 Questionnaires Demographic Questionnaire Likert Scale 6.3.6.2 User Experience Questionnaire 6.3.6.3 Presence Questionnaire 6.3.6.4 NASA TLX 6.3.6.5 Interview and Qualitative Observation References Chapter 7: Emerging Applications 7.1 Interactivity for Flexible Displays 7.1.1 Materials for Flexible Touch Panels 7.1.1.1 Flexible Electrode Carbon Conductors Metal Conductors Polymeric Conductors Ionic Conductors 7.1.1.2 Flexible Dielectric 7.1.1.3 Flexible Substrate and Cover 7.1.2 Flexible Touch-Sensing Techniques 7.1.2.1 Flexible Capacitive Touch Panels 7.1.2.2 Flexible Piezoelectric Touch Panels 7.1.2.3 Flexible Piezoresistive and Resistive Touch Panels 7.1.2.4 Flexible Triboelectric Touch Panels 7.1.2.5 Flexible Ion Touch Panels 7.1.3 Commercial Products Used in Interactive Displays 7.2 Usage in Extreme Conditions 7.2.1 Water 7.2.2 Vibration 7.2.3 Extreme Temperatures 7.2.4 Sunlight 7.3 Interactivity with Virtual Reality 7.3.1 The System Composition of VR 7.3.2 Applications of VR Devices 7.3.2.1 Data Visualization and Interactions 7.3.2.2 Training 7.3.2.3 E-Commerce 7.3.2.4 Education 7.3.2.5 Entertainment 7.4 Touch and Speech Combined Interactivity 7.4.1 Characteristics of Touch and Speech Interactions 7.4.2 Enabling Accurate and Efficient Input 7.4.3 Accommodating a Wide Range of Users and More Conditions 7.4.4 New Applications 7.5 Emotion Detection 7.6 Big-Data-Enabled Cyber Security References Chapter 8: Conclusion Index This textbook presents a comprehensive treatment of touch technologies, explaining current mainstream and new contact/non-contact based human-machine interactivity (HMI) techniques, which are ubiquitous in modern electronic devices and allow machines to exchange information with users in an efficient and reliable manner. The book provides a detailed study of HMI working principles and practical product examples. Haptic, which has become essential for users to gain immersive experience, is also discussed. The book concludes with an overview of novel applications enabled by emerging technologies, such as advanced materials, virtual reality and machine learning, providing a roadmap for possible development trends for touch interactivities. The book can be used as a graduate text for students in display and touch interface technology courses in electrical and computer engineering, and a professional reference for researchers, practicing engineers, and product designers working in broad areas of engineering. Helps students understand the working principles of current touch technologies; Offers design considerations for prototypes and products; Provides seamless connectivity between broad subject areas involved in HMI, including material science, microelectronic circuits, mechanical engineering, and digital signal processing