Intent Recognition for Human-Machine Interactions (SpringerBriefs in Computer Science)

Natural interaction is one of the hottest research issues in human-computer interaction. At present, there is an urgent need for intelligent devices (service robots, virtual humans, etc.) to be able to understand intentions in an interactive dialogue. Focusing on human-computer understanding based on deep learning methods, the book systematically introduces readers to intention recognition, unknown intention detection, and new intention discovery in human-computer dialogue. This book is the first to present interactive dialogue intention analysis in the context of natural interaction. In addition to helping readers master the key technologies and concepts of human-machine dialogue intention analysis and catch up on the latest advances, it includes valuable references for further research. Preface Contents List of Figures List of Tables About the Authors Part I: Overview Chapter 1: Dialogue System 1.1 Review of Dialogue System References Chapter 2: Intent Recognition 2.1 Review of the Literature on Intent Representation 2.1.1 Discrete Representation One Hot Representation Bag of Word Term Frequency-Inverse Document Frequency (TF-IDF) N-gram 2.1.2 Distributed Representation Matrix-Based Distributed Representation Neural Network-Based Distributed Representation 2.1.3 Summary 2.2 Review of Known Intent Classification 2.2.1 Review of Single-Model Intent Classification 2.2.2 Review of Bi-model Intent Classification 2.2.3 Summary 2.3 Review of Unknown Intent Detection 2.3.1 Unknown Intent Detection Based on Traditional Discriminant Model 2.3.2 Unknown Intent Detection Based on Open Set Recognition in Computer Vision 2.3.3 Unknown Intent Detection Based on Out-of-Domain Detection 2.3.4 Unknown Intent Detection Based on Other Methods 2.3.5 Summary 2.4 Review of New Intent Discovery 2.4.1 New Intent Discovery Based on Unsupervised Clustering 2.4.2 New Intent Discovery Based on Semi-Supervised Clustering 2.4.3 Summary 2.5 Conclusion References Part II: Intent Classification Chapter 3: Intent Classification Based on Single Model 3.1 Introduction 3.2 Comparison Systems 3.2.1 Baseline Systems 3.2.2 NNLM-Based Utterance Classifier 3.2.3 RNN-Based Utterance Classifier 3.2.4 LSTM- and GRU-Based Utterance Classifier 3.3 Experiments 3.3.1 Datasets 3.3.2 Experiment Settings 3.3.3 Experiment Results 3.4 Conclusion References Chapter 4: A Dual RNN Semantic Analysis Framework for Intent Classification and Slot 4.1 Introduction 4.2 Intent Classification and Slot Filling Task Methods 4.2.1 Deep Neural Network for Intent Detection 4.2.2 Recurrent Neural Network for Slot Filling 4.2.3 Joint Model for Two Tasks 4.3 Bi-Model RNN Structures for Joint Semantic Frame Parsing 4.3.1 Bi-model Structure with a Decoder 4.3.2 Bi-Model Structure without a Decoder 4.3.3 Asynchronous Training 4.4 Experiments 4.4.1 Datasets 4.4.2 Experiment Settings 4.4.3 Experiment Results 4.5 Conclusion References Part III: Unknown Intent Detection Chapter 5: Unknown Intent Detection Method Based on Model Post-Processing 5.1 Introduction 5.2 A Post-Processing for New Intent Detection 5.2.1 Classifiers BiLSTM CNN + CNN 5.2.2 SofterMax Temperature Scaling Probability Calibration Decision Boundary 5.2.3 Deep Novelty Detection 5.2.4 SMDN 5.3 Experiments 5.3.1 Datasets 5.3.2 Baselines 5.3.3 Experiment Settings Evaluation Hyper-Parameters 5.3.4 Experiment Results Single-Turn Dialogue Datasets Multi-Turn Dialogue Dataset 5.4 Conclusion References Chapter 6: Unknown Intent Detection Based on Large-Margin Cosine Loss 6.1 Introduction 6.2 New Intent Detection Model Based on Large Margin Cosine Loss Function 6.2.1 Large Margin Cosine Loss (LMCL) 6.3 Experiments 6.3.1 Datasets 6.3.2 Baselines 6.3.3 Experiment Setting Hyper-Parameter Setting 6.3.4 Experiment Results 6.4 Conclusion References Chapter 7: Unknown Intention Detection Method Based on Dynamic Constraint Boundary 7.1 Introduction 7.2 The Frame Structure of the Model 7.3 The Main Approach 7.3.1 Intent Representation 7.3.2 Pre-Training 7.3.3 Adaptive Decision Boundary Learning Decision Boundary Formulation Boundary Learning 7.3.4 Open Classification with Decision Boundary 7.4 Experiments 7.4.1 Datasets 7.4.2 Baselines 7.4.3 Experiment Settings Evaluation Metrics 7.4.4 Experiment Results 7.5 Discussion 7.5.1 Boundary Learning Process 7.5.2 Effect of Decision Boundary 7.5.3 Effect of Labeled Data 7.5.4 Effect of Known Classes 7.6 Conclusion References Part IV: Discovery of Unknown Intents Chapter 8: Discovering New Intents Via Constrained Deep Adaptive Clustering with Cluster Refinement 8.1 Introduction 8.2 New Intent Discovery Model Based on Self-Supervised Constrained Clustering 8.2.1 Intent Representation 8.2.2 Pairwise-Classification with Similarity Loss Supervised Step Unsupervised Step 8.2.3 Cluster Refinement with KLD Loss 8.3 Experiments 8.3.1 Datasets 8.3.2 Baseline 8.3.3 Experiment Settings Evaluation Metrics Hyper Parameter 8.3.4 Experiment Results Ablation Study Effect of the Number of Clusters Effect of Labeled Data Effect of Unknown Data Performance on Imbalanced Dataset Error Analysis 8.4 Conclusion References Chapter 9: Discovering New Intents with Deep Aligned Clustering 9.1 Introduction 9.2 Deep Aligned Clustering 9.2.1 Intent Representation 9.2.2 Transferring Knowledge From Known Intents Pre-training Predict K 9.2.3 Deep Aligned Clustering Unsupervised Learning by Clustering Self-Supervised Learning with Aligned Pseudo-Labels 9.3 Experiments 9.3.1 Datasets 9.3.2 Baselines Unsupervised Semi-supervised 9.3.3 Experiment Settings Evaluation Metrics Hyper Parameters Experiment Results Effect of the Alignment Strategy Estimate K Effect of Known Class Ratio Effect of the Number of Classes 9.4 Conclusion References Part V: Dialogue Intent Recognition Platform Chapter 10: Experiment Platform for Dialogue Intent Recognition Based on Deep Learning 10.1 Introduction 10.2 Open Intent Recognition Platform 10.2.1 Data Management 10.2.2 Models Open Intent Detection Open Intent Discovery 10.2.3 Training and Evaluation 10.2.4 Result Analysis Open Intent Detection Open Intent Discovery 10.3 Pipeline Framework 10.4 Experiments 10.5 Conclusion References Part VI: Summary and Future Work Chapter 11: Summary Appendix