Text Data Mining

Focuses on text data mining from an NLP perspectiveOffers a rich blend of fundamental theories, key techniques and predominant applicationsPresents the latest advances in the field of text data miningThis book discusses various aspects of text data mining. Unlike other books that focus on machine learning or databases, it approaches text data mining from a natural language processing (NLP) perspective.The book offers a detailed introduction to the fundamental theories and methods of text data mining, ranging from pre-processing (for both Chinese and English texts), text representation and feature selection, to text classification and text clustering. It also presents the predominant applications of text data mining, for example, topic modeling, sentiment analysis and opinion mining, topic detection and tracking, information extraction, and automatic text summarization. Bringing all the related concepts and algorithms together, it offers a comprehensive, authoritative and coherent overview. Written by three leading experts, it is valuable both as a textbook and as a reference resource for students, researchers and practitioners interested in text data mining. It can also be used for classes on text data mining or NLP. Foreword Preface Acknowledgments Contents About the Authors Acronyms 1 Introduction 1.1 The Basic Concepts 1.2 Main Tasks of Text Data Mining 1.3 Existing Challenges in Text Data Mining 1.4 Overview and Organization of This Book 1.5 Further Reading Exercises 2 Data Annotation and Preprocessing 2.1 Data Acquisition 2.2 Data Preprocessing 2.3 Data Annotation 2.4 Basic Tools of NLP 2.4.1 Tokenization and POS Tagging 2.4.2 Syntactic Parser 2.4.3 N-gram Language Model 2.5 Further Reading Exercises 3 Text Representation 3.1 Vector Space Model 3.1.1 Basic Concepts 3.1.2 Vector Space Construction 3.1.3 Text Length Normalization 3.1.4 Feature Engineering 3.1.5 Other Text Representation Methods 3.2 Distributed Representation of Words 3.2.1 Neural Network Language Model 3.2.2 C&W Model 3.2.3 CBOW and Skip-Gram Model 3.2.4 Noise Contrastive Estimation and Negative Sampling 3.2.5 Distributed Representation Based on the Hybrid Character-Word Method 3.3 Distributed Representation of Phrases 3.3.1 Distributed Representation Based on the Bag-of-Words Model 3.3.2 Distributed Representation Based on Autoencoder 3.4 Distributed Representation of Sentences 3.4.1 General Sentence Representation 3.4.2 Task-Oriented Sentence Representation 3.5 Distributed Representation of Documents 3.5.1 General Distributed Representation of Documents 3.5.2 Task-Oriented Distributed Representation of Documents 3.6 Further Reading Exercises 4 Text Representation with Pretraining and Fine-Tuning 4.1 ELMo: Embeddings from Language Models 4.1.1 Pretraining Bidirectional LSTM Language Models 4.1.2 Contextualized ELMo Embeddings for Downstream Tasks 4.2 GPT: Generative Pretraining 4.2.1 Transformer 4.2.2 Pretraining the Transformer Decoder 4.2.3 Fine-Tuning the Transformer Decoder 4.3 BERT: Bidirectional Encoder Representations from Transformer 4.3.1 BERT: Pretraining 4.3.2 BERT: Fine-Tuning 4.3.3 XLNet: Generalized Autoregressive Pretraining 4.3.4 UniLM 4.4 Further Reading Exercises 5 Text Classification 5.1 The Traditional Framework of Text Classification 5.2 Feature Selection 5.2.1 Mutual Information 5.2.2 Information Gain 5.2.3 The Chi-Squared Test Method 5.2.4 Other Methods 5.3 Traditional Machine Learning Algorithms for Text Classification 5.3.1 Naïve Bayes 5.3.2 Logistic/Softmax and Maximum Entropy 5.3.3 Support Vector Machine 5.3.4 Ensemble Methods 5.4 Deep Learning Methods 5.4.1 Multilayer Feed-Forward Neural Network 5.4.2 Convolutional Neural Network 5.4.3 Recurrent Neural Network 5.5 Evaluation of Text Classification 5.6 Further Reading Exercises 6 Text Clustering 6.1 Text Similarity Measures 6.1.1 The Similarity Between Documents 6.1.2 The Similarity Between Clusters 6.2 Text Clustering Algorithms 6.2.1 K-Means Clustering 6.2.2 Single-Pass Clustering 6.2.3 Hierarchical Clustering 6.2.4 Density-Based Clustering 6.3 Evaluation of Clustering 6.3.1 External Criteria 6.3.2 Internal Criteria 6.4 Further Reading Exercises 7 Topic Model 7.1 The History of Topic Modeling 7.2 Latent Semantic Analysis 7.2.1 Singular Value Decomposition of the Term-by-Document Matrix 7.2.2 Conceptual Representation and Similarity Computation 7.3 Probabilistic Latent Semantic Analysis 7.3.1 Model Hypothesis 7.3.2 Parameter Learning 7.4 Latent Dirichlet Allocation 7.4.1 Model Hypothesis 7.4.2 Joint Probability 7.4.3 Inference in LDA 7.4.4 Inference for New Documents 7.5 Further Reading Exercises 8 Sentiment Analysis and Opinion Mining 8.1 History of Sentiment Analysis and Opinion Mining 8.2 Categorization of Sentiment Analysis Tasks 8.2.1 Categorization According to Task Output 8.2.2 According to Analysis Granularity 8.3 Methods for Document/Sentence-Level Sentiment Analysis 8.3.1 Lexicon- and Rule-Based Methods 8.3.2 Traditional Machine Learning Methods 8.3.3 Deep Learning Methods 8.4 Word-Level Sentiment Analysis and Sentiment Lexicon Construction 8.4.1 Knowledgebase-Based Methods 8.4.2 Corpus-Based Methods 8.4.3 Evaluation of Sentiment Lexicons 8.5 Aspect-Level Sentiment Analysis 8.5.1 Aspect Term Extraction 8.5.2 Aspect-Level Sentiment Classification 8.5.3 Generative Modeling of Topics and Sentiments 8.6 Special Issues in Sentiment Analysis 8.6.1 Sentiment Polarity Shift 8.6.2 Domain Adaptation 8.7 Further Reading Exercises 9 Topic Detection and Tracking 9.1 History of Topic Detection and Tracking 9.2 Terminology and Task Definition 9.2.1 Terminology 9.2.2 Task 9.3 Story/Topic Representation and Similarity Computation 9.4 Topic Detection 9.4.1 Online Topic Detection 9.4.2 Retrospective Topic Detection 9.5 Topic Tracking 9.6 Evaluation 9.7 Social Media Topic Detection and Tracking 9.7.1 Social Media Topic Detection 9.7.2 Social Media Topic Tracking 9.8 Bursty Topic Detection 9.8.1 Burst State Detection 9.8.2 Document-Pivot Methods 9.8.3 Feature-Pivot Methods 9.9 Further Reading Exercises 10 Information Extraction 10.1 Concepts and History 10.2 Named Entity Recognition 10.2.1 Rule-based Named Entity Recognition 10.2.2 Supervised Named Entity Recognition Method 10.2.3 Semisupervised Named Entity Recognition Method 10.2.4 Evaluation of Named Entity Recognition Methods 10.3 Entity Disambiguation 10.3.1 Clustering-Based Entity Disambiguation Method 10.3.2 Linking-Based Entity Disambiguation 10.3.3 Evaluation of Entity Disambiguation 10.4 Relation Extraction 10.4.1 Relation Classification Using Discrete Features 10.4.2 Relation Classification Using Distributed Features 10.4.3 Relation Classification Based on Distant Supervision 10.4.4 Evaluation of Relation Classification 10.5 Event Extraction 10.5.1 Event Description Template 10.5.2 Event Extraction Method 10.5.3 Evaluation of Event Extraction 10.6 Further Reading Exercises 11 Automatic Text Summarization 11.1 Main Tasks in Text Summarization 11.2 Extraction-Based Summarization 11.2.1 Sentence Importance Estimation 11.2.2 Constraint-Based Summarization Algorithms 11.3 Compression-Based Automatic Summarization 11.3.1 Sentence Compression Method 11.3.2 Automatic Summarization Based on Sentence Compression 11.4 Abstractive Automatic Summarization 11.4.1 Abstractive Summarization Based on Information Fusion 11.4.2 Abstractive Summarization Based on the Encoder-Decoder Framework 11.5 Query-Based Automatic Summarization 11.5.1 Relevance Calculation Based on the Language Model 11.5.2 Relevance Calculation Based on Keyword Co-occurrence 11.5.3 Graph-Based Relevance Calculation Method 11.6 Crosslingual and Multilingual Automatic Summarization 11.6.1 Crosslingual Automatic Summarization 11.6.2 Multilingual Automatic Summarization 11.7 Summary Quality Evaluation and Evaluation Workshops 11.7.1 Summary Quality Evaluation Methods 11.7.2 Evaluation Workshops 11.8 Further Reading Exercises References This book discusses various aspects of text data mining. Unlike other books that focus on machine learning or databases, it approaches text data mining from a natural language processing (NLP) perspective. The book offers a detailed introduction to the fundamental theories and methods of text data mining, ranging from pre-processing (for both Chinese and English texts), text representation and feature selection, to text classification and text clustering. It also presents the predominant applications of text data mining, for example, topic modeling, sentiment analysis and opinion mining, topic detection and tracking, information extraction, and automatic text summarization. Bringing all the related concepts and algorithms together, it offers a comprehensive, authoritative and coherent overview. Written by three leading experts, it is valuable both as a textbook and as a reference resource for students, researchers and practitioners interested in text data mining. It can also be used for classes on text data mining or NLP. Chapter 1. Introduction -- Chapter 2. Data Annotation and Preprocessing -- Chapter 3. Text Representation -- Chapter 4. Text Representation with Pretraining and Fine-tuning -- Chapter 5. Text classification -- Chapter 6. Text Clustering -- Chapter 7. Topic Model -- Chapter 8. Sentiment Analysis and Opinion Mining -- Chapter 9. Topic Detection and Tracking -- Chapter 10. Information Extraction -- Chapter 11. Automatic Text Summarization.