Chemometric Methods in Analytical Spectroscopy Technology

This book discusses chemometric methods for spectroscopy analysis including NIR, MIR, Raman, NMR, and LIBS, from the perspective of practical applied spectroscopy. It covers all aspects of chemometrics associated with analytical spectroscopy, including representative sample selection algorithm, outlier detection algorithm, model updating and maintenance algorithm and strategy and calibration performance evaluation methods.To provide a systematic and comprehensive overview the latest progress of chemometric methods including recent scientific research and practical applications are presented. In addition the book also highlights the improvement of classical algorithms and the extension of common strategies. It is therefore useful as a reference book for researchers engaged in analytical spectroscopy technology, chemometrics, analytical instruments and other related fields. Preface Contents 1 Chemometric Methods in Analytical Spectroscopy Technology 1.1 Introduction 1.1.1 Overview of Chemometrics 1.1.2 Analysis of Spectroscopy Combined with Chemometrics 1.1.3 Beginning of Modern Spectroscopy Technology—The Contribution of Karl Norris References 2 Modern Spectral Analysis Techniques 2.1 Introduction 2.2 Near-Infrared Spectroscopy 2.2.1 Micro Near-Infrared Spectral Analysis Technology 2.2.2 Online Near-Infrared Spectral Analysis Technology 2.2.3 Standard Methods for Near-Infrared Spectroscopy 2.3 Mid-Infrared Spectroscopy 2.3.1 Portable Mid-Infrared Spectral Analysis Technology 2.3.2 Online Mid-Infrared Spectral Analysis Technology 2.4 Raman Spectroscopy 2.4.1 Fourier Transform Raman Spectroscopy 2.4.2 Surface Enhanced Raman Scattering Spectroscopy 2.4.3 Confocal Raman Spectroscopy 2.4.4 Spatial Offset Raman Spectroscopy 2.4.5 Transmitted Raman Spectroscopy 2.4.6 Portable Raman Spectral Analysis Technology 2.4.7 Fiber Raman Spectral Analysis Technology 2.5 Ultraviolet-Visible Spectroscopy 2.6 Molecular Fluorescence Spectroscopy 2.6.1 Three-Dimensional Fluorescence Spectroscopy 2.6.2 Laser-Induced Fluorescence Spectroscopy 2.7 Low-Field NMR Spectroscopy 2.8 Terahertz Spectroscopy 2.9 Laser-Induced Breakdown Spectroscopy 2.10 Spectral Imaging References 3 Basis of Matrices and Mathematical Statistics 3.1 Basis of Matrix 3.2 Matrix Representation of Lambert-Beer’s Law 3.3 Variance and Normal Distribution 3.4 Significance Test 3.5 Correlation Coefficient 3.6 Covariance and Covariance Matrix 3.7 Multivariable Graph Representation 3.7.1 Spatial Representation of Samples 3.7.2 Box Plot 3.7.3 Radar Chart References 4 Spectral Preprocessing Methods 4.1 Mean Centering 4.2 Auto-scaling 4.3 Normalization 4.4 Smoothing 4.4.1 Moving Average Smoothing 4.4.2 Savitzky-Golay Convolution Smoothing 4.4.3 Fourier Transform and Wavelet Transform 4.5 Continuum Removed 4.6 Adaptive Iteratively Reweighted Penalized Least Squares 4.7 Derivative 4.7.1 Norris Method 4.7.2 Savitzky-Golay Convolution for Derivative Calculation 4.7.3 Wavelet Transform for Derivative Calculation 4.7.4 Fractional Derivative 4.8 Standard Normal Variate and De-Trending 4.9 Multiplicative Scatter Correction 4.10 Vector Angle Conversion 4.11 Fourier Transform 4.12 Wavelet Transform 4.13 Image Moment Methods 4.14 External Parameter Orthogonalization 4.15 Generalized Least Squares Weighting 4.16 Loading Space Standardization 4.17 Oblique Projection 4.18 Orthogonal Signal Correction 4.18.1 Wold Algorithm 4.18.2 Fearn Algorithm 4.18.3 Direct Orthogonal Signal Correction Algorithm 4.18.4 Direct Orthogonal Algorithm 4.18.5 Application of Orthogonal Signal Correction Algorithm 4.19 Net Analyte Signal 4.20 Optical Path-Length Estimation and Correction 4.21 Two-Dimensional Correlation Spectroscopy References 5 Wavelength Selection Methods 5.1 Correlation Coefficient and Analysis of Variance Method 5.2 Simple-To-Use Interactive Self-modeling Mixture Analysis Method 5.3 Successive Projections Algorithm 5.4 Variable Importance in Projection 5.5 Interval Partial Least Squares Method 5.6 Moving Window PLS 5.7 Recursive Weighted PLS 5.8 Elimination of Uninformative Variables 5.9 Global Optimization Methods 5.9.1 Genetic Algorithm 5.9.2 Simulated Annealing Algorithm 5.9.3 Particle Swarm Optimization 5.9.4 Ant Colony Algorithm 5.10 Model Population Analysis-Based Methods 5.10.1 Competitive Adaptive Reweighted Sampling 5.10.2 Iteratively Retaining Informative Variables 5.10.3 Variable Combination Population Analysis 5.10.4 Other Methods 5.10.5 Wavelength Selection Method Based on Hybrid Strategy 5.11 The Selection of Spectral Preprocessing and Wavelength Selection Methods References 6 Spectral Dimensionality Reduction Methods 6.1 The Multicollinearity Problem 6.2 Principal Component Analysis 6.2.1 Theory of Principal Component Analysis 6.2.2 Determination of Principal Component Number 6.2.3 Algorithm of Principal Component Analysis 6.2.4 Application of Principal Component Analysis 6.2.5 Multivariate Resolution Alternating Least Squares 6.2.6 Band Target Entropy Minimization 6.2.7 Multilevel Simultaneous Component Analysis 6.3 Non-negative Matrix Factorization 6.4 Independent Component Analysis 6.5 Multi-dimensional Scaling Transformation 6.6 Isometric Mapping 6.7 Local Linear Embedding 6.8 T-Distributed Stochastic Neighborhood Embedding 6.9 Other Algorithms References 7 Linear Calibration Methods 7.1 Univariate Linear Regression 7.2 Multiple Linear Regression 7.3 Concentration Residual Augmented Classical Least Squares 7.4 Stepwise Linear Regression 7.5 Ridge Regression 7.6 Lasso Regression 7.7 Least Angle Regression 7.8 Elastic Net 7.9 Principal Component Regression 7.9.1 Theory 7.9.2 Method for Selecting the Optimal PCs 7.9.3 Partial Least Squares Regression References 8 Nonlinear Calibration Methods 8.1 Artificial Neural Network 8.1.1 Introduction 8.1.2 Back Propagation-Artificial Neural Network 8.1.3 Design of BP-ANN 8.1.4 Other Types of Neural Networks 8.1.5 Optimization of Neural Network Parameters 8.2 Support Vector Machine 8.2.1 Introduction 8.2.2 Support Vector Regression 8.2.3 Least Squares Support Vector Regression 8.2.4 Optimization of Support Vector Regression Parameters 8.3 Relevance Vector Machine 8.4 Kernel Partial Least Squares 8.5 Extreme Learning Machine 8.6 Gaussian Process Regression References 9 Method of Selecting Calibration Samples 9.1 Introduction 9.2 Kennard-Stone Method 9.3 Sample Set Partitioning Based on Joint X–Y Distances (SPXY) Method 9.4 Optimizable K-dissimilarity Selection Method 9.5 Other Methods References 10 Detection Methods for Outlier Samples 10.1 Detection of Outlier Samples During Calibration Process 10.2 Detection of Outlier Samples During the Prediction Process 10.3 Other Detection Methods References 11 Maintenance and Update of Calibration Model 11.1 Necessity 11.2 Recursive Exponentially Weighted PLS 11.3 Block-Wise Recursive PLS 11.4 Just-In-Time Learning and Active Learning References 12 Pattern Recognition Methods 12.1 Introduction 12.2 Unsupervised Pattern Recognition Methods 12.2.1 Similarity Coefficients and Distances 12.2.2 Hierarchical Cluster Analysis 12.2.3 K-Means Clustering 12.2.4 Fuzzy K-Means Clustering 12.2.5 Gaussian Mixture Model 12.2.6 Self-organizing Neural Network 12.3 Supervised Pattern Recognition Methods 12.3.1 Minimum Distance Discriminant Method 12.3.2 Canonical Variate Analysis 12.3.3 K-Nearest Neighbor 12.3.4 Soft Independent Modeling of Class Analogy 12.3.5 Logistic Regression 12.3.6 Soft-Max Classifier 12.3.7 Random Forest 12.3.8 Application of Regression Methods for Discriminant Analysis 12.4 Spectral Searching Methods 12.4.1 Introduction 12.4.2 Spectral Searching Algorithms 12.4.3 Improvements of Spectral Searching Algorithms 12.4.4 Spectral Searching Strategies and Applications References 13 Model Evaluation 13.1 Evaluation of Quantitative Calibration Model 13.1.1 Evaluation Parameters 13.1.2 Model Evaluation 13.2 Evaluation of Performance of Pattern Recognition Model References 14 Methods for Improving Prediction Ability of Model 14.1 Modeling Strategies for Improving the Robustness 14.2 Modeling Strategies Based on Local Samples 14.3 Ensemble Modeling Strategies 14.3.1 Bagging Ensemble Strategy 14.3.2 Boosting Ensemble Strategy 14.3.3 Stacked Ensemble Strategy 14.3.4 Stacked Generalization Strategy 14.4 Virtual Sample Modeling Strategy 14.5 Semi-supervised Learning Methods 14.6 Multi-target Regression Strategy References 15 Multi-spectral Fusion Technology 15.1 Fusion Strategies and Methods 15.2 Multi-block Partial Least Squares Method 15.3 Sequential and Orthogonal Partial Least Squares Method 15.4 Research on Application of Multi-Spectral Fusion 15.5 Future Prospect References 16 Multi-way Resolution and Calibration Methods 16.1 Introduction 16.2 Parallel Factor Analysis 16.3 Alternating Trilinear Decomposition 16.4 Multi-way Partial Least Squares References 17 Calibration Transfer Methods 17.1 Introduction 17.2 Traditional Algorithms 17.2.1 Spectral Subtraction Correction 17.2.2 Shenk’s Algorithm 17.2.3 Direct Standardization 17.2.4 Piecewise Direct Standardization 17.2.5 Procrustes Analysis 17.2.6 Target Transformation Factor Analysis 17.2.7 Maximum Likelihood Principal Component Analysis 17.2.8 Slope/Bias Correction 17.3 Improvement of Traditional Algorithms 17.4 New Algorithms 17.4.1 Canonical Correlation Analysis 17.4.2 Spectral Space Transformation 17.4.3 Alternating Trilinear Decomposition 17.4.4 Multi-task Learning 17.4.5 Generalized Least Squares 17.4.6 Other Algorithms 17.5 Global Calibration, Robust Calibration, and Model Update 17.6 Progress of Applications 17.6.1 SBC Method 17.6.2 SSC Method 17.6.3 Shenk’s Method 17.6.4 DS Method 17.6.5 PDS Method 17.6.6 CCA Method 17.6.7 Establishment of Global Model 17.6.8 Other Methods References 18 Deep Learning Methods 18.1 Stacked Auto-encoder 18.2 Convolution Neural Network 18.2.1 Basic Structure of CNN 18.2.2 Optimistic Algorithm 18.2.3 Loss Function 18.2.4 Activation Function 18.2.5 Methods to Avoid Over-Fitting 18.2.6 Classical Convolution Neural Network Architecture 18.2.7 Popular Deep Learning Software Framework 18.2.8 Design of Convolution Neural Networks 18.2.9 Training of Convolution Neural Networks 18.2.10 Advantages and Disadvantages of Convolution Neural Network 18.2.11 Applications of Convolution Neural Network 18.3 Deep Belief Network 18.4 Transfer Learning References 19 Chemometrics Software and Toolkits 19.1 Introduction 19.2 Basic Structure and Functions of Software 19.3 Common Software and Toolkits References 20 Discussion of Some Issues 20.1 Comparison of Different Spectroscopic Analysis 20.2 Selection of Chemometric Methods 20.2.1 Selection of Multivariate Calibration Methods 20.2.2 Selection of Pattern Recognition Methods 20.2.3 Selection of Spectral Preprocessing Methods and Spectral Variables 20.3 Influencing Factors of Model Prediction Ability 20.3.1 Effect of Calibration Samples 20.3.2 Effect of Reference Data 20.3.3 Effect of Spectral Measurement Methods 20.3.4 Effect of Spectral Acquisition Conditions 20.3.5 Effect of Instrument Performance 20.4 Outlook References