An Introduction to Machine Learning

Just like electricity, Machine Learning will revolutionize our life in many ways – some of which are not even conceivable today. This book provides a thorough conceptual understanding of Machine Learning techniques and algorithms. Many of the mathematical concepts are explained in an intuitive manner. The book starts with an overview of machine learning and the underlying Mathematical and Statistical concepts before moving onto machine learning topics. It gradually builds up the depth, covering many of the present day machine learning algorithms, ending in Deep Learning and Reinforcement Learning algorithms. The book also covers some of the popular Machine Learning applications. The material in this book is agnostic to any specific programming language or hardware so that readers can try these concepts on whichever platforms they are already familiar with. * Offers a comprehensive introduction to Machine Learning, while not assuming any prior knowledge of the topic; * Provides a complete overview of available techniques and algorithms in conceptual terms, covering various application domains of machine learning; * Not tied to any specific software language or hardware implementation. Preface......Page 3 Contents......Page 6 Figures......Page 14 Tables......Page 17 1.1 Introduction......Page 19 1.1.1 Resurgence of ML......Page 20 1.1.2 Relation with Artificial Intelligence (AI)......Page 21 1.2 Matrices......Page 22 1.2.3 Matrix Transpose......Page 23 1.2.4.2 Multiplying with Another Matrix......Page 24 1.2.6 Matrix Inversion......Page 25 1.2.7 Solving Equations Using Matrices......Page 26 1.3 Numerical Methods......Page 27 1.4 Probability and Statistics......Page 28 1.4.3 Expectation......Page 29 1.4.5 Maximum Likelihood......Page 30 1.5 Linear Algebra......Page 31 1.6.2 Slope......Page 32 1.7 Computer Architecture......Page 33 1.8 Next Steps......Page 34 2.1 Supervised Learning......Page 36 2.1.2 Regression Problem......Page 37 2.2 Unsupervised Learning......Page 38 2.4 Reinforcement Learning......Page 39 3.1 Introduction......Page 41 3.3 Problem Formulation......Page 42 3.4 Linear Regression......Page 43 3.4.1 Normal Method......Page 44 3.4.2 Gradient Descent Method......Page 46 3.4.2.2 Initial Value......Page 47 3.4.2.4 Learning Rate......Page 48 3.4.2.5 Convergence......Page 50 3.4.2.7 Putting Gradient Descent in Practice......Page 51 3.5 Logistic Regression......Page 52 3.5.1 Sigmoid Function......Page 53 3.5.3 Gradient Descent......Page 54 3.6 Next Steps......Page 55 3.7 Key Takeaways......Page 56 4.1 Nonlinear Contribution......Page 57 4.2 Feature Scaling......Page 58 4.3.1 Cost Contour......Page 59 4.3.2 Stochastic Gradient Descent......Page 60 4.3.2.1 Convergence for Stochastic Gradient Descent......Page 62 4.3.3 Mini Batch Gradient Descent......Page 63 4.3.4 Map Reduce and Parallelism......Page 64 4.4 Regularization......Page 65 4.4.3 Determining Appropriate λ......Page 68 4.4.4 Comparing Hypothesis......Page 69 4.5.1 One-vs-All Classification......Page 70 4.5.2.1 Basic Approach for SoftMax......Page 71 4.6 Key Takeaways and Next Steps......Page 72 5.1 Decision Boundary......Page 73 5.1.1 Nonlinear Decision Boundary......Page 74 5.2 Skewed Class......Page 76 5.2.2 Single Metric......Page 77 5.3 Naïve Bayes ́ Algorithm......Page 78 5.4 Support Vector Machines......Page 79 5.4.1 Kernel Selection......Page 82 6.1 K-Means......Page 83 6.1.2 Distance Calculation......Page 84 6.1.4 Cost Function......Page 85 6.1.5 Choice of Initial Random Centers......Page 86 6.1.6 Number of Clusters......Page 87 6.2 K-Nearest Neighbor (KNN)......Page 88 6.2.3 Limitations......Page 89 6.2.4 Finding the Nearest Neighbors......Page 90 6.3 Next Steps......Page 92 7.1 Decision Tree......Page 93 7.2 Information Gain......Page 95 7.3 Gini Impurity Criterion......Page 102 7.5 Random Forests......Page 105 7.5.3 Cross Validation in Random Forests......Page 106 7.6 Variable Importance......Page 107 7.7.1 Outliers......Page 108 7.8 Disadvantages of Random Forests......Page 109 7.9 Next Steps......Page 110 8.1 Test Set......Page 111 8.4 Determining the Number of Degrees......Page 112 8.5 Determining λ......Page 113 8.6.1 High Bias Case......Page 114 8.7 The Underlying Mathematics (Optional)......Page 115 8.9 Derived Data......Page 117 8.11 Test Data......Page 118 9.1 Logistic Regression Extended to Form Neural Network......Page 119 9.2 Neural Network as Oversimplified Brain......Page 121 9.3 Visualizing Neural Network Equations......Page 122 9.4 Matrix Formulation of Neural Network......Page 123 9.5 Neural Network Representation......Page 124 9.6 Starting to Design a Neural Network......Page 125 9.7 Training the Network......Page 126 9.7.1 Chain Rule......Page 127 9.7.2 Components of Gradient Computation......Page 128 9.7.3 Gradient Computation Through Backpropagation......Page 130 9.7.4 Updating Weights......Page 131 9.10 Next Steps......Page 132 10.1 Complexity of NLP......Page 133 10.2.2 Tokenizer......Page 135 10.2.3 Named Entity Recognizers......Page 136 10.2.4 Term Frequency-Inverse Document Frequency (tf-idf)......Page 137 10.2.5 Word Embedding......Page 138 10.2.6.1 Continuous Bag of Words......Page 139 10.2.6.2 Skip-Gram Model......Page 140 11.1 Recurrent Neural Networks......Page 142 11.1.1 Representation of RNN......Page 144 11.1.2 Backpropagation in RNN......Page 147 11.1.3 Vanishing Gradients......Page 148 11.2 LSTM......Page 149 11.3 GRU......Page 151 11.4.1 Representation and Training of SOM......Page 153 12 Principal Component Analysis......Page 156 12.1.2 Example 2......Page 157 12.2.2 Covariance Matrix......Page 158 12.2.6 Deriving Principal Components......Page 159 12.3.1 Data Characteristics......Page 160 12.3.3 Selecting Principal Components......Page 161 12.4.1 Image Compression......Page 163 12.4.2 Data Visualization......Page 164 12.5.1 Overfitting......Page 166 12.5.3 Model Interpretation......Page 167 13 Anomaly Detection......Page 168 13.2 Model......Page 169 13.2.1 Distribution Density......Page 170 13.2.3 Metric Value......Page 171 13.2.5 Validating and Tuning the Model......Page 172 13.3 Multivariate Gaussian Distribution......Page 173 13.3.2 Determining Covariance......Page 174 13.4 Anomalies in Time Series......Page 175 13.4.2 Time Series Anomaly Types......Page 176 13.4.3 Anomaly Detection in Time Series......Page 178 13.4.3.1 ARIMA......Page 179 13.4.3.2 Machine Learning Models......Page 182 14 Recommender Systems......Page 183 14.1.1 User ́s Affinity Toward Each Feature......Page 184 14.2 User ́s Preferences Known......Page 185 14.2.1 Characterizing Features......Page 186 14.3.1.2 Parameters Under Consideration......Page 187 14.3.1.5 Cost Function......Page 188 14.3.2 Predicting and Recommending......Page 189 14.4 New User......Page 190 14.4.1 Shortcomings of the Current Algorithm......Page 191 14.5 Tracking Changes in Preferences......Page 192 15.1 Convolution Explained......Page 194 15.2.1 Exact Shape Known......Page 196 15.2.3 Breaking Down Further......Page 197 15.3 Image Convolution......Page 198 15.4 Preprocessing......Page 200 15.5 Post-Processing......Page 201 15.6 Stride......Page 202 15.7 CNN......Page 203 15.8 Matrix Operation......Page 204 15.9 Refining the Filters......Page 205 15.12 ADAS and Convolution......Page 206 16.1.1 The Agent......Page 208 16.1.1.2 Rewards as Feedback for Agent......Page 210 16.1.2 The Environment......Page 211 16.1.3 Interaction Between Agent and Environment......Page 212 16.2.1 Deterministic Environment......Page 213 16.2.2 Stochastic Environment......Page 214 16.2.3 Markov States and MDP......Page 215 16.3 Agent ́s Objective......Page 216 16.4 Agent ́s Behavior......Page 217 16.6 Value Function......Page 218 16.6.2 Action-Value Function......Page 219 16.7.1.1 MDP Known......Page 221 16.7.1.3 MDP Partially Known......Page 222 16.8 Policy Iteration Method for Optimal Policy......Page 223 16.8.2 Policy Iteration......Page 224 17.1 Monte Carlo Learning......Page 225 17.1.1 State Value Estimation......Page 226 17.2 Estimating Action Values with TD Learning......Page 227 17.3.1 -greedy Policy......Page 229 17.4 Q-learning......Page 230 17.5 Scaling Through Function Approximation......Page 231 17.6 Policy-Based Methods......Page 232 17.6.2 Parameterized Policy......Page 233 17.6.3 Training the Model......Page 234 17.6.5 Actor-Critic Methods......Page 235 17.6.6 Reducing Variability in Gradient Methods......Page 236 17.7 Simulation-Based Learning......Page 237 17.8.1 Search Tree......Page 239 17.8.2 Monte Carlo Search Tree......Page 240 17.8.2.1 Trajectory Values......Page 241 17.8.2.2 Backup Procedure......Page 242 17.8.3.2 Expansion Phase......Page 243 17.8.3.5 Tree Policy......Page 244 17.8.4 Pseudo Code for MCTS Algorithm......Page 245 17.9 MCTS Tree Values for Two-Player Games......Page 247 17.10.1.1 Value Function and Policy Network......Page 248 17.10.1.4 Iterative Improvement Loop......Page 249 17.10.2.1 Supervised Training......Page 250 17.10.3.1 Node Value......Page 251 17.10.3.3 Expansion Phase......Page 252 17.10.3.6 Parallel Execution......Page 253 18.1 Pipeline Systems......Page 254 18.2 Data Quality......Page 255 18.2.2 Getting Data......Page 257 18.3.1 Momentum......Page 258 18.3.2 RMSProp......Page 259 18.3.3 ADAM (Adaptive Moment Estimation)......Page 260 18.4.1 TensorFlow......Page 261 18.4.2 MXNet......Page 262 18.4.4 The Microsoft Cognitive Toolkit......Page 263 18.5.1 Traditional Computer Systems......Page 264 18.5.2 GPU......Page 265 18.5.4 TPUs......Page 266 Biblio......Page 268 Index......Page 269