Practitioner's Guide to Data Science

This book aims to increase the visibility of Data Science in real-world, which differs from what you learn from a typical textbook. Many aspects of day-to-day Data Science work are almost absent from conventional statistics, Machine Learning, and Data Science curriculum. Yet these activities account for a considerable share of the time and effort for data professionals in the industry. Based on industry experience, this book outlines real-world scenarios and discusses pitfalls that Data Science practitioners should avoid. It also covers the Big Data cloud platform and the art of Data Science, such as soft skills. The authors use R as the primary tool and provide code for both R and Python. This book is for readers who want to explore possible career paths and eventually become data scientists. This book comprehensively introduces various Data Science fields, soft and programming skills in Data Science projects, and potential career paths. Traditional data-related practitioners such as statisticians, business analysts, and data analysts will find this book helpful in expanding their skills for future Data Science careers. Undergraduate and graduate students from analytics-related areas will find this book beneficial to learn real-world Data Science applications. Non-mathematical readers will appreciate the reproducibility of the companion R and Python codes. Key Features: • It covers both technical and soft skills. • It has a chapter dedicated to the Big Data cloud environment. For industry applications, the practice of data science is often in such an environment. • It is hands-on. We provide the data and repeatable R and Python code in notebooks. Readers can repeat the analysis in the book using the data and code provided. We also suggest that readers modify the notebook to perform analyses with their data and problems, if possible. The best way to learn Data Science is to do it! This book aims to increase the visibility of data science in real-world, which differs from what you learn from a typical textbook. Many aspects of day-to-day data science work are almost absent from conventional statistics, machine learning, and data science curriculum. Cover 1 Half Title 2 Series Page 3 Title Page 4 Copyright Page 5 Contents 6 List of Figures 12 Preface 16 About the Authors 22 Acknowledgment 24 1. Introduction 26 1.1. A Brief History of Data Science 26 1.2. Data Science Role and Skill Tracks 30 1.2.1. Engineering 31 1.2.2. Analysis 33 1.2.3. Modeling/Inference 35 1.3. What Kind of Questions Can Data Science Solve? 40 1.3.1. Prerequisites 40 1.3.2. Problem Type 42 1.4. Structure of Data Science Team 45 1.5. Data Science Roles 48 2. Soft Skills for Data Scientists 54 2.1. Comparison between Statistician and Data Scientist 54 2.2. Beyond Data and Analytics 56 2.3. Three Pillars of Knowledge 58 2.4. Data Science Project Cycle 58 2.4.1. Types of Data Science Projects 59 2.4.2. Problem Formulation and Project Planning Stage 61 2.4.3. Project Modeling Stage 63 2.4.4. Model Implementation and Post Production Stage 64 2.4.5. Project Cycle Summary 65 2.5. Common Mistakes in Data Science 66 2.5.1. Problem Formulation Stage 66 2.5.2. Project Planning Stage 67 2.5.3. Project Modeling Stage 68 2.5.4. Model Implementation and Post Production Stage 69 2.5.5. Summary of Common Mistakes 70 3. Introduction to the Data 72 3.1. Customer Data for a Clothing Company 72 3.2. Swine Disease Breakout Data 74 3.3. MNIST Dataset 76 3.4. IMDB Dataset 76 4. Big Data Cloud Platform 80 4.1. Power of Cluster of Computers 81 4.2. Evolution of Cluster Computing 82 4.2.1. Hadoop 82 4.2.2. Spark 83 4.3. Introduction of Cloud Environment 83 4.3.1. Open Account and Create a Cluster 84 4.3.2. R Notebook 85 4.3.3. Markdown cells 86 4.4. Leverage Spark Using R Notebook 87 4.5. Databases and SQL 95 4.5.1. History 95 4.5.2. Database, Table, and View 96 4.5.3. Basic SQL Statement 97 4.5.4. Advanced Topics in Database 101 5. Data Pre-processing 102 5.1. Data Cleaning 104 5.2. Missing Values 107 5.2.1. Impute Missing Values with Median/Mode 108 5.2.2. K-nearest Neighbors 109 5.2.3. Bagging Tree 111 5.3. Centering and Scaling 112 5.4. Resolve Skewness 113 5.5. Resolve Outliers 116 5.6. Collinearity 120 5.7. Sparse Variables 123 5.8. Re-encode Dummy Variables 124 6. Data Wrangling 128 6.1. Summarize Data 129 6.1.1. dplyr Package 129 6.1.2. apply(), lapply() and sapply() in base R 140 6.2. Tidy and Reshape Data 145 7. Model Tuning Strategy 150 7.1. Variance-Bias Trade-Off 151 7.2. Data Splitting and Resampling 159 7.2.1. Data Splitting 160 7.2.2. Resampling 170 8. Measuring Performance 176 8.1. Regression Model Performance 176 8.2. Classification Model Performance 180 8.2.1. Confusion Matrix 182 8.2.2. Kappa Statistic 184 8.2.3. ROC 185 8.2.4. Gain and Lift Charts 188 9. Regression Models 192 9.1. Ordinary Least Square 193 9.1.1. The Magic P-value 198 9.1.2. Diagnostics for Linear Regression 200 9.2. Principal Component Regression and Partial Least Square 205 10. Regularization Methods 212 10.1. Ridge Regression 213 10.2. LASSO 218 10.3. Elastic Net 222 10.4. Penalized Generalized Linear Model 224 10.4.1. Introduction to glmnet Package 224 10.4.2. Penalized Logistic Regression 230 11. Tree-Based Methods 242 11.1. Tree Basics 242 11.2. Splitting Criteria 246 11.2.1. Gini Impurity 247 11.2.2. Information Gain (IG) 248 11.2.3. Information Gain Ratio (IGR) 249 11.2.4. Sum of Squared Error (SSE) 251 11.3. Tree Pruning 253 11.4. Regression and Decision Tree Basic 256 11.4.1. Regression Tree 256 11.4.2. Decision Tree 260 11.5. Bagging Tree 266 11.6. Random Forest 270 11.7. Gradient Boosted Machine 274 11.7.1. Adaptive Boosting 275 11.7.2. Stochastic Gradient Boosting 276 12. Deep Learning 284 12.1. Feedforward Neural Network 288 12.1.1. Logistic Regression as Neural Network 288 12.1.2. Stochastic Gradient Descent 290 12.1.3. Deep Neural Network 292 12.1.4. Activation Function 295 12.1.5. Optimization 299 12.1.6. Deal with Overfitting 307 12.1.7. Image Recognition Using FFNN 308 12.2. Convolutional Neural Network 323 12.2.1. Convolution Layer 324 12.2.2. Padding layer 328 12.2.3. Pooling Layer 329 12.2.4. Convolution Over Volume 334 12.2.5. Image Recognition Using CNN 336 12.3. Recurrent Neural Network 344 12.3.1. RNN Model 346 12.3.2. Long Short Term Memory 349 12.3.3. Word Embedding 352 12.3.4. Sentiment Analysis Using RNN 354 A: Handling Large Local Data 362 A.1. readr 362 A.2. data.table— Enhanced data.frame 370 B: R Code for Data Simulation 382 B.1. Customer Data for Clothing Company 382 B.2. Swine Disease Breakout Data 387 Bibliography 392 Index 400 data,mining;,machine,learning;,python;,reproductible,data;,analytics data mining,machine learning,python,reproductible data,analytics