Practical Data Science with Python 3 : Synthesizing Actionable Insights From Data

Gain insight into essential data science skills in a holistic manner using data engineering and associated scalable computational methods. This book covers the most popular Python 3 frameworks for both local and distributed (in premise and cloud based) processing. Along the way, you will be introduced to many popular open-source frameworks, like, SciPy, scikitlearn, Numba, Apache Spark, etc. The book is structured around examples, so you will grasp core concepts via case studies and Python 3 code. As data science projects gets continuously larger and more complex, software engineering knowledge and experience is crucial to produce evolvable solutions. You'll see how to create maintainable software for data science and how to document data engineering practices. This book is a good starting point for people who want to gain practical skills to perform data science. All the code will be available in the form of IPython notebooks and Python 3 programs, which allow you to reproduce all analyses from the book and customize them for your own purpose. You'll also benefit from advanced topics like Machine Learning, Recommender Systems, and Security in Data Science. Practical Data Science with Python will empower you analyze data, formulate proper questions, and produce actionable insights, three core stages in most data science endeavors. What You'll Learn Play the role of a data scientist when completing increasingly challenging exercises using Python 3 Work work with proven data science techniques/technologies Review scalable software engineering practices to ramp up data analysis abilities in the realm of Big Data Apply theory of probability, statistical inference, and algebra to understand the data science practices Who This Book Is For Anyone who would like to embark into the realm of data science using Python 3. Contents......Page 3 Introduction......Page 9 1 Intro to Data Science......Page 10 Main Phases of a Data Science Project......Page 11 Brown Cow Model Case Study......Page 13 Big Data......Page 18 Big Data Example: MOOC Platforms......Page 19 How to Learn Data Science......Page 21 Domain Knowledge Attainment—Example......Page 23 Programming Skills Attainment—Example......Page 25 Overview of the Anaconda Ecosystem......Page 27 Managing Packages and Environments......Page 29 Sharing and Reproducing Environments......Page 32 Summary......Page 34 References......Page 35 2 Data Engineering......Page 37 E-Commerce Customer Segmentation: Case Study......Page 38 Creating a Project in Spyder......Page 41 Downloading the Dataset......Page 42 Exploring the Dataset......Page 44 Finding Associations Between Features......Page 46 Incorporating Custom Features......Page 53 Automating the Steps......Page 62 Inspecting Results......Page 64 Persisting Results......Page 68 Parquet Engines......Page 69 Restructuring Code to Cope with Large CSV Files......Page 70 Public Data Sources......Page 72 Summary......Page 77 References......Page 78 3 Software Engineering......Page 80 Characteristics of a Large-Scale Software System......Page 82 Software Engineering Knowledge Areas......Page 86 Rules, Principles, Conventions, and Standards......Page 88 Context Awareness and Communicative Abilities......Page 92 Reducing Cyclomatic Complexity......Page 96 Cone of Uncertainty and Having Time to Ask......Page 98 Fixing a Bug and Knowing How to Ask......Page 100 A Better Fix......Page 104 Scenario 1: The Developer Doesn’t Speak the Language of Business......Page 105 Scenario 2: The Developer Does Speak the Language of Business......Page 106 A More Advanced Fix......Page 107 Scenario 2: The Developer Does Speak the Language of Business......Page 108 Handling Legacy Code......Page 109 Understanding Bug-Free Code......Page 110 Understanding Faulty Code......Page 112 The Importance of APIs......Page 114 Fervent Flexibility Hurts Your API......Page 116 The Socio-* Pieces of Software Production......Page 118 Funny Elevator Case Study......Page 119 First Optimization Attempt......Page 121 Second Optimization Attempt......Page 122 Teammate- and Business-Friendly Variant......Page 124 Summary......Page 125 References......Page 126 4 Documenting......Page 127 JupyterLab in Action......Page 131 Experimenting with Code Execution......Page 132 Managing the Kernel......Page 140 Connecting to a Notebook’s Kernel......Page 141 Problem Specification......Page 142 Model Definition......Page 144 Path Finder’s Implementation......Page 146 Interaction with the Simulator......Page 151 Test Automation......Page 152 Refactoring the Simulator’s Notebook......Page 154 Document Structure......Page 156 Motivation......Page 158 Conclusion......Page 159 Summary......Page 162 References......Page 163 Augmented Descending Ball Project......Page 165 Version 1.1......Page 166 Path Finding Engine......Page 169 Retrospective of Version 1.1......Page 176 Version 1.2......Page 178 Enhancing the Input Subsystem......Page 179 Enhancing the Output Subsystem......Page 184 Retrospective of Version 1.2......Page 187 Version 1.3......Page 190 Establishing the Baseline......Page 192 Performance Optimization......Page 198 Abstractions vs. Latent Features......Page 206 Compressing the Ratings Matrix......Page 207 Summary......Page 211 References......Page 213 6 Data Visualization......Page 214 Visualizing Temperature Data Case Study......Page 215 Showing Stations on a Map......Page 216 Plotting Temperatures......Page 218 Closest Pair Case Study......Page 223 Version 1.0......Page 228 Version 2.0......Page 234 Analysis of the Running Time......Page 237 Version 3.0......Page 238 Interactive Information Radiators......Page 246 The Power of Domain-Specific Languages......Page 249 Summary......Page 257 References......Page 258 7 Machine Learning......Page 259 Exposition of Core Concepts and Techniques......Page 262 Overfitting......Page 275 Underfitting and Feature Interaction......Page 280 Collinearity......Page 282 Residuals Plot......Page 285 Regularization......Page 289 Predicting Financial Movements Case Study......Page 290 Data Preprocessing......Page 292 Discovering Trends in Time Series......Page 295 Transforming Features......Page 298 Streaming Amounts......Page 300 Feature Engineering......Page 307 Implementing Streaming Linear Regression......Page 312 Summary......Page 318 References......Page 320 8 Recommender Systems......Page 321 Introduction to Recommender Systems......Page 322 Simple Movie Recommender Case Study......Page 326 Introduction to LensKit for Python......Page 333 Summary......Page 342 References......Page 343 9 Data Security......Page 344 Checking for Compromise......Page 345 Introduction to the GDPR......Page 352 Membership Inference Attack......Page 362 Shadow Training......Page 364 Poisoning Attack......Page 366 Summary......Page 368 References......Page 369 10 Graph Analysis......Page 371 Usage Matrix As a Graph Problem......Page 372 Opposite Quality Attributes......Page 378 Partitioning the Model into a Bipartite Graph......Page 379 Scalable Graph Loading......Page 382 Social Networks......Page 387 Summary......Page 397 References......Page 398 11 Complexity & Heuristics......Page 399 From Simple to Complicated......Page 403 Counting the Occurrences of a Digit......Page 404 Estimating the Edge Betweenness Centrality......Page 411 The Count of Divisible Numbers......Page 415 From Disorder to Complex......Page 417 Exploring the KDD Cup 1999 Data......Page 418 Cynefin and Data Science......Page 422 References......Page 427 12 Deep Learning......Page 428 Intelligent Machines......Page 429 Intelligence As Mastery of Symbols......Page 432 Manual Feature Engineering......Page 433 Modeling the Aggregated State......Page 434 Tying All Pieces Together......Page 436 Machine-Based Feature Engineering......Page 437 Implementation from Scratch......Page 440 Implementation with PyTorch......Page 444 References......Page 450 Index......Page 452