Think Bayes : Bayesian statistics in Python

If you know how to program, you're ready to tackle Bayesian statistics. With this book, you'll learn how to solve statistical problems with Python code instead of mathematical formulas, using discrete probability distributions rather than continuous mathematics. Once you get the math out of the way, the Bayesian fundamentals will become clearer and you'll begin to apply these techniques to real-world problems. Bayesian statistical methods are becoming more common and more important, but there aren't many resources available to help beginners. Based on undergraduate classes taught by author Allen B. Downey, this book's computational approach helps you get a solid start. • Use your programming skills to learn and understand Bayesian statistics • Work with problems involving estimation, prediction, decision analysis, evidence, and Bayesian hypothesis testing • Get started with simple examples, using coins, dice, and a bowl of cookies • Learn computational methods for solving real-world problems Copyright 4 Table of Contents 5 Preface 11 Who Is This Book For? 11 Modeling 12 Working with the Code 12 Installing Jupyter 13 Conventions Used in This Book 14 O’Reilly Online Learning 14 How to Contact Us 15 Contributor List 15 Chapter 1. Probability 17 Linda the Banker 17 Probability 18 Fraction of Bankers 19 The Probability Function 20 Political Views and Parties 20 Conjunction 21 Conditional Probability 22 Conditional Probability Is Not Commutative 23 Condition and Conjunction 24 Laws of Probability 24 Theorem 1 25 Theorem 2 26 Theorem 3 26 The Law of Total Probability 27 Summary 29 Exercises 30 Chapter 2. Bayes’s Theorem 33 The Cookie Problem 33 Diachronic Bayes 35 Bayes Tables 36 The Dice Problem 38 The Monty Hall Problem 39 Summary 41 Exercises 42 Chapter 3. Distributions 45 Distributions 45 Probability Mass Functions 45 The Cookie Problem Revisited 48 101 Bowls 50 The Dice Problem 54 Updating Dice 55 Summary 56 Exercises 57 Chapter 4. Estimating Proportions 59 The Euro Problem 59 The Binomial Distribution 60 Bayesian Estimation 63 Triangle Prior 65 The Binomial Likelihood Function 67 Bayesian Statistics 68 Summary 69 Exercises 70 Chapter 5. Estimating Counts 73 The Train Problem 73 Sensitivity to the Prior 76 Power Law Prior 77 Credible Intervals 79 The German Tank Problem 80 Informative Priors 81 Summary 82 Exercises 82 Chapter 6. Odds and Addends 85 Odds 85 Bayes’s Rule 86 Oliver’s Blood 87 Addends 89 Gluten Sensitivity 92 The Forward Problem 93 The Inverse Problem 94 Summary 96 More Exercises 97 Chapter 7. Minimum, Maximum, and Mixture 99 Cumulative Distribution Functions 99 Best Three of Four 102 Maximum 104 Minimum 105 Mixture 106 General Mixtures 109 Summary 112 Exercises 113 Chapter 8. Poisson Processes 115 The World Cup Problem 115 The Poisson Distribution 116 The Gamma Distribution 117 The Update 119 Probability of Superiority 121 Predicting the Rematch 122 The Exponential Distribution 124 Summary 126 Exercises 126 Chapter 9. Decision Analysis 129 The Price Is Right Problem 129 The Prior 130 Kernel Density Estimation 131 Distribution of Error 132 Update 134 Probability of Winning 136 Decision Analysis 138 Maximizing Expected Gain 140 Summary 142 Discussion 142 More Exercises 143 Chapter 10. Testing 145 Estimation 145 Evidence 147 Uniformly Distributed Bias 148 Bayesian Hypothesis Testing 150 Bayesian Bandits 150 Prior Beliefs 151 The Update 152 Multiple Bandits 153 Explore and Exploit 154 The Strategy 156 Summary 158 More Exercises 158 Chapter 11. Comparison 161 Outer Operations 161 How Tall Is A? 163 Joint Distribution 164 Visualizing the Joint Distribution 165 Likelihood 167 The Update 168 Marginal Distributions 169 Conditional Posteriors 172 Dependence and Independence 173 Summary 174 Exercises 174 Chapter 12. Classification 177 Penguin Data 177 Normal Models 179 The Update 180 Naive Bayesian Classification 182 Joint Distributions 184 Multivariate Normal Distribution 186 A Less Naive Classifier 188 Summary 189 Exercises 189 Chapter 13. Inference 191 Improving Reading Ability 191 Estimating Parameters 193 Likelihood 194 Posterior Marginal Distributions 196 Distribution of Differences 197 Using Summary Statistics 200 Update with Summary Statistics 202 Comparing Marginals 203 Summary 204 Exercises 205 Chapter 14. Survival Analysis 207 The Weibull Distribution 207 Incomplete Data 210 Using Incomplete Data 212 Light Bulbs 215 Posterior Means 217 Posterior Predictive Distribution 218 Summary 220 Exercises 220 Chapter 15. Mark and Recapture 223 The Grizzly Bear Problem 223 The Update 225 Two-Parameter Model 227 The Prior 228 The Update 229 The Lincoln Index Problem 231 Three-Parameter Model 233 Summary 236 Exercises 237 Chapter 16. Logistic Regression 239 Log Odds 239 The Space Shuttle Problem 242 Prior Distribution 245 Likelihood 246 The Update 247 Marginal Distributions 248 Transforming Distributions 249 Predictive Distributions 251 Empirical Bayes 253 Summary 254 More Exercises 254 Chapter 17. Regression 257 More Snow? 257 Regression Model 259 Least Squares Regression 260 Priors 261 Likelihood 262 The Update 263 Marathon World Record 266 The Priors 268 Prediction 270 Summary 271 Exercises 271 Chapter 18. Conjugate Priors 273 The World Cup Problem Revisited 273 The Conjugate Prior 274 What the Actual? 276 Binomial Likelihood 277 Lions and Tigers and Bears 279 The Dirichlet Distribution 280 Summary 282 Exercises 283 Chapter 19. MCMC 285 The World Cup Problem 285 Grid Approximation 286 Prior Predictive Distribution 286 Introducing PyMC3 287 Sampling the Prior 288 When Do We Get to Inference? 290 Posterior Predictive Distribution 291 Happiness 292 Simple Regression 293 Multiple Regression 296 Summary 298 Exercises 299 Chapter 20. Approximate Bayesian Computation 303 The Kidney Tumor Problem 303 A Simple Growth Model 304 A More General Model 305 Simulation 307 Approximate Bayesian Computation 310 Counting Cells 311 Cell Counting with ABC 314 When Do We Get to the Approximate Part? 315 Summary 318 Exercises 319 Index 321 About the Author 337 Colophon 337 If You Know How To Program With Python, Youâ??re Ready To Tackle Bayesian Statistics. With This Book, You'll Learn How To Solve Statistical Problems With Python Code Instead Of Mathematical Formulas, Using Discrete Probability Distributions Instead Of Continuous Mathematics. Once You Get The Math Out Of The Way, The Bayesian Fundamentals Will Become Clearer, And Youâ??ll Begin To Apply These Techniques To Real-world Problems. Bayesian Statistical Methods Are Becoming More Common And More Important, But Not Many Resources Are Available To Help Beginners. Based On Undergraduate Classes Taught By Author Allen Downey, This Bookâ??s Computational Approach Helps You Get A Solid Start. Use Your Existing Programming Skills To Learn And Understand Bayesian Statistics Work With Problems Involving Estimation, Prediction, Decision Analysis, Evidence, And Hypothesis Testing Get Started With Simple Examples, Using Coins, Dice, And A Bowl Of Cookies Learn Computational Methods For Solving Real-world Problems "If you know how to program, you're ready to tackle Bayesian statistics. With this book, you'll learn how to solve statistical problems with Python code instead of mathematical formulas, using discrete probability distributions rather than continuous mathematics. Once you get the math out of the way, the Bayesian fundamentals will become clearer and you'll begin to apply these techniques to real-world problems. Bayesian statistical methods are becoming more common and more important, but there aren't many resources available to help beginners. Based on undergraduate classes taught by author Allen B. Downey, this book's computational approach helps you get a solid start." -- Page 4 de la couverture Printing History September, 2013: First Edition 2013-09-10: First release 2014-02-10: Second release 2014-08-22: Third release 2016-06-03: Fourth release