Probability Theory I - Random Variables and Distributions

This book provides a concise yet rigorous introduction to probability theory. Among the possible approaches to the subject, the most modern approach based on measure theory has been chosen: although it requires a higher degree of mathematical abstraction and sophistication, it is essential to provide the foundations for the study of more advanced topics such as stochastic processes, stochastic differential calculus and statistical inference. The text originated from the teaching experience in probability and applied mathematics courses within the mathematics degree program at the University of Bologna; it is suitable for second- or third-year students in mathematics, physics, or other natural sciences, assuming multidimensional differential and integral calculus as a prerequisite. The four chapters cover the following topics: measures and probability spaces; random variables; sequences of random variables and limit theorems; and expectation and conditional distribution. The text includes a collection of solved exercises. Preface A (R)evolution in Mathematics Probability in the Past Probability in the Present Bibliographic Note Contents Frequently Used Symbols and Notations Abbreviations 1 Measures and Probability Spaces 1.1 Measurable Spaces and Probability Spaces 1.1.1 Measurable Spaces 1.1.2 Probability Spaces 1.1.3 Algebras and σ-Algebras 1.1.4 Finite Additivity and σ-Additivity 1.2 Finite Spaces and Counting Problems 1.2.1 Cardinality of Sets 1.2.2 Three Reference Random Experiments: Drawings from an URN 1.2.3 Method of Successive Choices 1.2.4 Arrangements and Combinations 1.2.5 Binomial and Hypergeometric Probability 1.2.6 Examples 1.3 Conditional Probability and Independence of Events 1.3.1 Conditional Probability 1.3.2 Independence 1.3.3 Repeated Independent Trials 1.3.4 Examples 1.4 Distributions 1.4.1 Completion of a Probability Space 1.4.2 Borel σ-Algebra 1.4.3 Distributions 1.4.4 Discrete Distributions 1.4.5 Absolutely Continuous Distributions 1.4.6 Cumulative Distribution Functions (CDF) 1.4.7 Carathéodory's Extension Theorem 1.4.8 From CDFs to Distributions 1.4.9 Cumulative Distribution Functions on R d 1.4.10 Recap 1.5 Appendix 1.5.1 Proof of Proposition 1.3.30 1.5.2 Proof of Proposition 1.4.9 1.5.3 Proof of Carathéodory's Theorem 1.4.29 1.5.4 Proof of Theorem 1.4.33 2 Random Variables 2.1 Random Variables 2.1.1 Random Variables and Distributions 2.1.2 Discrete Random Variables 2.1.3 Absolutely Continuous Random Variables 2.1.4 Other Examples 2.2 Expected Value 2.2.1 Integral of Simple Random Variables 2.2.2 Integral of Non-negative Random Variables 2.2.3 Integral of Random Vectors 2.2.4 Integration with Distributions 2.2.5 Expected Value and Distributions 2.2.6 Jensen's Inequality 2.2.7 Lp Spaces and Inequalities 2.2.8 Covariance and Correlation 2.2.9 Linear Regression 2.2.10 Random Vectors: Marginal and Joint Distributions 2.3 Independence 2.3.1 Deterministic Dependence and Stochastic Independence 2.3.2 Product Measure and Fubini's Theorem 2.3.3 Independence of σ-Algebras 2.3.4 Independence of Random Vectors 2.3.5 Independence and Expected Value 2.4 Conditional Distribution and Expectation Given an Event 2.5 Characteristic Function 2.5.1 The Inversion Theorem 2.5.2 Multivariate Normal Distribution 2.5.3 Series Expansion of CHF and Moments 2.6 Complements 2.6.1 Sum of Random Variables 2.6.2 Examples 3 Sequences of Random Variables 3.1 Convergence for Sequences of Random Variables 3.1.1 Markov's Inequality 3.1.2 Relations Between Different Definitionsof Convergence 3.2 Law of Large Numbers 3.2.1 An Overview of the Monte Carlo Method 3.2.2 Bernstein Polynomials 3.3 Necessary and Sufficient Conditions for Weak Convergence 3.3.1 Convergence of Distribution Functions 3.3.2 Compactness in the Space of Distributions 3.3.3 Convergence of Characteristic Functions and Lévy's Continuity Theorem 3.3.4 Examples of Weak Convergence 3.4 Law of Large Numbers and Central Limit Theorem 4 Conditional Probability 4.1 The Discrete Case 4.1.1 Examples 4.2 Conditional Expectation 4.2.1 Main Properties 4.2.2 Changes of Probability Measure 4.2.3 Conditional Expectation Function 4.2.4 Least Square Monte Carlo 4.3 Conditional Probability 4.3.1 Conditional Distribution Function 4.3.2 From Joint Law to Conditional Marginals: The Absolutely Continuous Case 4.4 Appendix 4.4.1 Proof of Theorem 4.3.4 4.4.2 Proof of Proposition 4.3.20 5 Summary Exercises 5.1 Measures and Probability Spaces 5.2 Random Variables 5.3 Sequences of Random Variables A Dynkin's Theorems B Absolute Continuity B.1 Radon-Nikodym Theorem B.2 Representation of Open Sets in R B.3 Differentiability of Integral Functions B.4 Absolutely Continuous Functions C Uniform Integrability Tables for the Main Distributions Tables for the Main Distributions References Index