A Basic Course in Probability Theory (Universitext)

This text develops the necessary background in probability theory underlying diverse treatments of stochastic processes and their wide-ranging applications. In this second edition, the text has been reorganized for didactic purposes, new exercises have been added and basic theory has been expanded. General Markov dependent sequences and their convergence to equilibrium is the subject of an entirely new chapter. The introduction of conditional expectation and conditional probability very early in the text maintains the pedagogic innovation of the first edition; conditional expectation is illustrated in detail in the context of an expanded treatment of martingales, the Markov property, and the strong Markov property. Weak convergence of probabilities on metric spaces and Brownian motion are two topics to highlight. A selection of large deviation and/or concentration inequalities ranging from those of Chebyshev, Cramer–Chernoff, Bahadur–Rao, to Hoeffding have been added, with illustrative comparisons of their use in practice. This also includes a treatment of the Berry–Esseen error estimate in the central limit theorem. The authors assume mathematical maturity at a graduate level; otherwise the book is suitable for students with varying levels of background in analysis and measure theory. For the reader who needs refreshers, theorems from analysis and measure theory used in the main text are provided in comprehensive appendices, along with their proofs, for ease of reference. Rabi Bhattacharya is Professor of Mathematics at the University of Arizona. Edward Waymire is Professor of Mathematics at Oregon State University. Both authors have co-authored numerous books, including a series of four upcoming graduate textbooks in stochastic processes with applications. Preface to Second Edition 6 Preface to First Edition 8 Contents 11 I Random Maps, Distribution, and Mathematical Expectation 13 II Independence, Conditional Expectation 36 III Martingales and Stopping Times 64 IV Classical Central Limit Theorems 86 V Classical Zero--One Laws, Laws of Large Numbers and Large Deviations 97 VI Fourier Series, Fourier Transform, and Characteristic Functions 113 VII Weak Convergence of Probability Measures on Metric Spaces 145 VIII Random Series of Independent Summands 168 IX Kolmogorov's Extension Theorem and Brownian Motion 176 IX.1 A Wavelet Construction of Brownian Motion: The Lévy--Ciesielski Construction 182 X Brownian Motion: The LIL and Some Fine-Scale Properties 188 XI Strong Markov Property, Skorokhod Embedding, and Donsker's Invariance Principle 196 XII A Historical Note on Brownian Motion 215 XIII Some Elements of the Theory of Markov Processes and Their Convergence to Equilibrium 219 Appendix A Measure and Integration 232 A.1 Measures and the Carathéodory Extension 232 A.2 Integration and Basic Convergence Theorems 237 A.3 Product Measures 242 A.4 Riesz Representation on C(S) 244 Appendix B Topology and Function Spaces 248 Appendix C Hilbert Spaces and Applications in Measure Theory 253 C.1 Hilbert Spaces 253 C.2 Lebesgue Decomposition and the Radon--Nikodym Theorem 256 References 260 Symbol Index 262 Index 264 Front Matter....Pages i-xii Random Maps, Distribution, and Mathematical Expectation....Pages 1-23 Independence, Conditional Expectation....Pages 25-52 Martingales and Stopping Times....Pages 53-74 Classical Central Limit Theorems....Pages 75-85 Classical Zero–One Laws, Laws of Large Numbers and Large Deviations....Pages 87-102 Fourier Series, Fourier Transform, and Characteristic Functions....Pages 103-134 Weak Convergence of Probability Measures on Metric Spaces....Pages 135-157 Random Series of Independent Summands....Pages 159-166 Kolmogorov’s Extension Theorem and Brownian Motion....Pages 167-178 Brownian Motion: The LIL and Some Fine-Scale Properties....Pages 179-186 Strong Markov Property, Skorokhod Embedding, and Donsker’s Invariance Principle....Pages 187-205 A Historical Note on Brownian Motion....Pages 207-210 Some Elements of the Theory of Markov Processes and Their Convergence to Equilibrium....Pages 211-223 Back Matter....Pages 225-265