Introduction to Stochastic Processes Using R

This textbook presents some basic stochastic processes, mainly Markov processes. It begins with a brief introduction to the framework of stochastic processes followed by the thorough discussion on Markov chains, which is the simplest and the most important class of stochastic processes. The book then elaborates the theory of Markov chains in detail including classification of states, the first passage distribution, the concept of periodicity and the limiting behaviour of a Markov chain in terms of associated stationary and long run distributions. The book first illustrates the theory for some typical Markov chains, such as random walk, gambler's ruin problem, Ehrenfest model and Bienayme-Galton-Watson branching process; and then extends the discussion when time parameter is continuous. It presents some important examples of a continuous time Markov chain, which include Poisson process, birth process, death process, birth and death processes and their variations. These processesplay a fundamental role in the theory and applications in queuing and inventory models, population growth, epidemiology and engineering systems. The book studies in detail the Poisson process, which is the most frequently applied stochastic process in a variety of fields, with its extension to a renewal process. The book also presents important basic concepts on Brownian motion process, a stochastic process of historic importance. It covers its few extensions and variations, such as Brownian bridge, geometric Brownian motion process, which have applications in finance, stock markets, inventory etc. The book is designed primarily to serve as a textbook for a one semester introductory course in stochastic processes, in a post-graduate program, such as Statistics, Mathematics, Data Science and Finance. It can also be used for relevant courses in other disciplines. Additionally, it provides sufficient background material for studying inference in stochastic processes. The book thus fulfils the need of a concise but clear and student-friendly introduction to various types of stochastic processes. Preface Contents About the Authors List of Figures List of Tables 1 Basics of Stochastic Processes 1.1 Introduction 1.2 Kolmogorov Compatibility Conditions 1.3 Stochastic Processes with Stationary and Independent Increments 1.4 Stationary Processes 1.5 Introduction to R Software and Language References 2 Markov Chains 2.1 Introduction 2.2 Higher Step Transition Probabilities 2.3 Realization of a Markov Chain 2.4 Classification of States 2.5 Persistent and Transient States 2.6 First Passage Distribution 2.7 Periodicity 2.8 R Codes 2.9 Conceptual Exercises 2.10 Computational Exercises 2.11 Multiple Choice Questions References 3 Long Run Behavior of Markov Chains 3.1 Introduction 3.2 Long Run Distribution 3.3 Stationary Distribution 3.4 Computation of Stationary Distributions 3.5 Autocovariance Function 3.6 Bonus-Malus System 3.7 R Codes 3.8 Conceptual Exercises 3.9 Computational Exercises 3.10 Multiple Choice Questions References 4 Random Walks 4.1 Introduction 4.2 Random Walk with Countably Infinite State Space 4.3 Random Walk with Finite State Space 4.4 Gambler's Ruin Problem 4.5 Ehrenfest Chain and Birth-Death Chain 4.6 R Codes 4.7 Conceptual Exercises 4.8 Computational Exercises 4.9 Multiple Choice Questions References 5 Bienayme Galton Watson Branching Process 5.1 Introduction 5.2 Markov Property 5.3 Branching Property 5.4 Extinction Probability 5.5 Realization of a Process and Computation of Extinction Probability 5.6 R Codes 5.7 Conceptual Exercises 5.8 Computational Exercises 5.9 Multiple Choice Questions References 6 Continuous Time Markov Chains 6.1 Introduction 6.2 Definition and Properties 6.3 Transition Probability Function 6.4 Infinitesimal Generator 6.5 Computation of Transition Probability Function 6.6 Long Run Behavior 6.7 R Codes 6.8 Conceptual Exercises 6.9 Computational Exercises 6.10 Multiple Choice Questions References 7 Poisson Process 7.1 Introduction 7.2 Poisson Process as a Process with Stationary and Independent Increments 7.3 Poisson Process as a Point Process 7.4 Non-homogeneous Poisson Process 7.5 Superposition and Decomposition 7.6 Compound Poisson Process 7.7 R Codes 7.8 Conceptual Exercises 7.9 Computational Exercises 7.10 Multiple Choice Questions References 8 Birth and Death Process 8.1 Introduction 8.2 Birth Process 8.3 Death Process 8.4 Birth-Death Process 8.5 Linear Birth-Death Process 8.6 Long Run Behavior of a Birth-Death Process 8.7 R Codes 8.8 Conceptual Exercises 8.9 Computational Exercises 8.10 Multiple Choice Questions References 9 Brownian Motion Process 9.1 Introduction 9.2 Definition and Properties 9.3 Realization and Properties of Sample Path 9.4 Brownian Bridge 9.5 Geometric Brownian Motion Process 9.6 Variations of a Brownian Motion Process 9.7 R Codes 9.8 Conceptual Exercises 9.9 Computational Exercises 9.10 Multiple Choice Questions References 10 Renewal Process 10.1 Introduction 10.2 Renewal Function 10.3 Long Run Renewal Rate 10.4 Limit Theorems 10.5 Generalizations and Variations of Renewal Processes 10.6 R Codes 10.7 Conceptual Exercises 10.8 Computational Exercises 10.9 Multiple Choice Questions References Appendix A Solutions to Conceptual Exercises A.1 Chapter 2 A.2 Chapter 3 A.3 Chapter 4 A.4 Chapter 5 A.5 Chapter 6 A.6 Chapter 7 A.7 Chapter 8 A.8 Chapter 9 A.9 Chapter 10 Index