Stochastic Processes With Applications (Classics in Applied Mathematics)

This book develops systematically and rigorously, yet in an expository and lively manner, the evolution of general random processes and their large time properties such as transience, recurrence, and convergence to steady states. The emphasis is on the most important classes of these processes from the viewpoint of theory as well as applications, namely, Markov processes. The book features very broad coverage of the most applicable aspects of stochastic processes, including sufficient material for self-contained courses on random walk in one and multiple dimensions; Markov chains in discrete and continuous times, including birth-death processes; Brownian motion and diffusions; stochastic optimization; and stochastic differential equations. Most results are presented with complete proofs, while some very technical matters are relegated to a Theoretical Complements section at the end of each chapter in order not to impede the flow of the material. Chapter Applications, as well as numerous extensively worked examples, illustrate important applications of the subject to various fields of science, engineering, economics, and applied mathematics. The essentials of measure theoretic probability are included in an appendix to complete some of the more technical aspects of the text. Audience: This book can be used for a number of different courses for graduate students of mathematics, statistics, economics, engineering, and other fields who have some background in probability and analysis. It is also intended as a reference for researchers and professionals in many areas of science and technology whose work involves the application of probability. Contents: Preface to the Classics Edition; Preface; Sample Course Outline; Chapter I: Random Walk and Brownian Motion; Chapter II: Discrete-Parameter Markov Chains; Chapter III: Birth Death Markov Chains; Chapter IV: Continuous-Parameter Markov Chains; Chapter V: Brownian Motion and Diffusions; Chapter VI: Dynamic Programming and Stochastic Optimization; Chapter VII: An Introduction to Stochastic Differential Equations; Chapter 0: A Probability and Measure Theory Overview; Author Index; Subject Index; Errata. Contents......Page p0008.djvu Preface to the Classics Edition......Page p0012.djvu Preface......Page p0013.djvu Sample Course Outlines......Page p0015.djvu 1 WHAT IS A STOCHASTIC PROCESS?......Page p0016.djvu 2 THE SIMPLE RANDOM WALK......Page p0018.djvu 3 TRANSIENCE AND RECURRENCE PROPERTIES OF THESIMPLE RANDOM WALK......Page p0020.djvu 4 FIRST PASSAGE TIMES FOR THE SIMPLE RANDOM WALK......Page p0023.djvu 5 MULTIDIMENSIONAL RANDOM WALKS......Page p0026.djvu 6 CANONICAL CONSTRUCTION OF STOCHASTIC PROCESSES......Page p0030.djvu 7 BROWNIAN MOTION......Page p0032.djvu 8 THE FUNCTIONAL CENTRAL LIMIT THEOREM (FCLT)......Page p0035.djvu 9 RECURRENCE PROBABILITIES FOR BROWNIAN MOTION......Page p0039.djvu 10 FIRST PASSAGE TIME DISTRIBUTIONS FOR BROWNIANMOTION......Page p0042.djvu 11 THE ARCSINE LAW......Page p0047.djvu 12 THE BROWNIAN BRIDGE......Page p0050.djvu 13 STOPPING TIMES AND MARTINGALES......Page p0054.djvu 14 CHAPTER APPLICATION: FLUCTUATIONS OF RANDOMWALKS WITH SLOW TRENDS AND THE HURST PHENOMENON......Page p0068.djvu EXERCISES......Page p0077.djvu THEORETICAL COMPLEMENTS......Page p0105.djvu 1 MARKOV DEPENDENCE......Page p0124.djvu 2 TRANSITION PROBABILITIES AND THE PROBABILITY SPACE......Page p0125.djvu 3 SOME EXAMPLES......Page p0128.djvu 4 STOPPING TIMES AND THE STRONG MARKOV PROPERTY......Page p0132.djvu 5 A CLASSIFICATION OF STATES OF A MARKOV CHAIN......Page p0135.djvu 6 CONVERGENCE TO STEADY STATE FOR IRREDUCIBLE ANDAPERIODIC MARKOV PROCESSES ON FINITE SPACES......Page p0141.djvu 7 STEADY-STATE DISTRIBUTIONS FOR GENERALFINITE-STATE MARKOV PROCESSES......Page p0147.djvu 8 MARKOV CHAINS: TRANSIENCE AND RECURRENCEPROPERTIES......Page p0150.djvu 9 THE LAW OF LARGE NUMBERS AND INVARIANTDISTRIBUTIONS FOR MARKOV CHAINS......Page p0153.djvu 10 THE CENTRAL LIMIT THEOREM FOR MARKOV CHAINS......Page p0163.djvu 11 ABSORPTION PROBABILITIES......Page p0166.djvu 12 ONE-DIMENSIONAL NEAREST-NEIGHBOR GIBBS STATE......Page p0177.djvu 13 A MARKOVIAN APPROACH TO LINEAR TIME SERIESMODELS......Page p0181.djvu 14 MARKOV PROCESSES GENERATED BY ITERATIONS OFI.I.D. MAPS......Page p0189.djvu 15 CHAPTER APPLICATION: DATA COMPRESSION ANDENTROPY......Page p0199.djvu EXERCISES......Page p0204.djvu Theoretical Complement to Section 11.8......Page p0231.djvu 1 INTRODUCTION TO BIRTH—DEATH CHAINS......Page p0248.djvu 2 TRANSIENCE AND RECURRENCE PROPERTIES......Page p0249.djvu 3 INVARIANT DISTRIBUTIONS FOR BIRTH—DEATH CHAINS......Page p0253.djvu 4 CALCULATION OF TRANSITION PROBABILITIES BYSPECTRAL METHODS......Page p0256.djvu 5 CHAPTER APPLICATION: THE EHRENFEST MODEL OFHEAT EXCHANGE......Page p0261.djvu EXERCISES......Page p0267.djvu THEORETICAL COMPLEMENTS......Page p0271.djvu I INTRODUCTION TO CONTINUOUS-TIME MARKOV CHAINS......Page p0276.djvu 2 KOLMOGOROV'S BACKWARD AND FORWARD EQUATIONS......Page p0278.djvu 3 SOLUTIONS TO KOLMOGOROV'S EQUATIONS INEXPONENTIAL FORM......Page p0282.djvu 4 SOLUTIONS TO KOLMOGOROV'S EQUATIONS BYSUCCESSIVE APPROXIMATIONS......Page p0286.djvu 5 SAMPLE PATH ANALYSIS AND THE STRONG MARKOVPROPERTY......Page p0290.djvu 6 THE MINIMAL PROCESS AND EXPLOSION......Page p0303.djvu 7 SOME EXAMPLES......Page p0307.djvu 8 ASYMPTOTIC BEHAVIOR OF CONTINUOUS-TIME MARKOVCHAINS......Page p0318.djvu 9 CALCULATION OF TRANSITION PROBABILITIES BYSPECTRAL METHODS......Page p0329.djvu 10 ABSORPTION PROBABILITIES......Page p0333.djvu 11 CHAPTER APPLICATION: AN INTERACTING SYSTEMTHE SIMPLE SYMMETRIC VOTER MODEL......Page p0339.djvu EXERCISES......Page p0348.djvu THEORETICAL COMPLEMENTS......Page p0364.djvu I INTRODUCTION AND DEFINITION......Page p0382.djvu 2 KOLMOGOROV'S BACKWARD AND FORWARD EQUATIONS,MARTINGALES......Page p0386.djvu 3 TRANSFORMATION OF THE GENERATOR UNDERRELABELING OF THE STATE SPACE......Page p0396.djvu 4 DIFFUSIONS AS LIMITS OF BIRTH—DEATH CHAINS......Page p0401.djvu 5 TRANSITION PROBABILITIES FROM THE KOLMOGOROVEQUATIONS: EXAMPLES......Page p0404.djvu 6 DIFFUSIONS WITH REFLECTING BOUNDARIES......Page p0408.djvu 7 DIFFUSIONS WITH ABSORBING BOUNDARIES......Page p0417.djvu 8 CALCULATION OF TRANSITION PROBABILITIES BYSPECTRAL METHODS......Page p0423.djvu 9 TRANSIENCE AND RECURRENCE OF DIFFUSIONS......Page p0429.djvu 10 NULL AND POSITIVE RECURRENCE OF DIFFUSIONS......Page p0435.djvu 11 STOPPING TIMES AND THE STRONG MARKOV PROPERTY......Page p0438.djvu 12 INVARIANT DISTRIBUTIONS AND THE STRONG LAW OFLARGE NUMBERS......Page p0447.djvu 13 THE CENTRAL LIMIT THEOREM FOR DIFFUSIONS......Page p0453.djvu 14 INTRODUCTION TO MULTIDIMENSIONAL BROWNIANMOTION AND DIFFUSIONS......Page p0456.djvu 15 MULTIDIMENSIONAL DIFFUSIONS UNDER ABSORBINGBOUNDARY CONDITIONS AND CRITERIA FOR TRANSIENCEAND RECURRENCE......Page p0463.djvu 16 REFLECTING BOUNDARY CONDITIONS FORMULTIDIMENSIONAL DIFFUSIONS......Page p0475.djvu 17 CHAPTER APPLICATION: G. I. TAYLOR'S THEORY OFSOLUTE TRANSPORT IN A CAPILLARY......Page p0483.djvu EXERCISES......Page p0490.djvu THEORETICAL COMPLEMENTS......Page p0512.djvu 1 FINITE-HORIZON OPTIMIZATION......Page p0534.djvu 2 THE INFINITE-HORIZON PROBLEM......Page p0540.djvu 3 OPTIMAL CONTROL OF DIFFUSIONS......Page p0548.djvu 4 OPTIMAL STOPPING AND THE SECRETARY PROBLEM......Page p0557.djvu 5 CHAPTER APPLICATION: OPTIMALITY OF (S, s) POLICIES ININVENTORY PROBLEMS......Page p0564.djvu EXERCISES......Page p0572.djvu THEORETICAL COMPLEMENTS......Page p0574.djvu 1 THE STOCHASTIC INTEGRAL......Page p0578.djvu 2 CONSTRUCTION OF DIFFUSIONS AS SOLUTIONS OFSTOCHASTIC DIFFERENTIAL EQUATIONS......Page p0586.djvu 3 ITÖ'S LEMMA......Page p0597.djvu 4 CHAPTER APPLICATION: ASYMPTOTICS OF SINGULARDIFFUSIONS......Page p0606.djvu EXERCISES......Page p0613.djvu THEORETICAL COMPLEMENTS......Page p0622.djvu 1 PROBABILITY SPACES......Page p0640.djvu 2 RANDOM VARIABLES AND INTEGRATION......Page p0642.djvu 3 LIMITS AND INTEGRATION......Page p0646.djvu 4 PRODUCT MEASURES AND INDEPENDENCE,RADON—NIKODYM THEOREM AND CONDITIONALPROBABILITY......Page p0651.djvu 5 CONVERGENCE IN DISTRIBUTION IN FINITE DIMENSIONS......Page p0658.djvu 6 CLASSICAL LAWS OF LARGE NUMBERS......Page p0661.djvu 7 CLASSICAL CENTRAL LIMIT THEOREMS......Page p0664.djvu 8 FOURIER SERIES AND THE FOURIER TRANSFORM......Page p0668.djvu Author Index......Page p0680.djvu Subject Index......Page p0682.djvu Errata......Page p0688.djvu