Introduction to Stochastic Processes (Chapman & Hall/CRC Probability Series)

Emphasizing fundamental mathematical ideas rather than proofs, Introduction to Stochastic Processes, Second Edition provides quick access to important foundations of probability theory applicable to problems in many fields. Assuming that you have a reasonable level of computer literacy, the ability to write simple programs, and the access to software for linear algebra computations, the author approaches the problems and theorems with a focus on stochastic processes evolving with time, rather than a particular emphasis on measure theory.For those lacking in exposure to linear differential and difference equations, the author begins with a brief introduction to these concepts. He proceeds to discuss Markov chains, optimal stopping, martingales, and Brownian motion. The book concludes with a chapter on stochastic integration. The author supplies many basic, general examples and provides exercises at the end of each chapter.New to the Second Edition:- Expanded chapter on stochastic integration that introduces modern mathematical finance - Introduction of Girsanov transformation and the Feynman-Kac formula - Expanded discussion of Itô's formula and the Black-Scholes formula for pricing options - New topics such as Doob's maximal inequality and a discussion on self similarity in the chapter on Brownian motionApplicable to the fields of mathematics, statistics, and engineering as well as computer science, economics, business, biological science, psychology, and engineering, this concise introduction is an excellent resource both for students and professionals. Cover ... 1 Contents ... 7 0 Preliminaries ... 7 1 Finite Markov Chains ... 7 2 Countable Markov Chains ... 7 3 Continuous-Time Markov Chains ... 7 4 Optimal Stopping ... 8 5 Martingales ... 8 6 Renewal Processes ... 8 7 Reversible Markov Chains ... 8 8 Brownian Motion ... 8 9 Stochastic Integration ... 8 Preface to Second Edition ... 11 Preface to First Edition ... 13 Chapter 0 Preliminaries ... 17 0.1 Introduction ... 17 0.2 Linear Differential Equations ... 17 0.3 Linear Difference Equations ... 19 0.4 Exercises ... 22 Chapter 1 Finite Markov Chains ... 25 1.1 Definitions and Examples ... 25 1.2 Large-Time Behavior and Invariant Probability ... 30 1.3 Classification of States ... 33 1.3.1 Reducibility ... 35 1.3.2 Periodicity ... 37 1.3.3 Irreducible, aperiodic chains ... 38 1.3.4 Reducible or periodic chains ... 38 1.4 Return Times ... 40 1.5 Transient States ... 42 1.6 Examples ... 47 1.7 Exercises ... 51 Chapter 2 Countable Markov Chains ... 59 2.1 Introduction ... 59 2.2 Recurrence and Transience ... 61 2.3 Positive Recurrence and Null Recurrence ... 66 2.4 Branching Process ... 69 2.5 Exercises ... 73 Chapter 3 Continuous-Time Markov Chains ... 81 3.1 Poisson Process ... 81 3.2 Finite State Space ... 84 3.3 Birth-and-Death Processes ... 90 3.4 General Case ... 97 3.5 Exercises ... 98 Chapter 4 Optimal Stopping ... 103 4.1 Optimal Stopping of Markov Chains ... 103 4.2 Optimal Stopping with Cost ... 109 4.3 Optimal Stopping with Discounting ... 112 4.4 Exercises ... 114 Chapter 5 Martingales ... 117 5.1 Conditional Expectation ... 117 5.2 Definition and Examples ... 122 5.3 Optional Sampling Theorem ... 126 5.4 Uniform Integrability ... 130 5.5 Martingale Convergence Theorem ... 132 5.6 Maximal Inequalities ... 138 5.7 Exercises ... 141 Chapter 6 Renewal Processes ... 147 6.1 Introduction ... 147 6.2 Renewal Equation ... 152 6.3 Discrete Renewal Processes ... 160 6.4 M/G/1 and G/M/1 Queues ... 164 6.5 Exercises ... 167 Chapter 7 Reversible Markov Chains ... 171 7.1 Reversible Processes ... 171 7.2 Convergence to Equilibrium ... 173 7.3 Markov Chain Algorithms ... 178 7.4 A Criterion for Recurrence ... 182 7.5 Exercises ... 186 Chapter 8 Brownian Motion ... 189 8.1 Introduction ... 189 8.2 Markov Property ... 192 8.3 Zero Set of Brownian Motion ... 197 8.4 Brownian Motion in Several Dimensions ... 200 8.5 Recurrence and Transience ... 205 8.6 Fractal Nature of Brownian Motion ... 207 8.7 Scaling Rules ... 208 8.8 Brownian Motion with Drift ... 209 8.9 Exercises ... 211 Chapter 9 Stochastic Integration ... 215 9.1 Integration with Respect to Random Walk ... 215 9.2 Integration with Respect to Brownian Motion ... 216 9.3 Ito's Formula ... 221 9.4 Extensions of Ito's Formula ... 225 9.5 Continuous Martingales ... 232 9.6 Girsanov Transformation ... 234 9.7 Feynman-Kac Formula ... 237 9.8 Black-Scholes Formula ... 239 9.9 Simulation ... 244 9.10 Exercises ... 244 Suggestions for Further Reading ... 247 Index ... 249

Emphasizing fundamental mathematical ideas rather than proofs, Introduction to Stochastic Processes, Second Edition provides quick access to important foundations of probability theory applicable to problems in many fields. Assuming that you have a reasonable level of computer literacy, the ability to write simple programs, and the access to software for linear algebra computations, the author approaches the problems and theorems with a focus on stochastic processes evolving with time, rather than a particular emphasis on measure theory.

For those lacking in exposure to linear differential and difference equations, the author begins with a brief introduction to these concepts. He proceeds to discuss Markov chains, optimal stopping, martingales, and Brownian motion. The book concludes with a chapter on stochastic integration. The author supplies many basic, general examples and provides exercises at the end of each chapter.

New to the Second Edition:

Expanded chapter on stochastic integration that introduces modern mathematical finance

Introduction of Girsanov transformation and the Feynman-Kac formula

Expanded discussion of Itô's formula and the Black-Scholes formula for pricing options

New topics such as Doob's maximal inequality and a discussion on self similarity in the chapter on Brownian motion

Applicable to the fields of mathematics, statistics, and engineering as well as computer science, economics, business, biological science, psychology, and engineering, this concise introduction is an excellent resource both for students and professionals.

Booknews

A textbook introducing stochastic processes that evolve with time to graduate and advanced undergraduate students of mathematics, engineering, or science. Assumes an undergraduate-level knowledge of calculus-based probability and linear algebra (including eigenvalues and eigenvectors), and enough computer literacy to write simple programs and access software for linear algebra computation. No bibliography. Annotation c. Book News, Inc., Portland, OR (booknews.com)

"Emphasizing fundamental mathematical ideas rather than proofs, Introduction to Stochastic Processes, Second Edition provides quick access to important foundations of probability theory applicable to problems in many fields. Assuming that you have a reasonable level of computer literacy, the ability to write simple programs, and the access to software for linear algebra computations, the author approaches the problems and theorems with a focus on stochastic processes evolving with time, rather than a particular emphasis on measure theory.For those lacking in exposure to linear differential and difference equations, the author begins with a brief introduction to these concepts. He proceeds to discuss Markov chains, optimal stopping, martingales, and Brownian motion. The book concludes with a chapter on stochastic integration. The author supplies many basic, general examples and provides exercises at the end of each chapter.New to the Second Edition:Expanded chapter on stochastic integration that introduces modern mathematical financeIntroduction of Girsanov transformation and the Feynman-Kac formulaExpanded discussion of Itô's formula and the Black-Scholes formula for pricing optionsNew topics such as Doob's maximal inequality and a discussion on self similarity in the chapter on Brownian motionApplicable to the fields of mathematics, statistics, and engineering as well as computer science, economics, business, biological science, psychology, and engineering, this concise introduction is an excellent resource both for students and professionals."--Provided by publisher Emphasizing fundamental mathematical ideas rather than proofs, Introduction to Stochastic Processes, Second Edition provides quick access to important foundations of probability theory applicable to problems in many fields. Assuming that you have a reasonable level of computer literacy, the ability to write simple programs, and the access to software for linear algebra computations, the author approaches the problems and theorems with a focus on stochastic processes evolving with time, rather than a particular emphasis on measure theory. For those lacking in exposure to linear differential and difference equations, the author begins with a brief introduction to these concepts. He proceeds to discuss Markov chains, optimal stopping, martingales, and Brownian motion. The book concludes with a chapter on stochastic integration. The author supplies many basic, general examples and provides exercises at the end of each chapter. New to the Second Edition: Expanded chapter on stochastic integration that introduces modern mathematical finance Introduction of Girsanov transformation and the Feynman-Kac formula Expanded discussion of Itô's formula and the Black-Scholes formula for pricing options New topics such as Doob's maximal inequality and a discussion on self similarity in the chapter on Brownian motion Applicable to the fields of mathematics, statistics, and engineering as well as computer science, economics, business, biological science, psychology, and engineering, this concise introduction is an excellent resource both for students and professionals. "Focusing on mathematical ideas rather than proofs, Introduction to Stochastic Processes, Second Edition provides quick access to important foundations of probability theory applicable to problems in many fields. Approaching all problems and theorems without any measure theory, the book provides a concise and informal introduction to stochastic processes evolving with time." "This concise, informal introduction is designed to meet the needs of students and professionals not only in mathematics and statistics, but in the many fields in which the concepts presented are also important, including computer science, economics, business, biological sciences, psychology, and engineering. It acquaints readers with the possibilities of applying stochastic processes in their work."--BOOK JACKET Emphasizing fundamental mathematical ideas rather than proofs, this book provides access to important foundations of probability theory applicable to problems in many fields. It also discusses Markov chains, optimal stopping, martingales, and Brownian motion.