Markov Decision Processes

An up-to-date, unified and rigorous treatment of theoretical, computational and applied research on Markov decision process models. Concentrates on infinite-horizon discrete-time models. Discusses arbitrary state spaces, finite-horizon and continuous-time discrete-state models. Also covers modified policy iteration, multichain models with average reward criterion and sensitive optimality. Features a wealth of figures which illustrate examples and an extensive bibliography.

Examines several fundamentals concerning the manner in which Markov decision problems may be properly formulated and the determination of solutions or their properties. Coverage includes optimal equations, algorithms and their characteristics, probability distributions, modern development in the Markov decision process area, namely structural policy analysis, approximation modeling, multiple objectives and Markov games. Copiously illustrated with examples.

Booknews

A basic text covering some of the fundamentals of formulating and solving Markov decision problems, for graduate students with a moderate mathematical background who are learning to apply statistics in operational research, management science, systems engineering and related fields. Could also be used in an advanced undergraduate mathematics or statistics course. Annotation c. Book News, Inc., Portland, OR (booknews.com)