A text book that includes chapters on all aspects of solving Markov chains numerically.

Billy's Home Page   MARCA   MARCA-Models

``Introduction to the Numerical Solution of Markov Chains''


William J. Stewart

Preface,   Organization,   Acknowledgements
Table of Contents,   Ordering Information

NSMC Cover

Published by Princeton University Press, 41 William Street, Princeton, New Jersey 08540 ISBN 0-691-03699-3 QA274.7.S74 1994

About the Book

It is often possible to represent the behavior of a physical system by describing all the different states which it can occupy and by indicating how it moves from one state to another in time. If the future evolution of the system depends only on its current state, the system may be represented by a Markov process. When the state space is discrete, the term ``Markov Chain'' is employed.

In this book, the first to offer a systematic and detailed treatment of the numerical solution of Markov chains, William Stewart provides scientists on many levels with the power to put this theory to use in the actual world, where it has application in areas as diverse as engineering, economics and education. His efforts make essential reading in a rapidly growing field.

Here Stewart explores all aspects of numerically computing solutions of Markov chains, especially when the state space is huge. He provides extensive background to both discrete-time and continuous-time Markov chains and examines many different numerical computing methods --- direct, single and multi-vector iterative, and projection methods. Additionally, he considers recursive methods often used when the structure of the Markov chain is upper Hessenberg; iterative aggregation/disaggregation methods that are particularly appropriate when it is NCD (nearly completely decomposable) and reduced schemes for cases in which the chain is periodic. There are chapters on methods for computing transient solutions, on stochastic automata networks, and finally on currently available software. Throughout, Stewart draws on numerous examples and comparisons among the methods he so thoroughly explains.

Billy's Home Page   MARCA   MARCA-Models