
``Introduction to the Numerical Solution of Markov Chains''William J. Stewart 
Preface,
Organization,
Acknowledgements
Table of Contents,
Ordering Information
Published by Princeton University Press, 41 William Street, Princeton, New Jersey 08540 ISBN 0691036993 QA274.7.S74 1994
It is often possible to represent the behavior of a physical system by describing all the different states which it can occupy and by indicating how it moves from one state to another in time. If the future evolution of the system depends only on its current state, the system may be represented by a Markov process. When the state space is discrete, the term ``Markov Chain'' is employed.
In this book, the first to offer a systematic and detailed treatment of the numerical solution of Markov chains, William Stewart provides scientists on many levels with the power to put this theory to use in the actual world, where it has application in areas as diverse as engineering, economics and education. His efforts make essential reading in a rapidly growing field.
Here Stewart explores all aspects of numerically computing
solutions of Markov chains, especially when the state space is
huge. He provides extensive background to both discretetime
and continuoustime Markov chains and examines many different
numerical computing methods  direct, single and multivector
iterative, and projection methods. Additionally, he considers
recursive methods often used when the structure of the Markov
chain is upper Hessenberg; iterative aggregation/disaggregation
methods that are particularly appropriate when it is NCD (nearly
completely decomposable) and reduced schemes for cases in which
the chain is periodic. There are chapters on methods for computing
transient solutions, on stochastic automata networks, and finally
on currently available software. Throughout, Stewart draws on
numerous examples and comparisons among the methods he so
thoroughly explains.