Mathematical techniques of applied probability. Volume 2: Discrete time models: Techniques and applications.

*(English)*Zbl 0539.60065
Operations Research and Industrial Engineering. New York - London etc.: Academic Press. A Subsidiary of Harcourt Brace Jovanovich, Publishers. XIII, 286 p. $ 35.00 (1983).

The book is addressed to persons who wish to get acquaintance with initial notions of the theory of random processes using the objects as simple as possible but using rather advanced and strict mathematical tools in order to apply this knowledge in practice. Volume I consists of 5 chapters and ”is suited for a half-year course leading to a thorough understanding of the basic ideas of Markov chains”. Volume II consists of 4 chapters. ”Both of the first two volumes are designed as a text for a one-year course on discrete time stochastic modeling”. Every piece of the material is accompanied by examples and exercises. Indeed the content of these volumes is overlapped by the excellent books by Feller and Karlin. But the author knows it and he restricts himself in choosing the material and making the style of the work deliberately formal.

Chapter 1 contains main notions and definitions referring to probability theory: probability measure, sample space, \(\sigma\)-algebra, distribution, conditional probability, independence, stochastic process and others. All assertions are given in the form of theorems but the reader is referred to other texts for their proofs. In chapter 2 we find some auxiliary mathematical material - generating functions of probability distributions and analysis of their properties. The proofs are given only for some non-standard results. Recurrent event theory is the kernel of Volume I and is contained in chapter 3. Here a strict (but not intuitive as in most texts) definition of such events is given and main results concerning these events are considered. The proof of the discrete-time renewal theorem (Theorem 3.3.6) is not given as it ”does not involve probabilistic ideas”. I think that if one breaks a tradition here and uses the ideas of coupling then it gives the possibility both to involve probabilistic ideas and to receive the convergence rate estimates which are of great importance in applications. Among other problems it is worth to mention: the number of occurrences of a recurrent event, general recurrent event processes, central limit theorem for recurrent events, thorough consideration of Bernoulli trials.

Chapter 4 contains some elements of matrix theory which are necessary for Markov chain theory. It includes both initial notions (determinant, inverse, rank, generalized inverse) and problems of solving linear equations, diagonalization, spectral theory, Perron-Frobenius’s theorem. The culmination of volume I is chapter 5 which contains the main notions for Markov chains. The existence of recurrent events in the sense of chapter 3 is proved and on their base a classification of the states is given. The problems of decomposition of the state space and the existence of limit distributions are under consideration as well.

In chapter 6, which is in volume II, the author considers general problems of computing different characteristics for discrete Markov chains. The fundament of this is the theory of stochastic matrices. The formulas for finding transient probabilities and n-step first-passage time probabilities are given. Existence and different properties of solutions of the corresponding equations are also considered. Chapter 7 is devoted to the study of the limit behaviour of Markov chains. The problems of existence of limit probabilities (and convergence rates) for states of different types are investigated. Some criteria (as Foster’s criteria) for a classification of states of Markov chains are given. These criteria are often used in applications (e.g. queueing theory). A useful peculiarity is the use of the generalized inverse for finding stationary probabilities. Non-traditional is the material about the moments of first-passage times and occupation times. These problems often arise in applications and the author gives both the strict solution and the corresponding estimates.

In chapter 8 applications of Markov chains to discrete branching processes are considered. Equations for determining state-probabilities and surviving probabilities are derived. The problems of qualitative behaviour and classification of corresponding chains are investigated. Chapter 9 contains the consideration of some discrete queueing models. The author restricts himself to the simplest models adopting the analysis by imbedded Markov chains: geometric/geometric/1 model, geometric/G/1 model and GI/geometric/1 model. The qualitative and limit behaviour of these chains is studied.

Chapter 1 contains main notions and definitions referring to probability theory: probability measure, sample space, \(\sigma\)-algebra, distribution, conditional probability, independence, stochastic process and others. All assertions are given in the form of theorems but the reader is referred to other texts for their proofs. In chapter 2 we find some auxiliary mathematical material - generating functions of probability distributions and analysis of their properties. The proofs are given only for some non-standard results. Recurrent event theory is the kernel of Volume I and is contained in chapter 3. Here a strict (but not intuitive as in most texts) definition of such events is given and main results concerning these events are considered. The proof of the discrete-time renewal theorem (Theorem 3.3.6) is not given as it ”does not involve probabilistic ideas”. I think that if one breaks a tradition here and uses the ideas of coupling then it gives the possibility both to involve probabilistic ideas and to receive the convergence rate estimates which are of great importance in applications. Among other problems it is worth to mention: the number of occurrences of a recurrent event, general recurrent event processes, central limit theorem for recurrent events, thorough consideration of Bernoulli trials.

Chapter 4 contains some elements of matrix theory which are necessary for Markov chain theory. It includes both initial notions (determinant, inverse, rank, generalized inverse) and problems of solving linear equations, diagonalization, spectral theory, Perron-Frobenius’s theorem. The culmination of volume I is chapter 5 which contains the main notions for Markov chains. The existence of recurrent events in the sense of chapter 3 is proved and on their base a classification of the states is given. The problems of decomposition of the state space and the existence of limit distributions are under consideration as well.

In chapter 6, which is in volume II, the author considers general problems of computing different characteristics for discrete Markov chains. The fundament of this is the theory of stochastic matrices. The formulas for finding transient probabilities and n-step first-passage time probabilities are given. Existence and different properties of solutions of the corresponding equations are also considered. Chapter 7 is devoted to the study of the limit behaviour of Markov chains. The problems of existence of limit probabilities (and convergence rates) for states of different types are investigated. Some criteria (as Foster’s criteria) for a classification of states of Markov chains are given. These criteria are often used in applications (e.g. queueing theory). A useful peculiarity is the use of the generalized inverse for finding stationary probabilities. Non-traditional is the material about the moments of first-passage times and occupation times. These problems often arise in applications and the author gives both the strict solution and the corresponding estimates.

In chapter 8 applications of Markov chains to discrete branching processes are considered. Equations for determining state-probabilities and surviving probabilities are derived. The problems of qualitative behaviour and classification of corresponding chains are investigated. Chapter 9 contains the consideration of some discrete queueing models. The author restricts himself to the simplest models adopting the analysis by imbedded Markov chains: geometric/geometric/1 model, geometric/G/1 model and GI/geometric/1 model. The qualitative and limit behaviour of these chains is studied.

Reviewer: V.Kalashnikov

##### MSC:

60J10 | Markov chains (discrete-time Markov processes on discrete state spaces) |

60-01 | Introductory exposition (textbooks, tutorial papers, etc.) pertaining to probability theory |

60K25 | Queueing theory (aspects of probability theory) |

60F05 | Central limit and other weak theorems |

60-02 | Research exposition (monographs, survey articles) pertaining to probability theory |

60K05 | Renewal theory |