Information theory. Coding theorems for discrete memoryless systems.

*(English)*Zbl 0568.94012
Probability and Mathematical Statistics. (In Hungary: Disquisitiones Mathematicae Hungaricae, Vol. 12). New York-San Francisco-London: Academic Press (Harcourt Brace Jovanovich, Publishers) (Orig. publ. by Akadémiai Kiadó, Budapest). XI, 452 p. $ 64.00 (1981).

This is an important book on information theory which includes a comprehensive, rigorous and unified treatment of discrete memoryless sources and channels for graduate students and applied mathematicians. The book is divided into three chapters.

Chapter 1 (Information easures in simple coding problems) provides an introduction and a list of notations and conventions, and deals with noiseless source coding and some fundamental combinatorial results on which the rest of the book is based (the “types and typical sequences” approach proves to be compact and fruitful in quickly deriving most known results in discrete information theory).

Chapter 2 (Two-terminal systems) covers classical information theory, e.g. the noisy-channel coding theorem, rate distortion theory, the algorithms for computing the capacity and the rate-distortion function etc. The results are expressed in universal, exponentially tight form.

Chapter 3, devoted to multiterminal systems, is the most challenging and certainly required a huge effort to collect, unify and extend the bulk of literature that has been accumulated since 1970 in this area. The method of “types” adopted by the authors proves again to be an ideal means for proving and unifying the results from many different areas of information theory.

The book is of extremely high density, and its reading requires an effort which is greater than with most books on information theory. Several classical, and nonclassical, results are given as problems for the reader and some of them really prove to be tough. But the book is worth reading and re-reading for all people seriously interested in Shannon theory and will certainly be a milestone in the development of mathematical information theory and a reference book for many years.

Chapter 1 (Information easures in simple coding problems) provides an introduction and a list of notations and conventions, and deals with noiseless source coding and some fundamental combinatorial results on which the rest of the book is based (the “types and typical sequences” approach proves to be compact and fruitful in quickly deriving most known results in discrete information theory).

Chapter 2 (Two-terminal systems) covers classical information theory, e.g. the noisy-channel coding theorem, rate distortion theory, the algorithms for computing the capacity and the rate-distortion function etc. The results are expressed in universal, exponentially tight form.

Chapter 3, devoted to multiterminal systems, is the most challenging and certainly required a huge effort to collect, unify and extend the bulk of literature that has been accumulated since 1970 in this area. The method of “types” adopted by the authors proves again to be an ideal means for proving and unifying the results from many different areas of information theory.

The book is of extremely high density, and its reading requires an effort which is greater than with most books on information theory. Several classical, and nonclassical, results are given as problems for the reader and some of them really prove to be tough. But the book is worth reading and re-reading for all people seriously interested in Shannon theory and will certainly be a milestone in the development of mathematical information theory and a reference book for many years.

Reviewer: G. Longo

##### MSC:

94A15 | Information theory (general) |

94A24 | Coding theorems (Shannon theory) |

94-02 | Research exposition (monographs, survey articles) pertaining to information and communication theory |

94A29 | Source coding |

94A40 | Channel models (including quantum) in information and communication theory |

94A17 | Measures of information, entropy |

94A34 | Rate-distortion theory in information and communication theory |