Bayesian theory. (English) Zbl 0943.62009

Wiley Series in Probability and Statistics. Chichester: Wiley. xiv, 586 p. (2000).
This book presents an overview of the foundations and key theoretical concepts of Bayesian statistics. It consists of six chapters, two appendices and the bibliography.
In Chapter 1, “Introduction”, a brief historical introduction to the Bayes theorem and its author is given, as a prelude to a statement of the perspective adopted in the book regarding Bayesian Statistics. Next, an overview of the material covered in chapters 2 to 6 and appendices, and a Bayesian reading list are provided.
In Chapter 2, “Foundations”, the concept of rationality is explored in the context of representing beliefs or choosing actions in situations of uncertainty. An axiomatic basis is introduced for the foundations of decision theory. The dual concepts of probability and utility are formally defined and analyzed within this context. The criterion of maximizing expected utility is shown to be the only decision criterion that is compatible with the axiom system. The analysis of sequential decision problems is shown to reduce to successive applications of the methodology introduced. Statistical inference is viewed as a particular decision problem that may be analyzed within the framework of decision theory. The logarithmic score is established as the natural utility function to describe the preferences of an individual faced with a pure inference problem. Within this framework, the concept of discrepancy between probability distributions and the quantification of the amount of information in new data are defined in terms of expected loss and expected increase in utility, respectively.
In Chapter 3, “Generalisations”, the ideas and results of Chapter 2 are extended to a more general mathematical setting. An additional postulate concerning the comparison of a countable collection of events is introduced and is shown to provide a justification for restricting attention to countably additive probability as the basis for representing beliefs. The elements of mathematical probability theory are reviewed. The notions of options and utilities are extended to provide a very general mathematical framework for decision theory. A further additional postulate regarding preferences is introduced, and is shown to justify the criterion of maximizing expected utility within this more general framework. In the context of inference problems, generalized definitions of score functions and of measures of information and discrepancy are given.
In Chapter 4, “Modelling”, the relationship between beliefs about observable random quantities and their representation using conventional forms of statistical models is investigated. It is shown that judgements of exchangeability lead to representations that justify and clarify the use and interpretation of such familiar concepts as parameters, random samples, likelihoods and prior distributions. Beliefs that have certain additional invariance properties are shown to lead to representations involving familiar specific forms of parametric distributions, such as normal and exponential. The concept of a sufficient statistic is introduced and related to representations involving the exponential family of distributions. Various forms of partial exchangeability judgements about data structures involving several samples, structured layouts, covariates and designed experiments are investigated, and links established with a number of other commonly used statistical models.
In Chapter 5, “Inference”, the role of the Bayes theorem in the updating of beliefs about observables in the light of new information is identified and related to conventional mechanisms of predictive and parametric inference. The roles of sufficiency, ancillarity and stopping rules in such inference processes are also examined. Forms of common statistical decisions and inference summaries are introduced and the problems of implementing Bayesian procedures are discussed at length. In particular, conjugate, asymptotic and reference forms of analysis and numerical approximation approaches are detailed.
In Chapter 6, “Remodelling”, it is argued that whether viewed from the perspective of a sensitive individual modeller, or from that of a group of modellers, there are good reasons for systematically entertaining a range of possible belief models. A variety of decision problems are examined within this framework: some involving model choice only; some involving model choice followed by a terminal action, such as prediction; other involving only a terminal action. Throughout, a clear distinction is drawn between three perspectives: first, the case where the range of models under consideration is assumed to include the “true” belief model; secondly, the case where the range of models is being considered in order to provide a proxy for a specified, but intractable, actual belief model; finally, the case where the range of models is being considered in the absence of specification of an actual belief model. Links with hypothesis testing, significance testing and cross-validation are established.
Each of the chapters 2 to 6 concludes with a “Discussion and Further References” section, in which some of the key issues in the chapter are critically re-examined.
In Appendix A, called “Summary of Basic Formulae”, the authors present, in tabular format, summaries of the main univariate and multivariate probability distributions that appear in the text, together with summaries of the prior/posterior/predictive forms corresponding to these distributions in the context of conjugate and reference analyses.
In Appendix B, called “Non-Bayesian Theories”, the authors review the alternatives to the Bayesian approach; namely classical decision theory, frequentist procedures, likelihood theory, and fiducial and related theories. These alternatives are compared and contrasted in the context of point and interval estimation, and hypothesis and significance testing. Through counter-examples and general discussion, the authors indicate why they find all these alternatives seriously deficient as formal inference theories.


62C10 Bayesian problems; characterization of Bayes procedures
62A01 Foundations and philosophical topics in statistics
62-02 Research exposition (monographs, survey articles) pertaining to statistics
62C12 Empirical decision procedures; empirical Bayes procedures
62-01 Introductory exposition (textbooks, tutorial papers, etc.) pertaining to statistics