Stochastic complexity and modeling. (English) Zbl 0602.62008

While this paper may be regarded in a sense as a continuation of the program begun by Fisher on ”information”, the author’s view is that models should not (nay, cannot) be restricted to have a fixed number of parameters: Fisher’s information is thus inapplicable, and complexity and information must be considered directly in the observed data.
After noting three different notions of complexity, the author defines the stochastic complexity of a string of data, relative to a class of probabilistic models, as ”the fewest number of binary digits with which the data can be encoded by taking advantage of the selected models.” From the main result it appears that scepticism of Bayesian attempts to introduce improper priors is in order. The formalization of the concepts ”useful information” and ”prior information” in the data is also undertaken.
Reviewer: A.Dale


62A01 Foundations and philosophical topics in statistics
62F03 Parametric hypothesis testing
62M10 Time series, auto-correlation, regression, etc. in statistics (GARCH)
60F99 Limit theorems in probability theory
Full Text: DOI