×

An information approach to regularization parameter selection under model misspecification. (English) Zbl 1022.62004

Summary: We review the information approach to regularization parameter selection and its information complexity extension for the solution of discrete ill posed problems. An information criterion for regularization parameter selection was first proposed by R. Shibata [J. C. Willems (ed.), From Data to Models, 215-240 (1989)] in the context of ridge regression as an extension of K. Takeuchi’s [Math. Sci. 153, 12-18 (Japanese) (1976)] information criterion. In the information approach, the regularization parameter value is chosen to maximize the mean expected log likelihood (MELL) of a model whose parameters are estimated using the maximum penalized likelihood method. Under the Gaussian noise assumption such a choice coincides with the minimum of mean predictive error choice. Maximization of the MELL corresponds to minimization of the mean Kullback-Leibler information, that measures the deviation of the approximating (model) distribution from the true one. The resulting regularization parameter selection methods can handle possible functional and distributional misspecifications when the usual assumptions of Gaussian noise and/or linear relationship have been made but not met.
We also suggest that in engineering applications it is beneficial to find ways of lowering the risk of getting grossly under-regularized solutions and that the new information complexity regularization parameter selection method (RPSM) is one of the possibilities. Several examples of applying the reviewed RPSMs are given.

MSC:

62B10 Statistical aspects of information-theoretic topics
62J05 Linear regression; mixed models
62F12 Asymptotic properties of parametric estimators
62H12 Estimation in multivariate analysis

Keywords:

linear models
PDFBibTeX XMLCite
Full Text: DOI Link