×

Linear estimating equations for exponential families with application to Gaussian linear concentration models. (English) Zbl 1312.62068

Summary: In many families of distributions, maximum likelihood estimation is intractable because the normalization constant for the density which enters into the likelihood function is not easily available. The score matching estimator provides an alternative where this normalization constant is not required. For an exponential family, e.g. a Gaussian linear concentration model, the corresponding estimating equations become linear and the score matching estimator is shown to be consistent and asymptotically normally distributed as the number of observations increase to infinity, although not necessarily efficient. For linear concentration models that are also linear in the covariance we show that the score matching estimator is identical to the maximum likelihood estimator, hence in such cases it is also efficient. Gaussian graphical models and graphical models with symmetries form particularly interesting subclasses of linear concentration models and we investigate the potential use of the score matching estimator for this case.

MSC:

62H12 Estimation in multivariate analysis
62F10 Point estimation
62F12 Asymptotic properties of parametric estimators

Software:

gRc; HdBCS; glasso; CRAN
PDFBibTeX XMLCite
Full Text: DOI arXiv

References:

[1] Albert, A. A., On Jordan algebras of linear transformations, Trans. Amer. Math. Soc., 59, 524-555 (1946) · Zbl 0061.05101
[2] Almeida, M. P.; Gidas, B., A variational method for estimating the parameters of MRF from complete or incomplete data, Ann. Appl. Probab., 3, 103-136 (1993) · Zbl 0771.60091
[3] Anderson, T. W., Estimation of covariance matrices which are linear combinations or whose inverses are linear combinations of given matrices, (Bose, R. C.; Chakravarti, I. M.; Mahalanobis, P. C.; Rao, C. R.; Smith, K. J.C., Essays in Probability and Statistics (1970), University of North Carolina Press: University of North Carolina Press Chapel Hill, NC), 1-24 · Zbl 0265.62023
[4] Andersson, S. A., Invariant normal models, Ann. Statist., 3, 132-154 (1975) · Zbl 0373.62029
[5] Banerjee, O.; El Ghaoui, L.; d’Aspremont, A., Model selection through sparse maximum likelihood estimation for multivariate Gaussian or binary data, J. Mach. Learn. Res., 9, 485-516 (2008) · Zbl 1225.68149
[6] Banerjee, O.; Ghaoui, L. E.; d’Aspremont, A.; Natsoulis, G., Convex optimization techniques for fitting sparse Gaussian graphical models, (Proceedings of the 23rd International Conference on Machine Learning. Proceedings of the 23rd International Conference on Machine Learning, ICML ’06 (2006), ACM: ACM New York, NY, USA), 89-96
[7] Barndorff-Nielsen, O. E., Plausibility inference, J. R. Stat. Soc. Ser. B, 38, 103-131 (1976) · Zbl 0403.62008
[8] Barndorff-Nielsen, O. E., Information and Exponential Families in Statistical Theory (1978), John Wiley and Sons: John Wiley and Sons New York · Zbl 0387.62011
[9] Barndorff-Nielsen, O. E.; Cox, D. R., Inference and Asymptotics (1994), Chapman and Hall: Chapman and Hall London, UK · Zbl 0826.62004
[10] Bernardo, J. M., Expected information as expected utility, Ann. Statist., 7, 686-690 (1979) · Zbl 0407.62002
[11] Bickel, P. J.; Levina, E., Regularized estimation of large covariance matrices, Ann. Statist., 36, 199-227 (2008) · Zbl 1132.62040
[12] Buhl, S., On the existence of maximum likelihood estimators for graphical Gaussian models, Scand. J. Stat., 20, 263-270 (1993) · Zbl 0778.62046
[13] Chandrasekaran, V.; Parrilo, P. A.; Willsky, A. S., Latent variable graphical model selection via convex optimization, Ann. Statist., 40, 1935-1967 (2012) · Zbl 1257.62061
[14] Chow, C. K.; Liu, C. N., Approximating discrete probability distributions with dependence trees, IEEE Trans. Inform. Theory, 14, 462-467 (1968) · Zbl 0165.22305
[15] Dawid, A. P., Coherent measures of discrepancy, uncertainty and dependence, with applications to Bayesian predictive experimental design (1998), Department of Statistical Science, University College London, Technical Report 139
[16] Dawid, A. P., The geometry of proper scoring rules, Ann. Inst. Statist. Math., 59, 77-93 (2007) · Zbl 1108.62009
[17] Dawid, A. P.; Lauritzen, S. L., The geometry of decision theory, (Proceedings of the Second International Symposium on Information Geometry and Its Applications (2005), University of Tokyo), 22-28
[18] Dempster, A. P., Covariance selection, Biometrics, 28, 157-175 (1972)
[19] Dobra, A.; Hans, C.; Jones, B.; Nevins, J. R.; Yao, G.; West, M., Sparse graphical models for exploring gene expression data, J. Multivariate Anal., 90, 196-212 (2004), Special Issue on Multivariate Methods in Genomic Data Analysis · Zbl 1047.62104
[20] Edwards, D.; de Abreu, G.; Labouriau, R., Selecting high-dimensional mixed graphical models using minimal AIC or BIC forests, BMC Bioinform., 11, 18 (2010)
[21] Edwards, D.; Havránek, T., A fast model selection procedure for large families of models, J. Amer. Statist. Assoc., 82, 205-213 (1987) · Zbl 0607.62086
[22] Friedman, J.; Hastie, T.; Tibshirani, R., Sparse inverse covariance estimation with the graphical lasso, Biostatistics, 9, 432-441 (2008) · Zbl 1143.62076
[24] Gehrmann, H., Lattices of graphical Gaussian models with symmetries, Symmetry, 3, 653-679 (2011) · Zbl 1360.62266
[25] Gneiting, T.; Raftery, A. E., Strictly proper scoring rules, prediction, and estimation, J. Amer. Statist. Assoc., 102, 359-378 (2007) · Zbl 1284.62093
[26] Godambe, V. P., Estimating Functions (1991), Clarendon Press · Zbl 0745.00006
[27] Good, I. J., Rational decisions, J. R. Stat. Soc. Ser. B, 14, 107-114 (1952)
[28] Grünwald, P. D.; Dawid, A. P., Game theory, maximum entropy, minimum discrepancy, and robust Bayesian decision theory, Ann. Statist., 32, 1367-1433 (2004) · Zbl 1048.62008
[29] Hannan, E. J.; Quinn, B. G., The determination of an order of an autoregression, J. R. Stat. Soc. Ser. B, 41, 190-195 (1979) · Zbl 0408.62076
[30] Höglund, T., The exact estimate - a method of statistical estimation, Z. Wahrscheinlichkeitstheorie Verw. Gebiete, 29, 257-271 (1974) · Zbl 0283.62035
[31] Højsgaard, S.; Lauritzen, S. L., Inference in graphical Gaussian models with edge and vertex symmetries with the gRc package for R, J. Statist. Software, 23, 1-26 (2007)
[32] Højsgaard, S.; Lauritzen, S. L., Graphical Gaussian models with edge and vertex symmetries, J. R. Stat. Soc. Ser. B, 70, 1005-1027 (2008) · Zbl 1411.62177
[33] Huber, P. J., Robust estimation of a location parameter, Ann. Appl. Stat., 35, 73-101 (1964) · Zbl 0136.39805
[34] Huber, P. J., The behavior of maximum likelihood estimates under nonstandard conditions, (Cam, L. M.L.; Neyman, J., Proceedings of the Fifth Berkeley Symposium on Mathematical Statistics and Probability, vol. 1 (1967), University of California Press: University of California Press Berkeley, CA), 221-223 · Zbl 0212.21504
[35] Hyvärinen, A., Estimation of non-normalized statistical models by score matching, J. Mach. Learn. Res., 6, 695-709 (2005) · Zbl 1222.62051
[36] Hyvärinen, A., Some extensions of score matching, Comput. Statist. Data Anal., 51, 2499-2512 (2007) · Zbl 1161.62326
[37] Jensen, S. T., Covariance hypotheses which are linear in both the covariance and the inverse covariance, Ann. Statist., 16, 302-322 (1988) · Zbl 0653.62042
[38] Kruskal, J., On the shortest spanning subtree of a graph and the traveling salesman problem, Proc. Amer. Math. Soc., 7, 48-50 (1956) · Zbl 0070.18404
[39] Ma, S.; Gong, Q.; Bohnert, H. J., An Arabidopsis gene network based on the graphical Gaussian model, Genome Research, 17, 1614-1625 (2007)
[40] Mardia, K. V.; Kent, J. T.; Bibby, J. M., Multivariate Analysis (1979), Academic Press: Academic Press London · Zbl 0432.62029
[41] Martin-Löf, P., Exact tests, confidence regions and estimates, Synthese, 36, 195-206 (1977) · Zbl 0362.62050
[42] McCarthy, J., Measures of the value of information, Proc. Nat. Acad. Sci., 42, 654-655 (1956) · Zbl 0072.37501
[43] Okamoto, M., Distinctness of the eigenvalues of a quadratic form in a multivariate sample, Ann. Statist., 1, 763-765 (1973) · Zbl 0261.62043
[44] Rothman, A. J.; Bickel, P. J.; Levina, E.; Zhu, J., Sparse permutation invariant covariance estimation, Electron. J. Stat., 2, 494-515 (2008) · Zbl 1320.62135
[45] Uhler, C., Geometry of maximum likelihood estimation in Gaussian graphical models, Ann. Statist., 40, 238-261 (2012) · Zbl 1246.62140
[46] van der Vaart, A. W., Asymptotic Statistics (1998), Cambridge University Press: Cambridge University Press Cambridge, UK · Zbl 0910.62001
[47] van Erven, T.; Grünwald, P.; de Rooij, S., Catching up faster by switching sooner: a predictive approach to adaptive estimation with an application to the AIC-BIC dilemma (with discussion), J. R. Stat. Soc. Ser. B, 74, 361-417 (2012) · Zbl 1411.62073
This reference list is based on information provided by the publisher or from digital mathematics libraries. Its items are heuristically matched to zbMATH identifiers and may contain data conversion errors. In some cases that data have been complemented/enhanced by data from zbMATH Open. This attempts to reflect the references listed in the original paper as accurately as possible without claiming completeness or a perfect matching.