A simple approach to maximum intractable likelihood estimation. (English) Zbl 1327.62075

Summary: Approximate Bayesian Computation (ABC) can be viewed as an analytic approximation of an intractable likelihood coupled with an elementary simulation step. Such a view, combined with a suitable instrumental prior distribution permits maximum-likelihood (or maximum-a-posteriori) inference to be conducted, approximately, using essentially the same techniques. An elementary approach to this problem which simply obtains a nonparametric approximation of the likelihood surface which is then maximised is developed here and the convergence of this class of algorithms is characterised theoretically. The use of non-sufficient summary statistics in this context is considered. Applying the proposed method to four problems demonstrates good performance. The proposed approach provides an alternative for approximating the maximum likelihood estimator (MLE) in complex scenarios.


62E17 Approximations to statistical distributions (nonasymptotic)
62F10 Point estimation
62F12 Asymptotic properties of parametric estimators
62G07 Density estimation
65C05 Monte Carlo methods
Full Text: DOI arXiv Euclid


[1] Abraham, C., Biau, G. and Cadre, B. (2003). Simple estimation of the mode of a multivariate density. The Canadian Journal of Statistics 31: 23-34. · Zbl 1035.62046
[2] Beaumont, M. A., Zhang, W. and Balding, D. J. (2002). Approximate Bayesian computation in population genetics. Genetics 162: 2025-2035.
[3] Besag, J. (1975). Statistical Analysis of Non-Lattice Data. The Statistician 24:179-195.
[4] Biau, G., Cérou, F. and Guyader, A. (2012). New Insights into Approximate Bayesian Computation. ArXiv · Zbl 1307.62012
[5] Bickel, D. R. and Früwirth, R. (2006). On a fast, robust estimator of the mode: Comparisons to other robust estimators with applications. Computational Statistics & Data Analysis 50: 3500-3530. · Zbl 1445.62055
[6] Blum, M. G. B. (2010). Approximate Bayesian computation: a nonparametric perspective. Journal of the American Statistical Association 105: 1178-1187. · Zbl 1390.62052
[7] Bretó, C., Daihi, H., Ionides, E. L. and King, A. A. (2009). Time series analysis via mechanistic models. Annals of Applied Statistics 3: 319-348. · Zbl 1160.62080
[8] Cox, D. R. and Kartsonaki, C. (2012). The fitting of complex parametric models. Biometrika 99: 741-747. · Zbl 1437.62432
[9] Cox, D. R. and Reid, N. (2004). A note on pseudolikelihood constructed from marginal densities. Biometrika 91: 729-737. · Zbl 1162.62365
[10] Cox, D. R. and Smith, W. L. (1954). On the superposition of renewal processes. Biometrika 41: 91-9. · Zbl 0055.12401
[11] Cule, M. L., Samworth, R. J. and Stewart, M. I. (2010). Maximum likelihood estimation of a multi-dimensional log-concave density. Journal Royal Statistical Society B 72: 545-600. · Zbl 1329.62183
[12] Dean, T. A., Singh, S. S., Jasra A. and Peters G. W. (2011). Parameter estimation for hidden Markov models with intractable likelihoods. ArXiv · Zbl 1305.62303
[13] de Valpine, P. (2004). Monte Carlo state space likelihoods by weighted posterior kernel density estimation. Journal of the American Statistical Association 99: 523-536. · Zbl 1117.62314
[14] Didelot, X., Everitt, R. G., Johansen, A. M. and Lawson, D. J. (2011). Likelihood-free estimation of model evidence. Bayesian Analysis 6: 49-76. · Zbl 1330.62118
[15] Diggle, P. J. and Gratton, R. J. (1984) Monte Carlo Methods of Inference for Implicit Statistical Models. Journal of the Royal Statistical Society B 46:193-227. · Zbl 0561.62035
[16] Duong, T. and Hazleton, M. L. (2005). Cross-validation bandwidth matrices for multivariate kernel density estimation. Scandinavian Journal of Statistics 32:485-506. · Zbl 1089.62035
[17] Duong, T. (2011). ks: Kernel smoothing . R package version 1.8.5.
[18] Ehrlich, E., Jasra, A. and Kantas, N. (2012). Static parameter estimation for ABC approximations of hidden Markov models. ArXiv · Zbl 1323.65006
[19] Fan, Y., Nott, D. J. and Sisson, S. A. (2012). Approximate Bayesian Computation via Regression Density Estimation. Stat 2: 34-48.
[20] Fearnhead, P. and Prangle, D. (2012). Constructing Summary Statistics for Approximate Bayesian Computation: Semi-automatic ABC (with discussion). Journal of the Royal Statistical Society B 74: 419-474.
[21] Fukunaga, K. and Hostetler, L. D. (1975). The Estimation of the Gradient of a Density Function, with Applications in Pattern Recognition. IEEE Transactions on Information Theory 21: 32-40. · Zbl 0297.62025
[22] Gaetan, C. and Yao, J. F. (2003). A multiple-imputation Metropolis version of the EM algorithm. Biometrika 90: 643-654. · Zbl 1436.62078
[23] Gouriéroux, C., Monfort, A. and Renault, E. (1993). Indirect Inference. Journal of Applied Econometrics 8:S85-S118. · Zbl 1448.62202
[24] Ionides, E. L. (2005). Maximum Smoothed Likelihood Estimation. Statistica Sinica 15: 1003-1014. · Zbl 1086.62032
[25] Jaki, T. and West, R. W. (2008). Maximum Kernel Likelihood Estimation. Journal of Computational and Graphical Statistics 17: 976. · Zbl 1206.62061
[26] Jasra, A., Kantas, N. and Ehrlich, E. (2013). Approximate Inference for Observation Driven Time Series Models with Intractable Likelihoods. ArXiv · Zbl 1322.65011
[27] Jing, J., Koch, I. and Naito, K. (2012). Polynomial Histograms for Multivariate Density and Mode Estimation. Scandinavian Journal of Statistics 39: 75-96. · Zbl 1246.62097
[28] Johansen, A. M., Doucet, A., and Davy, M. (2008). Particle methods for maximum likelihood parameter estimation in latent variable models. Statistics and Computing 18: 47-57.
[29] Konakov, V. D. (1973). On asymptotic normality of the sample mode of multivariate distributions. Theory of Probability and its Applications 18: 836-842. · Zbl 0325.62033
[30] Lehmann, E. and Casella, G. (1998). Theory of Point Estimation (revised edition). Springer-Verlag, New York. · Zbl 0916.62017
[31] Lele, S. R., Dennis, B. and Lutscher, F. (2007). Data cloning: easy maximum likelihood estimation for complex ecological models using Bayesian Markov chain Monte Carlo methods. Ecology Letters 10: 551-563.
[32] Marin, J.-M., Pudlo, P., Robert, C. P. and Ryder, R. (2011). Approximate Bayesian Computational methods. Statistics and Computing 21: 289-291.
[33] Marin, J.-M., Pillai, N., Robert, C.P. and Rousseau, J. (2013). Relevant statistics for Bayesian model choice. ArXiv
[34] Marjoram, P., Molitor, J., Plagnol, V. and Tavaré, S. (2003). Markov chain Monte Carlo without likelihoods. Proceedings of the National Academy of Sciences of the United States of America : 15324-15328.
[35] Mengersen, K. L., Pudlo, P. and Robert, C. P. (2013). Bayesian computation via empirical likelihood. Proceedings of the National Academy of Sciences of the United States of America 110: 1321-1326.
[36] Nolan, J. P. (2001). Maximum likelihood estimation and diagnostics for stable distributions. In: O.E. Barndorff-Nielsen, T. Mikosh, and S. Resnick, Eds., Lévy Processes, Birkhauser, Boston, 379-400. · Zbl 0971.62008
[37] Parzen, E. (1962). On estimation of a probability density function and mode. Annals of Mathematical Statistics 33: 1065-1076. · Zbl 0116.11302
[38] Peters, G. W., Sisson, S. A. and Fan, Y. (2010). Likelihood-free Bayesian inference for \(\alpha-\)stable models. Computational Statistics & Data Analysis 56: 3743-3756. · Zbl 1255.62071
[39] Pritchard, J. K., Seielstad, M. T., Perez-Lezaun, A., and Feldman, M. T. (1999). Population Growth of Human Y Chromosomes: A Study of Y Chromosome Microsatellites. Molecular Biology and Evolution 16: 1791-1798.
[40] Robert, C. P., Cornuet, J., Marin, J. and Pillai, N. S. (2011). Lack of confidence in ABC model choice. Proceedings of the National Academy of Sciences of the United States of America 108: 15112-15117.
[41] Romano, J. P. (1988). On weak convergence and optimality of kernel density estimates of the mode. The Annals of Statistics 16: 629-647. · Zbl 0658.62053
[42] Rubio, F. J. and Johansen, A. M. (March, 2012). On Maximum Intractable Likelihood Estimation. University of Warwick, Dept. of Statistics. CRiSM working paper 12-04.
[43] Rudin, W. (1976). Principles of Mathematical Analysis . New York: McGraw-Hill. · Zbl 0346.26002
[44] Sisson, S. A., Fan, Y. and Tanaka, M. M. (2007). Sequential Monte Carlo without likelihoods. Proceedings of the National Academy of Sciences of the United States of America , 104: 1760-1765. · Zbl 1160.65005
[45] Sköld, M. and Roberts, G. O. (2003). Density estimation for the Metropolis-Hastings algorithm. Scandinavian Journal of Statistics 30: 699-718. · Zbl 1055.65020
[46] Toni, T., Welch, D., Strelkowa, N., Ipsen, A. and Stumpf, M. P. H. (2009). Approximate Bayesian computation scheme for parameter inference and model selection in dynamical systems. Journal of the Royal Society Interface 6: 187-202.
[47] Whitney, K. N. (1991). Uniform Convergence in probability and stochastic equicontinuity. Econometrica 59: 1161-1167. · Zbl 0743.60012
[48] Wilkinson, R. D. (2008). Approximate Bayesian computation (ABC) gives exact results under the assumption of model error. ArXiv
[49] Wuertz, D. and R core team members (2010). fBasics: Rmetrics - Markets and Basic Statistics . R package version 2110.79.
This reference list is based on information provided by the publisher or from digital mathematics libraries. Its items are heuristically matched to zbMATH identifiers and may contain data conversion errors. It attempts to reflect the references listed in the original paper as accurately as possible without claiming the completeness or perfect precision of the matching.