On Monte Carlo methods for estimating ratios of normalizing constants. (English) Zbl 0936.62028

Summary: Recently, estimating ratios of normalizing constants have played an important role in Bayesian computations. Applications of estimating ratios of normalizing constants arise in many aspects of Bayesian statistical inference. We present an overview and discuss the current Monte Carlo methods for estimating ratios of normalizing constants. Then we propose a new ratio importance sampling method and establish its theoretical framework.
We find that the ratio importance sampling method can be better than the current methods, for example, the bridge sampling method [X.-L. Meng and W. H. Wong, Stat. Sin. 6, No. 4, 831-860 (1996; Zbl 0857.62017)] and the path sampling method [X.-L. Gelman and X.-L. Meng, Path sampling for computing normalizing constants: identities and theory. Tech. Rep. 377, Dpt. Stat., Univ. Chicago (1994)], in the sense of minimizing asymptotic relative mean-square errors of estimators. An example is given for illustrative purposes. Finally, we present two special applications and the general implementation issues for estimating ratios of normalizing constants.


62F15 Bayesian inference
65C05 Monte Carlo methods
65C40 Numerical analysis or methods applied to Markov chains
65C60 Computational problems in statistics (MSC2010)


Zbl 0857.62017


Full Text: DOI


[1] BERGER, J. O. and PERICCHI, L. R. 1996. The intrinsic Bay es factor for model selection and prediction. J. Amer. Statist. Assoc. 91 109 122. Z. JSTOR: · Zbl 0870.62021
[2] CHEN, M.-H. 1994a. Simulating ratios of normalization constants for the Gibbs Sampler. Technical report, Dept. Mathematical Sciences, Worcester Poly technic Inst.
[3] CHEN, M.-H. 1994b. Importance-weighted marginal Bayesian posterior density estimation. J. Amer. Statist. Assoc. 89 818 824. Z. JSTOR: · Zbl 0804.62040
[4] CHEN, M.-H. and SCHMEISER, B. W. 1993. Performance of the Gibbs, hit-and-run, and Metropolis samplers. J. Comput. Graph. Statist. 2 251 272. Z. JSTOR: · Zbl 04516295
[5] CHIB, S. 1995. Marginal likelihood from the Gibbs output. J. Amer. Statist. Assoc. 90 1313 1321. Z. JSTOR: · Zbl 0868.62027
[6] DEVROy E, L. 1986. Non-Uniform Random Variate Generation. Springer, New York. Z. · Zbl 0593.65005
[7] GELFAND, A. E. and SMITH, A. F. M. 1990. Sampling based approaches to calculating marginal densities. J. Amer. Statist. Assoc. 85 398 409. Z. JSTOR: · Zbl 0702.62020
[8] GELFAND, A. E., SMITH, A. F. M. and LEE, T. M. 1992. Bayesian analysis of constrained parameter and truncated data problems using Gibbs sampling. J. Amer. Statist. Assoc. 87 523 532. Z. JSTOR:
[9] GELMAN, A. and MENG, X.-L. 1994. Path sampling for computing normalizing constants: identities and theory. Technical Report 377, Dept. Statistics, Univ. Chicago. Z.
[10] GELMAN, A. and MENG, X.-L. 1996. Simulating normalizing constants: from importance sampling to bridge sampling to path sampling. Technical Report 440, Dept. Statistics, Univ. Chicago. Z. · Zbl 0966.65004
[11] GEMAN, S. and GEMAN, D. 1984. Stochastic relaxation, Gibbs distributions and the Bayesian restoration of images. IEEE Transactions on Pattern Analy sis and Machine Intelligence 6 721 741. Z. · Zbl 0573.62030
[12] GEWEKE, J. 1989. Bayesian inference in econometrics models using Monte Carlo integration. Econometrica 57 1317 1340. Z. JSTOR: · Zbl 0683.62068
[13] GEWEKE, J. 1994. Bayesian comparison of econometric models. Technical Report 532, Federal Reserve Bank of Minneapolis and Univ. Minnesota. Z.
[14] GEy ER, C. J. 1994. Estimating normalizing constants and reweighting mixtures in Markov chain Monte Carlo. Rev. Technical Report 568, School of Statistics, Univ. Minnesota. Z.
[15] GREEN, P. J. 1992. Discussion of “Constrained Monte Carlo maximum likelihood for dependent data,” by C. J. Gey er and E. A. Thompson. J. Roy. Statist. Soc. Ser. B 54 657 699. Z. JSTOR:
[16] HASTINGS, W. K. 1970. Monte Carlo sampling methods using Markov chains and their applications. Biometrika 57 97 109. Z. · Zbl 0219.65008
[17] HE, X. and SHAO, Q. M. 1996. A general Bahadur representation of M-estimators and its application to linear regression with nonstochastic designs. Ann. Statist. 24 2608 2630. Z. · Zbl 0867.62012
[18] JANSSEN, P., JURECKOVA, J. and VERAVERBEKE, N. 1985. Rate of convergence of oneand two-step M-estimators with applications to maximum likelihood and Pitman estimators. Ann. Statist. 13 1222 1229. Z. · Zbl 0585.62057
[19] MENG, X. L. and WONG, W. H. 1996. Simulating ratios of normalizing constants via a simple identity: a theoretical exploration. Statist. Sinica 6 831 860. Z. · Zbl 0857.62017
[20] METROPOLIS, N., ROSENBLUTH, A. W., ROSENBLUTH, M. N., TELLER, A. H. and TELLER, E. 1953. Equations of state calculations by fast computing machines. J. Chem. Phy s. 21 1087 1092. Z.
[21] MULLER, P. 1991. A generic approach to posterior integration and Gibbs sampling. Technical \" Report 91-09, Dept. Statistics, Purdue Univ. Z.
[22] TANNER, M. A. and WONG, W. H. 1987. The calculation of posterior distributions by data augmentation. J. Amer. Statist. Assoc. 82 528 549. Z. Z. JSTOR: · Zbl 0619.62029
[23] TIERNEY, L. 1994. Markov chains for exploring posterior distributions with discussion. Ann. Statist. 22 1701 1762. · Zbl 0829.62080
[24] WORCESTER, MASSACHUSETTS 01609-2280 EUGENE, OREGON 97403-1222 E-MAIL: mhchen@wpi.edu E-MAIL: qmshao@darkwing.uoregon.edu
This reference list is based on information provided by the publisher or from digital mathematics libraries. Its items are heuristically matched to zbMATH identifiers and may contain data conversion errors. It attempts to reflect the references listed in the original paper as accurately as possible without claiming the completeness or perfect precision of the matching.