×

Generalized multiple importance sampling. (English) Zbl 1420.62038

Summary: Importance sampling (IS) methods are broadly used to approximate posterior distributions or their moments. In the standard IS approach, samples are drawn from a single proposal distribution and weighted adequately. However, since the performance in IS depends on the mismatch between the targeted and the proposal distributions, several proposal densities are often employed for the generation of samples. Under this multiple importance sampling (MIS) scenario, extensive literature has addressed the selection and adaptation of the proposal distributions, interpreting the sampling and weighting steps in different ways. In this paper, we establish a novel general framework with sampling and weighting procedures when more than one proposal is available. The new framework encompasses most relevant MIS schemes in the literature, and novel valid schemes appear naturally. All the MIS schemes are compared and ranked in terms of the variance of the associated estimators. Finally, we provide illustrative examples revealing that, even with a good choice of the proposal densities, a careful interpretation of the sampling and weighting procedures can make a significant difference in the performance of the method.

MSC:

62D05 Sampling theory, sample surveys
62F15 Bayesian inference
PDF BibTeX XML Cite
Full Text: DOI arXiv Euclid

References:

[1] Abramowitz, M. and Stegun, I. A., eds. (1992). Handbook of Mathematical Functions with Formulas, Graphs, and Mathematical Tables. Dover, New York. · Zbl 0171.38503
[2] Bugallo, M. F., Elvira, V., Martino, L., Luengo, D., Míguez, J. and Djuric, P. M. (2017). Adaptive importance sampling: The past, the present, and the future. IEEE Signal Process. Mag.34 60-79.
[3] Cappé, O., Guillin, A., Marin, J. M. and Robert, C. P. (2004). Population Monte Carlo. J. Comput. Graph. Statist.13 907-929.
[4] Cappé, O., Douc, R., Guillin, A., Marin, J.-M. and Robert, C. P. (2008). Adaptive importance sampling in general mixture classes. Stat. Comput.18 447-459.
[5] Cornuet, J.-M., Marin, J.-M., Mira, A. and Robert, C. P. (2012). Adaptive multiple importance sampling. Scand. J. Stat.39 798-812. · Zbl 1319.62059
[6] Douc, R. and Cappé, O. (2005). Comparison of resampling schemes for particle filtering. In ISPA 2005. Proceedings of the 4th International Symposium on Image and Signal Processing and Analysis 64-69. IEEE, New York.
[7] Douc, R., Guillin, A., Marin, J.-M. and Robert, C. P. (2007a). Convergence of adaptive mixtures of importance sampling schemes. Ann. Statist.35 420-448. · Zbl 1132.60022
[8] Douc, R., Guillin, A., Marin, J.-M. and Robert, C. P. (2007b). Minimum variance importance sampling via population Monte Carlo. ESAIM Probab. Stat.11 427-447. · Zbl 1181.60028
[9] Elvira, V., Martino, L., Luengo, D. and Bugallo, M. F. (2015). Efficient multiple importance sampling estimators. IEEE Signal Process. Lett.22 1757-1761.
[10] Elvira, V., Martino, L., Luengo, D. and Corander, J. (2015b). A gradient adaptive population importance sampler. In IEEE International Conf. on Acoustics, Speech and Signal Processing (ICASSP) 4075-4079. · Zbl 1394.94827
[11] Elvira, V., Martino, L., Luengo, D. and Bugallo, M. F. (2016). Heretical multiple importance sampling. IEEE Signal Process. Lett.23 1474-1478.
[12] Elvira, V., Martino, L., Luengo, D. and Bugallo, M. F. (2017). Improving Population Monte Carlo: Alternative weighting and resampling schemes. Signal Process.131 77-91.
[13] Geweke, J. (1989). Bayesian inference in econometric models using Monte Carlo integration. Econometrica57 1317-1339. · Zbl 0683.62068
[14] Gordon, N., Salmond, D. and Smith, A. F. M. (1993). Novel approach to nonlinear and non-Gaussian Bayesian state estimation. IEE Proc. F, Commun. Radar Signal Process.140 107-113.
[15] Gwanyama, P. W. (2004). The HM-GM-AM-QM inequalities. College Math. J.35 47-50.
[16] Haario, H., Saksman, E. and Tamminen, J. (1999). Adaptive proposal distribution for random walk Metropolis algorithm. Comput. Statist.14 375-396. · Zbl 0941.62036
[17] Haario, H., Saksman, E. and Tamminen, J. (2001). An adaptive Metropolis algorithm. Bernoulli7 223-242. · Zbl 0989.65004
[18] Hardy, G. H., Littlewood, J. E. and Pólya, G. (1952). Inequalities, 2nd ed. Cambridge Univ. Press, Cambridge.
[19] He, H. Y. and Owen, A. B. (2014). Optimal mixture weights in multiple importance sampling. Preprint. Available at arXiv:1411.3954.
[20] Hesterberg, T. (1995). Weighted average importance sampling and defensive mixture distributions. Technometrics37 185-194. · Zbl 0822.62002
[21] Kahn, H. and Marshall, A. W. (1953). Methods of reducing sample size in Monte Carlo computations. J. Oper. Res. Soc. Am.1 263-278. · Zbl 1414.90373
[22] Kong, A., Liu, J. S. and Wong, W. H. (1994). Sequential imputations and Bayesian missing data problems. J. Amer. Statist. Assoc.9 278-288. · Zbl 0800.62166
[23] Kong, A., McCullagh, P., Meng, X.-L., Nicolae, D. and Tan, Z. (2003). A theory of statistical models for Monte Carlo integration. J. R. Stat. Soc. Ser. B. Stat. Methodol.65 585-618. · Zbl 1067.62054
[24] Liang, F. (2002). Dynamically weighted importance sampling in Monte Carlo computation. J. Amer. Statist. Assoc.97 807-821. · Zbl 1058.65006
[25] Liu, J. S. (2008). Monte Carlo Strategies in Scientific Computing. Springer, New York. · Zbl 1132.65003
[26] Martino, L., Elvira, V., Luengo, D. and Corander, J. (2015a). An adaptive population importance sampler: Learning from uncertainty. IEEE Trans. Signal Process.63 4422-4437. · Zbl 1394.94827
[27] Martino, L., Elvira, V., Luengo, D. and Corander, J. (2017). Layered adaptive importance sampling. Stat. Comput.27 599-623. · Zbl 06737687
[28] Niederreiter, H. (1992). Random Number Generation and Quasi-Monte Carlo Methods. CBMS-NSF Regional Conference Series in Applied Mathematics63. SIAM, Philadelphia, PA. · Zbl 0761.65002
[29] Owen, A. (2013). Monte Carlo Theory, Methods and Examples. Available at http://statweb.stanford.edu/ owen/mc/.
[30] Owen, A. and Zhou, Y. (2000). Safe and effective importance sampling. J. Amer. Statist. Assoc.95 135-143. · Zbl 0998.65003
[31] Robert, C. P. and Casella, G. (2004). Monte Carlo Statistical Methods, 2nd ed. Springer, New York. · Zbl 1096.62003
[32] Tan, Z. (2004). On a likelihood approach for Monte Carlo integration. J. Amer. Statist. Assoc.99 1027-1036. · Zbl 1084.65007
[33] Veach, E. and Guibas, L. (1995). Optimally combining sampling techniques for Monte Carlo rendering. In SIGGRAPH 1995 Proceedings 419-428.
This reference list is based on information provided by the publisher or from digital mathematics libraries. Its items are heuristically matched to zbMATH identifiers and may contain data conversion errors. It attempts to reflect the references listed in the original paper as accurately as possible without claiming the completeness or perfect precision of the matching.