zbMATH — the first resource for mathematics

Modeling publication bias using weighted distributions in a Bayesian framework. (English) Zbl 1042.62525
Summary: Meta-analysis refers to the quantitative synthesis of evidence from a set of related studies. Inference based on naive use of meta-analysis may be erroneous, however, due to publication bias, the tendency of investigators or editors to base decisions regarding submission or acceptance of manuscripts for publication depending on the strength of the investigator’s study findings. Weighted distributions are ideally suited to model this phenomenon, since the weight function is proportional to the probability that the measurement (in this case, the results of a study) gets observed (published). Models induced by several competing weight functions are compared, including one model which does not account for publication bias, using the education data of L. V. Hedges and I. Olkin, Statistical methods for meta-analysis. (1985; Zbl 0666.62002). This allows us to investigate the sensitivity of the overall effect estimates over a range of models. Here, such models are fit hierarchically from a Bayesian perspective using non-informative priors in order to let the data drive the inference. Bayesian calculations are carried out using Markov-chain Monte-Carlo methods such as Gibbs sampling, the Metropolis algorithm, and Monte-Carlo estimation. Several questions of interest are posed, and possible solutions suggested.

62F15 Bayesian inference
65C40 Numerical analysis or methods applied to Markov chains
Zbl 0666.62002
Full Text: DOI
[1] Bayarri, M. J.; Degroot, M. H.: The analysis of published significant results. Tech. report no. 91-21 (1991) · Zbl 0781.62004
[2] Begg, C. B.: A measure to aid in the interpretation of published clinical trials. Stat. med. 4, 1-9 (1985)
[3] Begg, C. B.; Berlin, J. A.: Publication bias: a problem in interpreting medical data (with discussion). J. roy. Stat. soc. Ser. A 151, 419-463 (1988)
[4] Berlin, J. A.; Begg, C. B.; Louis, T. A.: An assessment of publication bias using a sample of published clinical trials. J. amer. Statist. assoc. 84, 381-392 (1989)
[5] Box, G.: Sampling Bayes inference in scientific modeling and robustness. J. roy. Statist. soc. Ser. A 143, 382-430 (1980) · Zbl 0471.62036
[6] Bozarth, J. D.; Roberts, R. R.: Signifying significant significance. Amer. psychol. 27, 774-775 (1972)
[7] British Medical Journal, 1983. The editor regrets .... (Editorial), 280, 508.
[8] Chalmers, I.: The Oxford database of perinatal trials. (1988)
[9] Chalmers, T. C.; Frank, C. S.; Reitman, D.: Minimizing the three stages of publication bias. J. amer. Med. assoc. 263, 1392-1395 (1990)
[10] Chib, S.; Greenberg, E.: Understanding the metropolis-Hastings algorithm. Amer. statist. 49, 327-335 (1995)
[11] Dear, K. B. G.; Begg, C. B.: An approach for assessing publication bias prior to performing a meta-analysis. Stat. sci. 7, 237-245 (1992)
[12] Dickersin, K.: The existence of publication bias and risk factors for its occurrence. J. amer. Med. assoc. 263, 1385-1389 (1990)
[13] Dickersin, K.; Min, Y. I.; Meinert, C. L.: Factors influencing publication of research results. J. amer. Med. assoc. 267, 374-378 (1992)
[14] Easterbrook, P. J.; Berlin, J. A.; Gopalan, R.; Matthews, D. R.: Publication bias in clinical research. Lancet 337, 867-872 (1991)
[15] Geisser, S. W.; Eddy: A predictive approach to model selection. J. amer. Stat. assoc. 74, 153-160 (1979) · Zbl 0401.62036
[16] Gelfand, A. E.; Dey, D. K.; Chang, H.: Model determination using predictive distributions with implementation via sampling-based methods. Bayesian statistics 4, 147-167 (1992)
[17] Gelfand, A. E.; Smith, A. F. M.: Sampling-based approaches to calculating marginal densities. J. amer. Statist. assoc. 85, 398-409 (1990) · Zbl 0702.62020
[18] Gelman, A.; Rubin, D. B.: Inference from iterative simulation using multiple sequences (with discussion). Statist. sci. 7, No. 4, 457-511 (1992) · Zbl 1386.65060
[19] Geman, S.; Geman, D.: Stochastic relaxation, Gibbs distributions, and the Bayesian restoration of images. IEEE trans. Pattern anal. Machine intelligence 6, 721-741 (1984) · Zbl 0573.62030
[20] Geweke, J.: Bayesian inference in econometric models using Monte Carlo integration. Econometrica 57, 1317-1339 (1989) · Zbl 0683.62068
[21] Givens, G. H.; Smith, D. D.; Tweedie, R. L.: Estimating and adjusting for publication bias using data augmentation in Bayesian meta-analysis. Tech. report 95-31 (1995)
[22] Hedges, L. V.: Estimation of effect size under nonrandom sampling: the effects of censoring studies yielding statistically significant mean differences. J. educational statist. 9, 61-85 (1984)
[23] Hedges, L. V.: Modeling publication selection effects in meta-analysis. Statist. sci. 7, 246-255 (1992)
[24] Hedges, L. V.; Olkin, I.: Statistical methods for meta-analysis. (1985) · Zbl 0666.62002
[25] Iyengar, S.; Greenhouse, J. B.: Selection models and the file drawer problem. Statist. sci. 3, 109-135 (1988)
[26] Jeffreys, H.: Theory of probability. (1961) · Zbl 0116.34904
[27] Lane, D. M.; Dunlap, W. P.: Estimating effect size: bias resulting from the significance criterion in editorial decions. Bri. J. Math. statist. Psychol. 31, 107-112 (1978)
[28] Larose, D. T.; Dey, D. K.: Weighted distributions viewed in the context of model selection: a Bayesian perspective. Test 5, No. 1, 227-246 (1996) · Zbl 0852.62030
[29] Light, R. J.; Pillemer, D. B.: Summing up: the science of reviewing research. (1984)
[30] Mahoney, M. J.: Publication prejudices: an experimental study of confirmatory bias in the peer review system. Cogn. therapy res. 1, 161-175 (1977)
[31] Melton, A. W.: Editorial. J. exp. Psychol. 64, 553-557 (1962)
[32] Mengersen, K. L.; Tweedie, R. L.; Biggerstaff, B. J.: The impact of method choice in meta-analysis. Aust. J. Stat. 7, 19-44 (1995)
[33] Metropolis, N.; Rosenbluth, A. W.; Rosenbluth, M. N.; Teller, A. H.; Teller, E.: Equation of state calculations by fast computing machines. J. chem. Phys. 21, 1087-1092 (1953)
[34] Morris, C.; Normand, S. L.: Hierarchical models for combining information and for meta-analyses (with discussion). Bayesian statistics 4, 3121-3134 (1992)
[35] Normand, S. L.: Random effects methods for dose-response meta-analyses. Manuscript (1994)
[36] Olkin, I.: Publication bias statist. Sci.. 7, No. 2, 235-236 (1992)
[37] Patil, G. P.; Taillie, C.: Probing encountered data, meta-analysis and weighted distribution methods. Statistical data analysis and inference, 317-345 (1989) · Zbl 0743.62002
[38] Pettit, L. I.; Young, K. D. S.: Measuring the effect of observations on Bayes factors. Biomet. 77, 455-466 (1990)
[39] Rao, C. R.: Weighted distributions arising out of methods of ascertainment: what population does a sample represent?. A celebration of statistics: the ISI centenary vol, 543-569 (1985) · Zbl 0603.62013
[40] Raftery, A. E.: Hypothesis testing and model selection via posterior simulation. Manuscript (1994)
[41] Rosenthal, R.: The file drawer problem and tolerance for null results. Psychol. bull. 86, 638-641 (1979)
[42] Simes, R. J.: Publication bias: the case for an international registry of clinical trials. J. clin. Oncol. 4, 1529-1541 (1986)
[43] Smith, D. D.; Givens, G. H.; Tweedie, R. L.: Adjustment for publication and quality bias in Bayesian meta-analysis. Tech. report (1997)
[44] Sterling, T. D.: Publication decisions and their possible effects on inferences drawn from tests of significance or vice versa. J. amer. Statist. assoc. 54, 30-34 (1959)
[45] Sterling, T. D.; Rosenbaum, W. L.; Weinkam, J. J.: Publication decisions revisited: the effect of the outcome of statistical tests on the decision to publish and vice versa. Amer. statistician 49, 108-112 (1995)
[46] Thompson, S. G.: Controversies in meta-analysis: the case of the trials of serum cholesterol reduction. Statist. meth. Med. res. 2, 173-192 (1993)
This reference list is based on information provided by the publisher or from digital mathematics libraries. Its items are heuristically matched to zbMATH identifiers and may contain data conversion errors. It attempts to reflect the references listed in the original paper as accurately as possible without claiming the completeness or perfect precision of the matching.