×

Reproducible econometric simulations. (English) Zbl 1279.65017

Summary: Reproducibility of economic research has attracted considerable attention in recent years. So far, the discussion has focused mainly on reproducibility of empirical analyses. This paper addresses a further aspect of reproducibility, the reproducibility of computational experiments. More specifically, we contribute to the emerging literature on reproducibility in economics along three lines: (i) we document how simulations of various types are an integral part of publications in modern econometrics, (ii) we provide some general guidelines about how to set up reproducible simulation experiments, and, finally, (iii) we provide a case study from time series econometrics that illustrates the main issues arising in connection with reproducibility, emphasizing the use of modular tools.

MSC:

65C60 Computational problems in statistics (MSC2010)
62P20 Applications of statistics to economics
91B84 Economic time series analysis
PDFBibTeX XMLCite
Full Text: DOI Link

References:

[1] Anderson, R.D., W. H. Greene, Bruce D. McCullough, and H. D. Vinod. 2008. “The Role of Data/Code Archives in the Future of Economic Research.” Journal of Economic Methodology 15: 99-119. · doi:10.1080/13501780801915574
[2] Barr, R.D., B. L. Golden, J. P. Kelly, M. G. C. Resende, and W. R. Stewart, Jr. 1995. “Designing and Reporting on Computational Experiments with Heuristic Methods.” Journal of Heuristics 1: 9-32. · Zbl 0853.68154 · doi:10.1007/BF02430363
[3] Crowder, H., R. S. Dembo, and J. M. Mulvey. 1979. “On Reporting Computational Experiments with Mathematical Software.” ACM Transactions on Mathematical Software 5 (2): 193-203. · doi:10.1145/355826.355833
[4] Flegal, J. M., M. Haran, and G. L. Jones. 2008. “Markov Chain Monte Carlo: Can we Trust the Third Significant Figure?“ Statistical Science 23 (2): 250-260. · Zbl 1327.62017 · doi:10.1214/08-STS257
[5] Gentle, J. E. 2003. Random Number Generation and Monte Carlo Methods. 2nd ed. New York: Springer-Verlag. · Zbl 1028.65004
[6] Hansen, B. E. 1992. “Testing for Parameter Instability in Linear Models.” Journal of Policy Modeling 14: 517-533. · doi:10.1016/0161-8938(92)90019-9
[7] Hansen, B. E. 1997. “Approximate Asymptotic p values for Structural-Change Tests.” Journal of Business & Economic Statistics 15: 60-67.
[8] Hellekalek, P. 1998. “Good Random Number Generators are (not so) Easy to Find.” Mathematics and Computers in Simulation 46: 485-505. · Zbl 0931.65001 · doi:10.1016/S0378-4754(98)00078-0
[9] Hoaglin, D. C., and D. F. Andrews. 1975. “The Reporting of Computation-Based Results in Statistics.” The American Statistician 29: 122-126. · Zbl 0322.62022 · doi:10.2307/2683438
[10] Jackson, R. H. F., P. T. Boggs, S. G. Nash, and S. Powell. 1991. “Guidelines for Reporting Results of Computational Experiments. Report of the Ad Hoc Committee.” Mathematical Programming 49: 413-425.
[11] Koenker, R. 1996. “Reproducible Econometric Research.“ Working paper, Econometrics Lab, University of Illinois at Urbana-Champaign. .
[12] Koenker, R., and A. Zeileis. 2009. \?On Reproducible Econometric Research.“ Journal of Applied Econometrics 24: 833-847. · doi:10.1002/jae.1083
[13] L’Ecuyer, P. 2006. “Random Number Generation.” In Handbooks in Operations Research and Management Science, Vol. 13: Simulation, edited by S. G. Henderson and B. L. Nelson, 55-81. North-Holland, Amsterdam: Elsevier.
[14] Lee, C.-Y., J. Bard, M. Pinedo, and W. E. Wilhelm. 1993. “Guidelines for Reporting Computational Results in IIE Transactions.” IIE Transactions 25 (6): 1-26.
[15] MacKinnon, J. G., A. A. Haug, and L. Michelis. 1999. “Numerical Distribution Functions of Likelihood Ratio Tests for Cointegration.” Journal of Applied Econometrics 14: 563-577. · doi:10.1002/(SICI)1099-1255(199909/10)14:5<563::AID-JAE530>3.0.CO;2-R
[16] Matsumoto, M., and T. Nishimura. 1998. “Mersenne Twister: A 623-Dimensionally Equidistributed Uniform Pseudo-Random Number Generator.” ACM Transactions on Modeling and Computer Simulation 8 (1): 3-30. · Zbl 0917.65005 · doi:10.1145/272991.272995
[17] McCullough, B. D., and H. D. Vinod. 2003. “Verifying the Solution from a Nonlinear Solver: A Case Study.” American Economic Review 93: 873-892. · doi:10.1257/000282803322157133
[18] McCullough, B. D., K. A. McGeary, and T. D. Harrison. 2008. “Do Economics Journal Archives Promote Replicable Research?” Canadian Journal of Economics 41 (4): 1406-1420. · doi:10.1111/j.1540-5982.2008.00509.x
[19] Moon, H. R., and F. Schorfheide. 2009. “Estimation with Overidentifying Inequality Moment Conditions.” Journal of Econometrics 153: 136-154. · Zbl 1431.62126 · doi:10.1016/j.jeconom.2009.05.003
[20] Nyblom, J. 1989. “Testing for the Constancy of Parameters Over Time.” Journal of the American Statistical Association 84: 223-230. · Zbl 0677.62018 · doi:10.2307/2289867
[21] Ploberger, W., and W. Krämer. 1992. “The CUSUM Test with OLS Residuals.” Econometrica 60 (2): 271-285. · Zbl 0744.62155 · doi:10.2307/2951597
[22] R Development Core Team.2012. R: A Language and Environment for Statistical Computing. R Foundation for Statistical Computing, Vienna, Austria, 2012. ISBN 3-900051-07-0. .
[23] Ripley, B. D. 1990. “Thoughts on Pseudorandom Number Generators.” Journal of Computational and Applied Mathematics 31: 153-163. · Zbl 0701.65006 · doi:10.1016/0377-0427(90)90346-2
[24] Santner, T. J., B. J. Williams, and W. I. Notz. 2003. The Design and Analysis of Computer Experiments. New York: Springer-Verlag. · Zbl 1041.62068
[25] Sarkar, D. 2008. lattice: Multivariate Data Visualization with R. New York: Springer-Verlag. · Zbl 1166.62003
[26] Zeileis, A., F. Leisch, K. Hornik, and C. Kleiber. 2002. “strucchange: An R Package for Testing for Structural Change in Linear Regression Models.” Journal of Statistical Software 7 (2): 1-38. http://www.jstatsoft.org/v07/i02/.
This reference list is based on information provided by the publisher or from digital mathematics libraries. Its items are heuristically matched to zbMATH identifiers and may contain data conversion errors. In some cases that data have been complemented/enhanced by data from zbMATH Open. This attempts to reflect the references listed in the original paper as accurately as possible without claiming completeness or a perfect matching.