Meng, Xiao-Li From unit root to Stein’s estimator to Fisher’s \(k\) statistics: If you have a moment, I can tell you more. (English) Zbl 1086.62001 Stat. Sci. 20, No. 2, 141-162 (2005). Summary: Any general textbook that discusses moment generating functions (MGFs) shows how to obtain a moment of positive-integer order via differentiation, although usually the presented examples are only illustrative, because the corresponding moments can be calculated in more direct ways. It is thus somewhat unfortunate that very few textbooks discuss the use of MGFs when it becomes the simplest, and sometimes the only, approach for analytic calculation and manipulation of moments. Such situations arise when we need to evaluate the moments of ratios and logarithms, two of the most common transformations in statistics. Such moments can be obtained by differentiating and integrating a joint MGF of the underlying untransformed random variables in appropriate ways. These techniques are examples of multivariate Laplace transform methods and can also be derived from the fact that moments of negative orders can be obtained by integrating an MGF. This article reviews, extends and corrects various results scattered in the literature on this joint-MGF approach, and provides four applications of independent interest to demonstrate its power and beauty. The first application, which motivated this article, is for the exact calculation of the moments of a well-known limiting distribution under the unit-root AR(1) model. The second, which builds on S. M. Stigler’s [Stat. Sci. 5, 147–155 (1990)] Galtonian perspective, reveals a straightforward, non-Bayesian constructive derivation of the Stein estimator, as well as convenient expressions for studying its risk and bias. The third finds an exceedingly simple bound for the bias of a sample correlation from a bivariate normal population, namely the magnitude of the relative bias is not just of order \(n^{-1}\), but actually is bounded above by \(n^{-1}\) for all sample sizes \(n\geq 2\). The fourth tackles the otherwise intractable problem of studying the finite-sample optimal bridge in the context of bridge sampling for computing normalizing constants. A by-product of the joint-MGF approach is that positive-order fractional moments can be easily obtained from an MGF without invoking the concept of fractional differentiation, a method used by R. A. Fisher in his study of \(k\) statistics 45 years before it reappeared in the probability literature. Cited in 10 Documents MSC: 62A01 Foundations and philosophical topics in statistics 62-03 History of statistics 44A10 Laplace transform 01A60 History of mathematics in the 20th century 60E10 Characteristic functions; other transforms 62M10 Time series, auto-correlation, regression, etc. in statistics (GARCH) Keywords:AR(1) model; bias; bridge sampling; Efron-Morris estimator; fractional derivative; history of statistics; James-Stein estimator; Laplace transform; normalizing constant; R. A. Fisher; unit root; Wiener process Software:tsbridge × Cite Format Result Cite Review PDF Full Text: DOI References: [1] Abadir, K. M. (1993). The limiting distribution of the autocorrelation coefficient under a unit root. Ann. Statist. 21 1058–1070. · Zbl 0778.62014 · doi:10.1214/aos/1176349164 [2] Abadir, K. M. and Larsson, R. (1996). The joint moment generating function of quadratic forms in multivariate autoregressive series. Econometric Theory 12 682–704. JSTOR: [3] Abadir, K. M. and Larsson, R. (2001). The joint moment generating function of quadratic forms in multivariate autoregressive series: The case with deterministic components. Econometric Theory 17 222–246. JSTOR: · Zbl 0976.62084 · doi:10.1017/S0266466601171070 [4] Bennett, C. H. (1976). Efficient estimation of free energy differences from Monte Carlo data. J. Computational Phys. 22 245–268. · doi:10.1016/0021-9991(76)90078-4 [5] Billingsley, P. (1995). Probability and Measure , 3rd ed. Wiley, New York. · Zbl 0822.60002 [6] Bock, M. E., Judge, G. G. and Yancey, T. A. (1984). A simple form for the inverse moments of non-central \(\chi^2\) and \(F\) random variables and certain confluent hypergeometric functions. J. Econometrics 25 217–234. · Zbl 0559.62011 · doi:10.1016/0304-4076(84)90048-4 [7] Bowman, K. O. and Shenton, L. R. (1992). Some exact expressions for the mean and higher moments of functions of sample moments. Ann. Inst. Statist. Math. 44 781–798. · Zbl 0772.62007 · doi:10.1007/BF00053406 [8] Brandwein, A. C. and Strawderman, W. E. (1990). Stein estimation: The spherically symmetric case. Statist. Sci. 5 356–369. · Zbl 0955.62611 · doi:10.1214/ss/1177012104 [9] Ceperley, D. M. (1995). Path integrals in the theory of condensed helium. Rev. Modern Phys. 67 279–355. [10] Chan, N. H. and Wei, C. Z. (1987). Asymptotic inference for nearly nonstationary AR(1) processes. Ann. Statist. 15 1050–1063. · Zbl 0638.62082 · doi:10.1214/aos/1176350492 [11] Chao, M. T. and Strawderman, W. E. (1972). Negative moments of positive random variables. J. Amer. Statist. Assoc. 67 429–431. · Zbl 0238.60008 · doi:10.2307/2284399 [12] Cressie, N. and Borkent, M. (1986). The moment generating function has its moments. J. Statist. Plann. Inference 13 337–344. · Zbl 0592.60009 · doi:10.1016/0378-3758(86)90143-6 [13] Cressie, N., Davis, A. S., Folks, J. L. and Policello, G. E. (1981). The moment-generating function and negative integer moments. Amer. Statist. 35 148–150. JSTOR: · Zbl 0474.60015 · doi:10.2307/2683982 [14] Davies, N., Pate, M. B. and Petruccelli, J. D. (1985). Exact moments of the sample cross correlations of multivariate autoregressive moving average time series. Sankhyā Ser. B 47 325–337. · Zbl 0593.62090 [15] De Gooijer, J. G. (1980). Exact moments of the sample autocorrelations from series generated by general ARIMA processes of order \((p, d, q)\), \(d=0\) or 1. J. Econometrics 14 365–379. · Zbl 0457.62075 · doi:10.1016/0304-4076(80)90033-0 [16] DiCiccio, T. J., Kass, R. E., Raftery, A. and Wasserman, L. (1997). Computing Bayes factors by combining simulation and asymptotic approximations. J. Amer. Statist. Assoc. 92 903–915. JSTOR: · Zbl 1050.62520 · doi:10.2307/2965554 [17] Dickey, D. A. and Fuller, W. A. (1979). Distribution of the estimators for autoregressive time series with a unit root. J. Amer. Statist. Assoc. 74 427–431. · Zbl 0413.62075 · doi:10.2307/2286348 [18] Efron, B. and Morris, C. N. (1973). Stein’s estimation rule and its competitors—An empirical Bayes approach. J. Amer. Statist. Assoc. 68 117–130. JSTOR: · Zbl 0275.62005 · doi:10.2307/2284155 [19] Elliott, G., Rothenberg, T. and Stock, J. H. (1996). Efficient tests for an autoregression unit root. Econometrica 64 813–836. JSTOR: · Zbl 0888.62088 · doi:10.2307/2171846 [20] Evans, G. B. A. and Savin, N. E. (1981). Testing for unit roots. I. Econometrica 49 753–779. JSTOR: · Zbl 0468.62021 · doi:10.2307/1911521 [21] Evans, G. B. A. and Savin, N. E. (1984). Testing for unit roots. II. Econometrica 52 1241–1269. JSTOR: · Zbl 0579.62014 · doi:10.2307/1910998 [22] Feller, W. (1971). An Introduction to Probability Theory and Its Applications 2 , 2nd ed. Wiley, New York. · Zbl 0219.60003 [23] Fisher, R. A. (1915). Frequency distribution of the values of the correlation coefficient in samples from an indefinitely large population. Biometrika 10 507–521. · Zbl 0070.37304 [24] Fisher, R. A. (1929). Moments and product moments of sampling distributions. Proc. London Math. Soc. (2 ) 30 199–238. · JFM 55.0924.01 [25] Fisher, R. A. (1930). The moments of the distribution for normal samples of measures of departure from normality. Proc. Roy. Soc. London Ser. A 130 16–28. · JFM 56.1106.16 [26] From, S. G. and Saxena, K. M. L. (1989). Estimating parameters from mixed samples using sample fractional moments. J. Statist. Plann. Inference 21 231–244. · Zbl 0666.62025 · doi:10.1016/0378-3758(89)90007-4 [27] Gelman, A. and Meng, X.-L. (1998). Simulating normalizing constants: From importance sampling to bridge sampling to path sampling. Statist. Sci. 13 163–185. · Zbl 0966.65004 · doi:10.1214/ss/1028905934 [28] Gonzalo, J. and Pitarakis, J. (1998). On the exact moments of asymptotic distributions in an unstable AR(1) with dependent errors. Internat. Econom. Rev. 39 71–88. [29] Gradshteyn, I. S. and Ryzhik, I. M. (1992). Table of Integrals , Series , and Products , corrected and enlarged ed. Academic Press, San Diego, CA. · Zbl 0918.65001 [30] Hoque, A. (1985). The exact moments of forecast error in the general dynamic model. Sankhyā Ser. B 47 128–143. · Zbl 0596.62094 [31] Hotelling, H. (1953). New light on the correlation coefficient and its transforms (with discussion). J. Roy. Statist. Soc. Ser. B 15 193–232. JSTOR: · Zbl 0052.14905 [32] James, W. and Stein, C. (1961). Estimation with quadratic loss. Proc. Fourth Berkeley Symp. Math. Statist. Probab. 1 361–379. Univ. California Press, Berkeley. · Zbl 1281.62026 [33] Johnson, N. L., Kotz, S. and Balakrishnan, N. (1995). Continuous Univariate Distributions 2 , 2nd ed. Wiley, New York. · Zbl 0821.62001 [34] Johnson, N. L., Kotz, S. and Kemp, A. W. (1992). Univariate Discrete Distributions , 2nd ed. Wiley, New York. · Zbl 0773.62007 [35] Johnson, P. D., Jr. (1975). An algebraic definition of fractional differentiation. Fractional Calculus and Its Applications. Lecture Notes in Math. 457 226–231. Springer, Berlin. · Zbl 0339.26014 [36] Jones, M. C. (1986). Expressions for inverse moments of positive quadratic forms in normal variables. Austral. J. Statist. 28 242–250. · Zbl 0616.62070 · doi:10.1111/j.1467-842X.1986.tb00604.x [37] Jones, M. C. (1987a). Inverse factorial moments. Statist. Probab. Lett. 6 37–42. Correction 6 369. · Zbl 0633.60022 · doi:10.1016/0167-7152(87)90056-3 [38] Jones, M. C. (1987b). On moments of ratios of quadratic forms in normal variables. Statist. Probab. Lett. 6 129–136. Correction 6 369. · Zbl 0645.62054 · doi:10.1016/0167-7152(87)90086-1 [39] Khuri, A. and Casella, G. (2002). The existence of the first negative moment revisited. Amer. Statist. 56 44–47. JSTOR: · Zbl 1182.62020 · doi:10.1198/000313002753631358 [40] Kong, A., McCullagh, P., Meng, X.-L., Nicolae, D. and Tan, Z. (2003). A theory of statistical models for Monte Carlo integration (with discussion). J. R. Stat. Soc. Ser. B Stat. Methodol. 65 585–618. JSTOR: · Zbl 1067.62054 · doi:10.1111/1467-9868.00404 [41] Laue, G. (1980). Remarks on the relation between fractional moments and fractional derivatives of characteristic functions. J. Appl. Probab. 17 456–466. JSTOR: · Zbl 0428.60026 · doi:10.2307/3213035 [42] Lin, J. L. (2003). Discussion of “From unit root to Stein’s estimator to Fisher’s \(k\) statistics: If you have a moment, I can tell you more,” by X.-L. Meng. Presented at the NSF-NBER Time-Series Conference, Chicago, September 19–20, 2003. [43] Maatta, J. M. and Casella, G. (1990). Developments in decision-theoretic variance estimation (with discussion). Statist. Sci. 5 90–120. · Zbl 0955.62529 [44] Mathai, A. M. (1991). On fractional moments of quadratic expressions in normal variables. Comm. Statist. Theory Methods 20 3159–3174. · Zbl 0800.62260 · doi:10.1080/03610929108830694 [45] Mehta, J. S. and Swamy, P. A. V. B. (1978). The existence of moments of some simple Bayes estimators of coefficients in a simultaneous equation model. J. Econometrics 7 1–13. · Zbl 0384.62028 · doi:10.1016/0304-4076(78)90002-7 [46] Meng, X.-L. and Schilling, S. (1996). Fitting full-information item factor models and an empirical investigation of bridge sampling. J. Amer. Statist. Assoc. 91 1254–1267. · Zbl 0925.62220 · doi:10.2307/2291744 [47] Meng, X.-L. and Schilling, S. (2002). Warp bridge sampling. J. Comput. Graph. Statist. 11 552–586. JSTOR: · doi:10.1198/106186002457 [48] Meng, X.-L. and Wong, W. H. (1996). Simulating ratios of normalizing constants via a simple identity: A theoretical exploration. Statist. Sinica 6 831–860. · Zbl 0857.62017 [49] Morin, D. (1992). Exact moments of ratios of quadratic forms. Metron 30 59–78. · Zbl 0785.62056 [50] Nankervis, J. C. and Savin, N. E. (1988). The exact moments of the least-squares estimator for the autoregressive model: Corrections and extensions. J. Econometrics 37 381–388. · doi:10.1016/0304-4076(88)90012-7 [51] Neveu, J. (1965). Mathematical Foundations of the Calculus of Probability . Holden-Day, San Francisco. · Zbl 0137.11301 [52] Nielsen, B. (1997). Bartlett correction of the unit root test in autoregressive models. Biometrika 84 500–504. · Zbl 0882.62083 · doi:10.1093/biomet/84.2.500 [53] Olkin, I. and Pratt, J. W. (1958). Unbiased estimation of certain correlation coefficients. Ann. Math. Statist. 29 201–211. · Zbl 0094.14403 · doi:10.1214/aoms/1177706717 [54] Ott, J. (1979). Maximum likelihood estimation by counting methods under polygenic and mixed models in human pedigrees. American J. Human Genetics 31 161–175. [55] Peters, T. A. (1989). The exact moments of OLS in dynamic regression models with nonnormal errors. J. Econometrics 40 279–305. · Zbl 0677.62086 · doi:10.1016/0304-4076(89)90086-9 [56] Piegorsch, W. W. and Casella, G. (1985). The existence of the first negative moment. Amer. Statist. 39 60–62. Comments by N. L. Johnson, 39 240 and J. Hannan, 39 326. JSTOR: · doi:10.2307/2683910 [57] Pitarakis, J. (1998). Moment generating functions and further exact results for seasonal autoregressions. Econometric Theory 14 770–782. JSTOR: · doi:10.1017/S0266466698146030 [58] Provost, S. B. and Rudiuk, E. M. (1994). The exact density function of the ratio of two dependent linear combinations of chi-square variables. Ann. Inst. Statist. Math. 46 557–571. · Zbl 0817.62005 [59] Rao, M. M. (1978). Asymptotic distribution of an estimator of the boundary parameter of an unstable process. Ann. Statist. 6 185–190. Correction 8 1403. · Zbl 0378.62018 · doi:10.1214/aos/1176344077 [60] Romero, M. (2003). On two topics with no bridge: Bridge sampling with dependent draws and bias of the multiple imputation variance estimator. Ph.D dissertation, Dept. Statistics, Univ. Chicago. [61] Ross, B. (1975). A brief history and exposition of the fundamental theory of fractional calculus. Fractional Calculus and Its Applications . Lecture Notes in Math. 457 1–36. Springer, Berlin. · Zbl 0293.00010 [62] Sawa, T. (1972). Finite sample properties of the \(k\)-class estimators. Econometrica 40 653–680. JSTOR: · Zbl 0258.62067 · doi:10.2307/1912960 [63] Sawa, T. (1978). The exact moments of the least squares estimator for the autoregressive model. J. Econometrics 8 159–172. · Zbl 0394.62069 · doi:10.1016/0304-4076(78)90025-8 [64] Servidea, J. (2002). Bridge sampling with dependent random draws: Techniques and strategy. Ph.D. dissertation, Dept. Statistics, Univ. Chicago. [65] Shepp, L. A. and Lloyd, S. P. (1966). Ordered cycle lengths in a random permutation. Trans. Amer. Math. Soc. 121 340–357. JSTOR: · Zbl 0156.18705 · doi:10.2307/1994483 [66] Springer, M. D. (1979). The Algebra of Random Variables. Wiley, New York. · Zbl 0399.60002 [67] Stein, C. (1981). Estimation of the mean of a multivariate normal distribution. Ann. Statist. 9 1135–1151. · Zbl 0476.62035 · doi:10.1214/aos/1176345632 [68] Stigler, S. M. (1990). The 1988 Neyman memorial lecture: A Galtonian perspective on shrinkage estimators. Statist. Sci. 5 147–155. · Zbl 0955.62610 [69] Stuart, A. and Ord, J. K. (1987). Kendall ’ s Advanced Theory of Statistics 1 . Distribution Theory , 5th ed. Oxford Univ. Press, London. · Zbl 0621.62001 [70] Tanaka, K. (1996). Time Series Analysis : Nonstationary and Noninvertible Distribution Theory. Wiley, New York. · Zbl 0861.62062 [71] Tsui, A. K. and Ali, M. M. (1994). Exact distributions, density functions and moments of the least squares estimator in a first-order autoregressive model. Comput. Statist. Data Anal. 17 433–454. · Zbl 0937.62638 · doi:10.1016/0167-9473(94)90022-1 [72] Voter, A. F. (1985). A Monte Carlo method for determining free-energy differences and transition state theory rate constants. J. Chemical Physics 82 1890–1899. [73] White, J. S. (1958). The limiting distribution of the serial correlation coefficient in the explosive case. Ann. Math. Statist. 29 1188–1197. · Zbl 0099.13004 · doi:10.1214/aoms/1177706450 [74] White, J. S. (1959). The limiting distribution of the serial correlation coefficient in the explosive case. II. Ann. Math. Statist. 30 831–834. · Zbl 0133.42403 · doi:10.1214/aoms/1177706213 [75] Williams, J. D. (1941). Moments of the ratio of the mean square successive difference to the mean square difference in samples from a normal universe. Ann. Math. Statist. 12 239–241. · Zbl 0025.20002 · doi:10.1214/aoms/1177731756 [76] Wolfe, S. J. (1975). On moments of probability distribution functions. Fractional Calculus and Its Applications . Lecture Notes in Math. 457 306–316. Springer, Berlin. · Zbl 0306.60009 This reference list is based on information provided by the publisher or from digital mathematics libraries. Its items are heuristically matched to zbMATH identifiers and may contain data conversion errors. In some cases that data have been complemented/enhanced by data from zbMATH Open. This attempts to reflect the references listed in the original paper as accurately as possible without claiming completeness or a perfect matching.