×

Linear algebra and multivariate analysis in statistics: development and interconnections in the twentieth century. (English) Zbl 1501.01006

This article discusses the mutual influences of linear algebra and multivariate analysis on each other. The authors first sketch the development of matrix algebra (very briefly; Cayley, Jacobi, Hamilton and Gibbs are mentioned) and then discuss the development of multivariate analysis (Pearson, Wishart, Fisher, Kendall). Then they describe several results from linear algebra with applications in multivariate analysis, namely Schur complements, his determinant lemma, and the Banachiewicz inversion formula. For more recent developments, the authors mainly provide references.

MSC:

01A60 History of mathematics in the 20th century
62-03 History of statistics
PDFBibTeX XMLCite
Full Text: DOI

References:

[1] Aitken, A. C., Note on selection from a multivariate normal population, Proceedings of the Edinburgh Mathematical Society, 4, 106-110 (1935) · Zbl 0010.40606
[2] Aitken, A. C., A further note on multivariate selection, Proceedings of the Edinburgh Mathematical Society, 5, 37-40 (1936) · Zbl 0015.22004
[3] Aitken, A. C., Determinants and matrices (1939), Edinburgh: Oliver & Boyd, Edinburgh · Zbl 0022.10005
[4] Aitken, A. C., Statistical mathematics (1939), Edinburgh: Oliver & Boyd, Edinburgh · Zbl 0022.24804
[5] Aldous, D. J., Random walks on finite groups and rapidly mixing Markov chains, Séminaire De Probabilitée, 17, 243-297 (1983) · Zbl 0514.60067
[6] Aldous, D. J.; Diaconis, P., Shuffling cards and stopping times, American Mathematical Monthly, 93, 333-348 (1986) · Zbl 0603.60006
[7] Aldous, D. J.; Diaconis, P., Longest increasing subsequences: from patience to the Baik-Deift-Johansson theorem, Bulletin of the American Mathematical Society, 36, 413-432 (1999) · Zbl 0937.60001
[8] Amari, S-I, Information geometry and its applications (2016), New York: Springer, New York · Zbl 1350.94001
[9] Anderson, T. W., An introduction to multivariate statistical analysis (1958), New York: Wiley, New York · Zbl 0083.14601
[10] Anderson, T. W., Samuel Stanley Wilks, 1906-1964, Annals of Mathematical Statistics, 36, 1-23 (1965)
[11] Barndorff-Nielsen, O. E.; Schou, G., On the parametrization of autoregressive models by partial autocorrelation, Journal of Multivariate Analysis, 3, 408-419 (1973) · Zbl 0275.62074
[12] Bartholomew, D. J.; Knott, M., Latent variable models and factor analysis (1999), London: Arnold, London · Zbl 1066.62528
[13] Bartlett, M. S., Further aspects of the theory of multiple regression, Proceedings of the Cambridge Philosophical Society, 34, 33-40 (1938) · JFM 64.0547.05
[14] Bartlett, M. S., Multivariate analysis, Journal of the Royal Statistical Society Series B, 10, 176-197 (1947) · Zbl 0030.40403
[15] Bartlett, M. S., An inverse matrix adjustment arising in discriminant analysis, Annals of Mathematical Statistics, 22, 107-111 (1951) · Zbl 0042.38203
[16] Bartlett, M. S.; Whittaker, J. M., Obituary, A. C. Aitken, Biographical Memoirs of Fellows of the Royal Society, 14, 1-14 (1968)
[17] Bingham, N. H., Szegő’s theorem and its probabilistic descendants, Probability Surveys, 9, 287-324 (2012) · Zbl 1285.60037
[18] Bingham, N. H.; Fry, J. M., Regression linear models in statistics (2010), Heidelberg: Springer, Heidelberg · Zbl 1245.62085
[19] Bingham, N H, and Symons, T L, ‘Gaussian random fields: with and without covariances’, Theory of Probability and Mathematical Statistics (Special Issue in memory of M I Yadrenko, ed A Olenko), to appear, 2022; arXiv:2111.11960.
[20] Birkhoff, G.; MacLane, S., A survey of modern algebra (1953), New York: Macmillan, New York · Zbl 0052.25402
[21] Blower, G., Random matrices: high-dimensional phenomena (2009), Cambridge: Cambridge University Press, Cambridge · Zbl 1278.60003
[22] Bougerol, P.; Lacroix, J., Products of random matrices with applications to Schrödinger operators (1985), Basel: Birkhäuser, Basel · Zbl 0572.60001
[23] Box, J. F., Fisher, R. A.: The life of a scientist (1978), New York: Wiley, New York
[24] Brechenmacher, F., Une histoire de l’universalité des matrices mathématiques, Revue de Synthèse, 131, 569-603 (2010) · Zbl 1225.15003
[25] Bryan, J. G., The generalized discriminant function: mathematical foundation and computational routine, Harvard Educational Review, 21, 90-95 (1951)
[26] Burkill, H.; Ledermann, W.; Hooley, C.; Perfect, H., Obituary, Leon Mirsky, Bulletin of the London Mathematical Society, 18, 1945-206 (1986) · Zbl 0581.01019
[27] Cox, D. R.; Wermuth, N., Multivariate dependencies: models, analysis and interpretation (1996), Boca Raton: Chapman & Hall/CRC, Boca Raton · Zbl 0880.62124
[28] Cox, D. R.; Wermuth, N., Causality: a statistical view, International Statistical Review, 72, 285-305 (2004)
[29] Cramér, H., Random variables and probability distributions (1937), Cambridge: Cambridge University Press, Cambridge · JFM 63.1080.01
[30] Cramér, H., Mathematical methods of statistics (1946), Princeton: Princeton University Press, Princeton · Zbl 0063.01014
[31] Curnow, R. N., Applied statistics at the university of reading: the first forty years (2006), Reading: University of Reading, Reading
[32] Dawid, A. P., Conditional independence in statistical theory (with discussion), Journal of the Royal Statistical Society Series B, 41, 1-31 (1979) · Zbl 0408.62004
[33] Dempster, A. P., Elements of continuous multivariate analysis (1969), Reading, MA: Addison-Wesley, Reading, MA · Zbl 0197.44904
[34] Dempster, A. P., Covariance selection, Biometrics, 28, 157-175 (1972)
[35] Denison, D. G T.; Holmes, C. C.; Mallick, B. K.; Smith, A. F M., Bayesian methods for nonlinear classification and regression (2002), New York: Wiley, New York · Zbl 0994.62019
[36] Eaton, M. L., Multivariate statistics: a vector space approach (1983), Wiley · Zbl 0587.62097
[37] Eaton, M. L., Group invariance in statistics (1989), Inst. Math. Stat · Zbl 0749.62005
[38] Eckart, C.; Young, G., The approximation of one matrix by another of lower rank, Psychometrika, 1, 211-218 (1936) · JFM 62.1075.02
[39] Edgeworth, F. Y., Note on the calculation of correlation between organs, Philosophical Magazine (Series 5), 36, 350-351 (1893) · JFM 25.0356.03
[40] Einstein, A., Die grundlage der allgemeinen relativitätstheorie, Annalen der Physik (4), 49, 770-822 (1916) · JFM 46.1293.01
[41] Eisenhart, C, ‘Boscovich and the combination of observations’, in L L Whyte (ed), Roger Joseph Boscovich, Allen & Unwin, 1961, 200-212.
[42] Farebrother, R W, Fitting linear relationships: a history of the calculus of observations, Springer, 1999, 1750-1900. · Zbl 0934.62003
[43] Fisher, R. A., Applications of vector analysis to geometry, Messenger of Mathematica, 42, 161-178 (1913) · JFM 44.0686.02
[44] Fisher, R. A., The use of multiple measurements in taxonomic problems, Annals of Eugenics, 7, 179-198 (1936)
[45] Gantmacher, F. H., The theory of matrices, 1, 2 (1959), New York: Chelsea, New York · Zbl 0085.01001
[46] Golub, G. H.; Van Loan, C. F., Matrix computations (1996), Baltimore: Johns Hopkins University Press, Baltimore · Zbl 0865.65009
[47] Good, I. J., Some applications of the singular decomposition of a matrix, Technometrics, 11, 823-831 (1969) · Zbl 0186.33803
[48] Gower, J. C., Some distance properties of latent root and vector methods used in multivariate analysis, Biometrika, 53, 325-338 (1966) · Zbl 0192.26003
[49] Guivarc’h, Y, ‘Quelques propriétés asymptotiques des produits des matrices aléatoires’, in Ecole d’Eté de Probabilités de Saint-Flour VIII-1978 (Lecture Notes in Mathematics, 774), New York: Springer, 1980, 177-234. · Zbl 0433.60007
[50] Hand, D. J., Discrimination and classification (1981), New York: Wiley, New York · Zbl 0587.62119
[51] Hand, D. J., Construction and assessment of classification rules (1997), New York: Wiley, New York · Zbl 0997.62500
[52] Harville, D. A., Matrix algebra from a statistician’s perspective (1997), Heidelberg: Springer, Heidelberg · Zbl 0881.15001
[53] Hawkins, T, ‘The theory of matrices in the 19th century’, in Proceedings of the International Congress of Mathematics Vancouver 1974, vol 2, 1975a, 561-570. · Zbl 0438.15004
[54] Hawkins, T., Cauchy and the spectral theory of matrices, Historia Mathematica, 2, 1-29 (1975) · Zbl 0296.01014
[55] Horn, R. A.; Johnson, C. R., Matrix analysis (1985), Cambridge: Cambridge University Press, Cambridge · Zbl 0576.15001
[56] Horn, R. A.; Johnson, C. R., Topics in matrix analysis (1991), Cambridge: Cambridge University Press, Cambridge · Zbl 0729.15001
[57] Hotelling, H., The generalization of student’s ratio, Annals of Mathematical Statistics, 2, 360-378 (1931) · Zbl 0004.26503
[58] Hotelling, H., Analysis of a complex of statistical variables into principal components, Journal of Educational Psychology, 24, 417-441 (1933) · JFM 59.1182.04
[59] Hotelling, H., Relations between two sets of variates, Biometrika, 28, 321-377 (1936) · Zbl 0015.40705
[60] Katz, NM; Sarnak, PC, Zeros of zeta functions and symmetry, Bulletin of the American Mathematical Society, 36, 1-26 (1999) · Zbl 0921.11047
[61] Katz, N. M.; Sarnak, P. C., Random matrices, Frobenius eigenvalues and monodromy (1999), Providence: American Mathematical Society, Providence · Zbl 0958.11004
[62] Keating, J. P.; Snaith, N. C., Random matrices and L-functions, Journal of Physics A: Mathematical and General, 36, 2859-2881 (2003) · Zbl 1074.11053
[63] Kendall, M. G., A course in multivariate analysis (1957), London: Griffin, London
[64] Kendall, M. G.; Stuart, A., The advanced theory of statistics, volume 1, distribution theory (1977), London: Griffin, London · Zbl 0353.62013
[65] Kendall, M. G.; Stuart, A., The advanced theory of statistics, volume 2, inference and relationship (1979), London: Griffin, London · Zbl 0416.62001
[66] Kruskal, J. B., Multidimensional scaling by optimising goodness-of-fit to a non-parametric hypothesis, Psychometrika, 29, 1-27 (1964) · Zbl 0123.36803
[67] Kruskal, J. B., Non-metric multidimensional scaling: a numerical method, Psychometrika, 29, 115-129 (1964) · Zbl 0123.36804
[68] Krzanowski, W. J., The algebraic basis of classical multivariate methods, The Statistician, 20, 51-61 (1971)
[69] Krzanowski, W. J., Principles of multivariate analysis: a user’s perspective (1988), Oxford: Oxford University Press, Oxford · Zbl 0678.62001
[70] Lachenbruch, P. A.; Mickey, M. R., Estimation of error rates in discriminant analysis, Technometrics, 10, 1-11 (1968)
[71] Landsberg, J. M., Tensors: geometry and applications (2012), Providence: American Mathematical Society, Providence · Zbl 1238.15013
[72] Lauritzen, S. L., Graphical models (1996), Oxford: Oxford University Press, Oxford · Zbl 0907.62001
[73] Lawley, D. N., The estimation of factor loadings by the method of maximum likelihood, Proceedings of the Royal Society of Edinburgh, Section A, 60, 64-82 (1940) · JFM 66.0653.05
[74] Lawley, D. N., Further investigations in factor estimation, Proceedings of the Royal Society of Edinburgh, Section A, 61, 176-185 (1941) · Zbl 0063.03459
[75] Lawley, D. N., A note on Karl Pearson’s selection formulae, Proceedings of the Royal Society of Edinburgh, Section A, 62, 28-30 (1942) · Zbl 0060.31112
[76] Lawley, D. N.; Maxwell, E. A., Factor analysis as a statistical method (1971), New York: American Elsevier, New York · Zbl 0251.62042
[77] Ledermann, W., Sampling distribution and selection in a normal population, Biometrika, 30, 295-304 (1939) · Zbl 0020.15002
[78] Ledrappier, F, ‘Quelques propriétés des exposants characteristiques’, in Ecole d’Eté de Probabilités de Saint-Flour XII-1982 (Lecture Notes in Mathematics, 1097), Heidelberg: Springer, 1984, 305-396. · Zbl 0578.60029
[79] Lehmann, E. H., Theory of point estimation (1983), New York: Wiley, New York · Zbl 0522.62020
[80] Magnus, J. R.; Neudecker, H., Matrix differential calculus (1988), New York: Wiley, New York
[81] Mahalanobis, P. C., On the generalized distance in statistics, Proceedings of the National Institute of Science, India, 2, 49-55 (1936) · Zbl 0015.03302
[82] Mardia, K. V.; Kent, J. T.; Bibby, J. M., Multivariate analysis (1979), London: Academic Press, London · Zbl 0432.62029
[83] McCullagh, P., Tensor methods in statistics (2017), Boca Raton: Chapman & Hall/CRC, Boca Raton
[84] McLachlan, G. J., Discriminant analysis and statistical pattern recognition (1992), New York: Wiley, New York · Zbl 0850.62481
[85] Markoff, A. A., Wahrscheinlichkeitsrechnung (1912), Leipzig: B G Teubner, Leipzig · JFM 43.0291.01
[86] Mehta, M. L., Random matrices and the statistical theory of energy levels (1967), London: Academic Press, London · Zbl 0925.60011
[87] Mirsky, L., An introduction to linear algebra (1955), Oxford: Oxford University Press, Oxford · Zbl 0066.26305
[88] Muir, T., The theory of determinants in the historical order of development, 1 (1906), London: Macmillan, London · JFM 45.1242.04
[89] Pastur, L.; Shcherbina, M., Eigenvalue distributions of large random matrices (2011), Providence: American Mathematical Society, Providence · Zbl 1244.15002
[90] Pearl, J., Causality: models, reasoning and inference (2009), Cambridge: Cambridge University Press, Cambridge · Zbl 1188.68291
[91] Pearl, J., Causal inference in statistics: an overview, Statistics Surveys, 3, 96-146 (2009) · Zbl 1300.62013
[92] Pearson, K., On lines and planes of closest fit to a system of points in space, Philosophical Magazine, Series 6, 2, 559-572 (1901) · JFM 32.0710.04
[93] Pearson, K., Mathematical contributions to the theory of natural selection. XI. On the influence of natural selection on the variability and correlation of organs, Philosophical Transactions of the Royal Society, 200, 1-66 (1903) · JFM 34.0270.01
[94] Pearson, K., On the general theory of the influence of selection on correlation and variation, Biometrika, 8, 437-443 (191112)
[95] Puntanen, S, and Styan, G P H, ‘Schur complements in probability and statistics’, in Chapter 6, 163-226 in Zhang, F (2005).
[96] Ramsey, A. S., Dynamics, part II (1937), Cambridge University Press · JFM 63.1303.02
[97] Ramsey, F. L., Characterization of the partial autocorrelation function, Annals of Statistics, 2, 1296-1301 (1974) · Zbl 0301.62046
[98] Rao, C. R., Information and the accuracy attainable in the estimation of statistical parameters, Bulletin of the Calcutta Mathematical Society, 37, 81-91 (1945) · Zbl 0063.06420
[99] Rao, C. R., The utilisation of multiple measurements in problems of biological classification, Journal of the Royal Statistical Society Supplement, 9, 159-203 (1948) · Zbl 0034.07902
[100] Rao, C. R., Linear statistical inference and its applications (1965), New York: Wiley, New York · Zbl 0137.36203
[101] Ricci, G.; Levi-Civita, T., Méthodes de calcul différentiel absolu et leurs applications, Mathematische Annalen, 54, 125-201 (1900) · JFM 31.0297.01
[102] Robinson, G. K., That BLUP is a good thing: the estimation of random effects, Statistical Science, 6, 15-32 (1991) · Zbl 0955.62500
[103] Rue, H.; Held, L., Gaussian Markov random fields (2005), Boca Raton: Chapman and Hall/CRC, Boca Raton · Zbl 1093.60003
[104] Schur, I., Über Potenzreihen, die im Innern der Einheitskreises beschränkt sind, Journal für die reine und angewandte Mathematik, 147, 205-232 (1917) · JFM 46.0475.01
[105] Searle, S. R., Matrix algebra useful for statistics (1982), New York: Wiley, New York · Zbl 0555.62002
[106] Seneta, E., Non-negative matrices and Markov chains (1981), Heidelberg: Springer, Heidelberg · Zbl 0471.60001
[107] Sherman, J.; Morrison, W. J., Adjustment of an inverse matrix corresponding to a change in one element of a given matrix, Annals of Mathematical Statistics, 21, 124-127 (1950) · Zbl 0037.00901
[108] Simon, B., Orthogonal polynomials on the unit circle. Part 1: classical theory (2005), Providence: American Mathematical Society, Providence · Zbl 1082.42020
[109] Stewart, G. W., On the early history of the singular values decomposition, SIAM Review, 35, 551-566 (1993) · Zbl 0799.01016
[110] Stigler, S. M., The measurement of uncertainty before 1900 (1986), Harvard: Harvard University Press, Harvard · Zbl 0656.62005
[111] Stigler, S. M., Studies in the history of probability and statistics L: Karl Pearson and the rule of three, Biometrika, 99, 1-14 (2012) · Zbl 1234.62004
[112] Stigler, S. M., Mahalanobis & Fisher: mathematical statistics as a global enterprise, Sanhkyā B, 80, S167-S178 (2018)
[113] Todhunter, I, ‘On the method of least squares’. Trans. Cambridge Phil. Soc., 11 (Part II), 1869, 219; Addition, 238. · JFM 02.0115.03
[114] Turnbull, H. W., The theory of determinants, matrices and invariants (1928), London: Blackie, London · JFM 54.0129.12
[115] Turnbull, H. W.; Aitken, A. C., An introduction to the theory of canonical matrices (1932), London: Blackie, London · Zbl 0005.19303
[116] Verblunsky, S., On positive harmonic functions (second paper), Proceedings of the London Mathematical Society, 40, 290-320 (1936) · JFM 61.0523.01
[117] Weatherburn, C. E., Elementary vector analysis, with applications to geometry and mechanics (1921), G. Bell and Sons · JFM 48.0866.08
[118] Webb, A. R.; Copsey, K. D., Statistical pattern recognition (2011), New York: Wiley, New York · Zbl 1237.68006
[119] Whittle, P, ‘A Realised Path. The Cambridge Statistical Laboratory up to 1993’ (revised 2002), www.statslab.cam.ac.uk/history-statistical- laboratory. Statistical Laboratory, University of Cambridge, Cambridge.
[120] Wilkinson, J. H., The algebraic eigenvalue problem (1965), Oxford: Oxford University Press, Oxford · Zbl 0258.65037
[121] Wilks, S. S., Certain generalizations in the analysis of variance, Biometrika, 24, 471-494 (1932) · JFM 58.1172.02
[122] Wilks, S. S., Mathematical statistics (1962), New York: Wiley, New York · Zbl 0173.45805
[123] Wishart, J., The generalised product moment distribution in samples from a normal multivariate population, Biometrika, 20A, 32-52 (1928) · JFM 54.0565.02
[124] Wishart, J., Multivariate analysis, Journal of the Royal Statistical Society Series C, 4, 103-116 (1955)
[125] Wishart, J.; Bartlett, M. S., The generalised product moment distribution in a normal system, Proceedings of the Cambridge Philosophical Society, 29/2, 260-270 (1933) · JFM 59.0513.02
[126] Woodbury, M. A., Inverting modified matrices (1950), Princeton: Princeton University Press, Princeton
[127] Ycart, B., A case of mathematical eponymy: the vandermonde determinant, Revue d’Histoire des Mathématiques, 19, 43-77 (2013) · Zbl 1287.01012
[128] Zhang, F., The Schur complement and its applications (2005), Heidelberg: Springer, Heidelberg · Zbl 1075.15002
This reference list is based on information provided by the publisher or from digital mathematics libraries. Its items are heuristically matched to zbMATH identifiers and may contain data conversion errors. In some cases that data have been complemented/enhanced by data from zbMATH Open. This attempts to reflect the references listed in the original paper as accurately as possible without claiming completeness or a perfect matching.