Weighted least squares fitting using ordinary least squares algorithms. (English) Zbl 0873.62058

Summary: A general approach for fitting a model to a data matrix by weighted least squares (WLS) is studied. This approach consists of iteratively performing (steps of) existing algorithms for ordinary least squares (OLS) fitting of the same model. The approach is based on minimizing a function that majorizes the WLS loss function. The generality of the approach implies that, for every model for which an OLS fitting algorithm is available, the present approach yields a WLS fitting algorithm. In the special case where the WLS weight matrix is binary, the approach reduces to missing data imputation.


62H25 Factor analysis and principal components; correspondence analysis
62H12 Estimation in multivariate analysis
62P15 Applications of statistics to psychology


Full Text: DOI


[1] Bailey, R. A., & Gower, J. C. (1990). Approximating a symmetric matrix.Psychometrika, 55, 665–675. · Zbl 0721.65018
[2] Bijleveld, C. & de Leeuw, J. (1991). Fitting longitudinal reduced rank regression models by alternating least squares.Psychometrika, 56, 443–447. · Zbl 0760.62062
[3] Bollen, K. A. (1989).Structural equations with latent variables. New York: Wiley. · Zbl 0731.62159
[4] Browne, M. W. (1984). Asymptotically distribution-free methods for the analysis of covariance structures.British Journal of Mathematical and Statistical Psychology, 37, 62–83. · Zbl 0561.62054
[5] Carroll, J. D., De Soete, G., & Pruzansky, S. (1989). Fitting of the latent class model via iteratively reweighted least squares candecomp with nonnegativity constraints. In R. Coppi & S. Bolasco (Eds.),Multiway data analysis (pp. 463–472). Amsterdam: Elsevier Science Publishers.
[6] Cliff, N. (1966). Orthogonal rotation to congruence.Psychometrika, 31, 33–42.
[7] Commandeur, J. J. F. (1991). Matching configurations. Leiden: DSWO Press.
[8] de Leeuw, J. & Heiser, W. (1980). Multidimensional scaling with restrictions on the configuration. In P. R. Krishnaiah (Ed.),Multivariate analysis V (pp. 501–522). Amsterdam: North Holland. · Zbl 0468.62054
[9] Dempster, A. P., Laird, N. M., & Rubin, D. B. (1977). Maximum likelihood from incomplete data via the EM algorithm.Journal of the Royal Statistical Society, Series B, 39, 1–38. · Zbl 0364.62022
[10] Gabriel, K. R., & Zamir, S. (1979). Lower rank approximation of matrices by least squares with any choice of weights.Technometrics, 21, 489–498. · Zbl 0471.62004
[11] Gifi, A. (1990).Nonlinear multivariate analysis. Chichester: Wiley. · Zbl 0697.62048
[12] Green, B. F. (1952). The orthogonal approximation of an oblique structure in factor analysis.Psychometrika, 17, 429–440. · Zbl 0049.37601
[13] Harman, H. H., & Jones, W. H. (1966). Factor analysis by minimizing residuals (Minres).Psychometrika, 31, 351–368.
[14] Harshman, R. A. (1978, August).Models for analysis of asymmetrical relationships among N objects or stimuli. Paper presented at the First Joint Meeting of the Psychometric Society and the Society for Mathematical Psychology, Hamilton, Ontario.
[15] Harshman, R. A., Green, P. E., Wind, Y., & Lundy, M. E. (1982). A model for the analysis of asymmetric data in marketing research,Marketing Science, 1, 205–242.
[16] Harshman, R. A., & Lundy, M. E. (1984). The PARAFAC model for three-way factor analysis and multidimensional scaling. In H. G. Law, C. W. Snyder, J. A. Hattie, & R. P. McDonald (Eds.),Research methods for multimode data analysis (pp. 122–215). New York: Praeger.
[17] Heiser, W. J. (1987). Correspondence Analysis with least absolute residuals.Computational Statistics and Data Analysis, 5, 337–356. · Zbl 0624.62052
[18] Heiser, W. J. (1995). Convergent computation by iterative majorization: theory and applications in multidimensional data analysis. In W. J. Krzanowski (Ed.),Recent advances in descriptive multivariate analysis (pp. 157–189). Oxford: Oxford University Press.
[19] Jöreskog, K. G., & Sörbom, D. (1993).LISREL 8 User’s guide. Chicago: Scientific Software International.
[20] Kiers, H. A. L. (1989). An alternating least squares algorithm for fitting the two- and three-way DEDICOM model and the IDIOSCAL model.Psychometrika, 54, 515–521. · Zbl 04561111
[21] Kiers, H. A. L. (1990). Majorization as a tool for optimizing a class of matrix functions.Psychometrika, 55, 417–428. · Zbl 0733.62067
[22] Kiers, H. A. L. (1993). An alternating least squares algorithm for PARAFAC2 and DEDICOM3.Computational Statistics and Data Analysis, 16, 103–118. · Zbl 0875.62260
[23] Kiers, H. A. L., & ten Berge, J. M. F. (1992). Minimization of a class of matrix trace functions by means of refined majorization.Psychometrika, 57, 371–382. · Zbl 0782.62067
[24] Kiers, H. A. L., ten Berge, J. M. F., Takane, Y., & de Leeuw, J. (1990). A generalization of Takane’s algorithm for DEDICOM.Psychometrika, 55, 151–158. · Zbl 0717.62003
[25] Takane, Y. (1985). Diagonal Estimation in DEDICOM.Proceedings of the 1985 Annual Meeting of the Behaviormetric Society (pp. 100–101). Sapporo, Japan: Behaviormetric Society.
[26] ten Berge, J. M. F., & Kiers, H. A. L. (1989). Fitting the off-diagonal DEDICOM model in the least-squares sense by a generalization of the Harman & Jones MINRES procedure of factor analysis.Psychometrika, 54, 333–337. · Zbl 04550503
[27] ten Berge, J. M. F., & Kiers, H. A. L. (1993). An alternating least squares method for the weighted approximation of a symmetric matrix.Psychometrika, 58, 115–118. · Zbl 0775.62148
[28] ten Berge, J. M. F., Kiers, H. A. L., & Commandeur, J. J. F. (1993). Orthogonal Procrustes rotation for matrices with missing values.British Journal of Mathematical and Statistical Psychology, 46, 119–134. · Zbl 0771.62082
[29] Verboon, P. (1994).A robust approach to nonlinear multivariate analysis. Leiden: DSWO Press. · Zbl 0868.62052
[30] Verboon, P., & Heiser, W. J. (1992). Resistant orthogonal Procrustes analysis.Journal of Classification, 9, 237–256. · Zbl 0755.62050
[31] Verboon, P., & Heiser, W. J. (1994). Resistant lower rank approximation of matrices by iterative majorization.Computational Statistics and Data Analysis, 18, 457–467. · Zbl 0825.65128
This reference list is based on information provided by the publisher or from digital mathematics libraries. Its items are heuristically matched to zbMATH identifiers and may contain data conversion errors. It attempts to reflect the references listed in the original paper as accurately as possible without claiming the completeness or perfect precision of the matching.