Majorization as a tool for optimizing a class of matrix functions. (English) Zbl 0733.62067

The paper considers the monotonic minimization over X(n\(\times p)\) of the trace function \[ f(x)=c+tr AX\quad +\sum^{m}_{j=1}tr B_ jXC_ jX', \] where c is a scalar independent of X, A(p\(\times n)\), \(B_ j(n\times n)\), and \(C_ j(p\times p)\). The optimization problem is related to fitting statistical models by least squares. Three cases are considered: unrestricted X, \(X'X=I_ p\) (columnwise orthogonal X), and \(rank(X)<\min (n,p)\) (case of reduced rank). The approach is to majorize this function by one having a simple quadratic shape and whose minimum is easily obtained.
“It is shown that the parameter set that minimizes the majorizing function also decreases the matrix trace function, which in turn provides a monotonically convergent algorithm for minimizing the matrix function iteratively. Three algorithms based on majorization for solving certain least squares problems are shown to be special cases. In addition, by means of several examples, it is noted how algorithms may be provided for a wide class of statistical optimization tasks for which no satisfactory algorithms seem available.” (Author’s abstract.)


62H25 Factor analysis and principal components; correspondence analysis
62H99 Multivariate analysis
90C90 Applications of mathematical programming
Full Text: DOI


[1] Bijleveld, C., & de Leeuw, J. (1987, June).Fitting linear dynamics systems by alternating least squares. Paper presented at the European Meeting of the Psychometric Society, Twente, The Netherlands.
[2] Carroll, J. D., & Chang, J. J. (1972, March).IDIOSCAL: A generalization of INDSCAL allowing IDIOsyncratic reference systems as well as an analytic approximation to INDSCAL. Paper presented at the Spring Meeting of the Psychometric Society, Princeton, N.J.
[3] Cliff, N. (1966). Orthogonal rotation to congruence.Psychometrika, 31, 33–42.
[4] de Leeuw, J. (1988). Convergence of the majorization method for Multidimensional Scaling.Journal of Classification, 5, 163–180. · Zbl 0692.62056
[5] de Leeuw, J., & Heiser, W. (1980). Multidimensional scaling with restrictions on the configuration. In P. R. Krishnaiah (Ed.),Multivariate analysis V (pp. 501–522). Amsterdam: North Holland Publishing Company. · Zbl 0468.62054
[6] de Leeuw, J., Young, F. W., & Takane, Y. (1976). Additive structure in qualitative data: an alternating least squares method with optimal scaling features.Psychometrika, 41, 471–503. · Zbl 0351.92031
[7] Eckart, C., & Young, G. (1936). The approximation of one matrix by another of lower rank.Psychometrika, 1, 211–218. · JFM 62.1075.02
[8] Harshman, R. A. (1978, August).Models for analysis of asymmetric relationships among N objects or stimuli. Paper presented at the first joint meeting of the Psychometic Society and the Society for Mathematical Psychology, Hamilton, Ontario.
[9] Heiser, W. J. (1987). Correspondence Analysis with least absolute residuals.Computational Statistics and Data Analysis, 5, 337–356. · Zbl 0624.62052
[10] Kiers, H. A. L. (1989). INDSCAL for the analysis of categorical data. In R. Coppi & S. Bolasco (Eds.),Multiway data analysis (pp. 155–167). Amsterdam: Elsevier Science.
[11] Kiers, H. A. L., ten Berge, J. M. F., Takane, Y., & de Leeuw, J. (1990). A generalization of Takane’s algorithm for DEDICOM.Psychometrika, 55, 151–158. · Zbl 0717.62003
[12] Kroonenberg, P. M. (1983).Three mode principal component analysis: Theory and applications. Leiden: DSWO. · Zbl 0513.62059
[13] Kroonenberg, P. M., & de Leeuw, J. (1980). Principal component analysis of three-mode data by means of alternating least squares algorithms.Psychometrika, 45, 69–97. · Zbl 0431.62035
[14] Lingoes, J. C., & Borg, I. (1978). A direct approach to individual differences scaling using increasingly complex transformations.Psychometrika, 43, 491–519. · Zbl 0395.62077
[15] Meulman, J. J. (1986).A distance approach to nonlinear multivariate analysis. Leiden: DSWO.
[16] Ramsay, J. O. (1975). Solving implicit equations in psychometric data analysis.Psychometrika, 40, 337–360. · Zbl 0311.92024
[17] ten Berge, J. M. F., Knol, D. L., & Kiers, H. A. L. (1988). A treatment of the Orthomax rotation family in terms of diagonalization, and a re-examination of a singular value approach to Varimax rotation.Computational Statistics Quarterly, 3, 207–217. · Zbl 0726.62090
[18] Wold, H. (1966). Estimation of principal components and related models by iterative least squares. In P. R. Krishnaiah (Ed.),Multivariate analysis II (pp. 391–420). New York: Academic Press. · Zbl 0214.46103
[19] Young, F. W., de Leeuw, J., & Takane, Y. (1976). Regression with qualitative and quantitative variables: an alternating least squares method with optimal scaling features.Psychometrika, 41, 505–529. · Zbl 0351.92032
This reference list is based on information provided by the publisher or from digital mathematics libraries. Its items are heuristically matched to zbMATH identifiers and may contain data conversion errors. It attempts to reflect the references listed in the original paper as accurately as possible without claiming the completeness or perfect precision of the matching.