## A survey of multilinear subspace learning for tensor data.(English)Zbl 1210.68083

Summary: Increasingly large amount of multidimensional data are being generated on a daily basis in many applications. This leads to a strong demand for learning algorithms to extract useful information from these massive data. This paper surveys the field of multilinear subspace learning (MSL) for dimensionality reduction of multidimensional data directly from their tensorial representations. It discusses the central issues of MSL, including establishing the foundations of the field via multilinear projections, formulating a unifying MSL framework for systematic treatment of the problem, examining the algorithmic aspects of typical MSL solutions, and categorizing both unsupervised and supervised MSL algorithms into taxonomies. Lastly, the paper summarizes a wide range of MSL applications and concludes with perspectives on future research directions.

### MSC:

 68T05 Learning and adaptive systems in artificial intelligence 68T10 Pattern recognition, speech recognition

### Software:

FRGC; Algorithm 862
Full Text:

### References:

 [1] J. Ye, R. Janardan, Q. Li, GPCA: an efficient dimension reduction scheme for image compression and retrieval, in: The Tenth ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, 2004, pp. 354-363. [2] Yan, S.; Xu, D.; Yang, Q.; Zhang, L.; Tang, X.; Zhang, H., Multilinear discriminant analysis for face recognition, IEEE transactions on image processing, 16, 1, 212-220, (2007) [3] Lu, J.; Plataniotis, K.N.; Venetsanopoulos, A.N., Face recognition using kernel direct discriminant analysis algorithms, IEEE transactions on neural networks, 14, 1, 117-126, (2003) [4] Lu, H.; Wang, J.; Plataniotis, K.N., A review on face and gait recognition: system, data and algorithms, (), 303-330 [5] Li, J.; Zhang, L.; Tao, D.; Sun, H.; Zhao, Q., A prior neurophysiologic knowledge free tensor-based scheme for single trial eeg classification, IEEE transactions on neural systems and rehabilitation engineering, 17, 2, 107-115, (2009) [6] H. Lu, K.N. Plataniotis, A.N. Venetsanopoulos, Regularized common spatial patterns with generic learning for EEG signal classification, in: Proceedings of 31st International Conference of the IEEE Engineering in Medicine and Biology Society, 2009. [7] Ye, J.; Li, T.; Xiong, T.; Janardan, R., Using uncorrelated discriminant analysis for tissue classification with gene expression data, IEEE/ACM transactions on computational biology bioinformatics, 1, 4, 181-190, (2004) [8] Sahambi, H.S.; Khorasani, K., A neural-network appearance-based 3-D object recognition using independent component analysis, IEEE transactions on neural networks, 14, 1, 138-149, (2003) [9] Renard, N.; Bourennane, S., Dimensionality reduction based on tensor modeling for classification methods, IEEE transactions on geoscience and remote sensing, 47, 4, 1123-1131, (2009) [10] Chellappa, R.; Roy-Chowdhury, A.; Zhou, S., Recognition of humans and their activities using video, (2005), Morgan & Claypool Publishers San Rafael, California [11] Green, R.D.; Guan, L., Quantifying and recognizing human movement patterns from monocular video images-part II: applications to biometrics, IEEE transactions on circuits and systems for video technology, 14, 2, 191-198, (2004) [12] Hardoon, D.R.; Shawe-Taylor, J., Decomposing the tensor kernel support vector machine for neuroscience data with structure labels, Machine learning, 79, 1-2, 29-46, (2010) [13] X. He, Incremental semi-supervised subspace learning for image retrieval, in: ACM Conference on Multimedia 2004, 2004, pp. 2-8. [14] Shechtman, E.; Caspi, Y.; Irani, M., Space-time super-resolution, IEEE transactions on pattern analysis and machine intelligence, 27, 4, 531-545, (2005) [15] J. Sun, D. Tao, C. Faloutsos, Beyond streams and graphs: dynamic tensor analysis, in: Proceedings the 12th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, 2006, pp. 374-383. [16] Sun, J.; Xie, Y.; Zhang, H.; Faloutsos, C., Less is more: sparse graph mining with compact matrix decomposition, Statistical analysis and data mining, 1, 1, 6-22, (2008) [17] Sun, J.; Tao, D.; Papadimitriou, S.; Yu, P.S.; Faloutsos, C., Incremental tensor analysis: theory and applications, ACM transactions on knowledge discovery from data, 2, 3, 11:1-11:37, (2008) [18] Shakhnarovich, G.; Moghaddam, B., Face recognition in subspaces, (), 141-168 [19] Jolliffe, I.T., Principal component analysis, (2002), Springer Series in Statistics · Zbl 1011.62064 [20] Li, S.Z.; Jain, A.K., Introduction, (), 1-11 [21] Lu, H.; Plataniotis, K.N.; Venetsanopoulos, A.N., MPCA: multilinear principal component analysis of tensor objects, IEEE transactions on neural networks, 19, 1, 18-39, (2008) [22] Tao, D.; Li, X.; Wu, X.; Hu, W.; Maybank, S.J., Supervised tensor learning, Knowledge and information systems, 13, 1, 1-42, (2007) [23] Tao, D.; Li, X.; Wu, X.; Maybank, S.J., General tensor discriminant analysis and Gabor features for gait recognition, IEEE transactions on pattern analysis and machine intelligence, 29, 10, 1700-1715, (2007) [24] Tao, D.; Li, X.; Wu, X.; Maybank, S.J., Tensor rank one discriminant analysis a convergent method for discriminative multilinear subspace selection, Neurocomputing, 71, 10-12, 1866-1882, (2008) [25] Lu, H.; Plataniotis, K.N.; Venetsanopoulos, A.N., Uncorrelated multilinear principal component analysis for unsupervised multilinear subspace learning, IEEE transactions on neural networks, 20, 11, 1820-1836, (2009) [26] Lu, H.; Plataniotis, K.N.; Venetsanopoulos, A.N., Uncorrelated multilinear discriminant analysis with regularization and aggregation for tensor object recognition, IEEE transactions on neural networks, 20, 1, 103-123, (2009) [27] Xu, D.; Yan, S.; Zhang, L.; Lin, S.; Zhang, H.-J.; Huang, T.S., Reconstruction and recognition of tensor-based objects with concurrent subspaces analysis, IEEE transactions on circuits and systems for video technology, 18, 1, 36-47, (2008) [28] Lu, H.; Plataniotis, K.N.; Venetsanopoulos, A.N., A taxonomy of emerging multilinear discriminant analysis solutions for biometric signal recognition, (), 21-45 [29] Greub, W.H., Multilinear algebra, (1967), Springer-Verlag Berlin · Zbl 0169.35302 [30] Lathauwer, L.D.; Moor, B.D.; Vandewalle, J., A multilinear singular value decomposition, SIAM journal of matrix analysis and applications, 21, 4, 1253-1278, (2000) · Zbl 0962.15005 [31] Lathauwer, L.D.; Moor, B.D.; Vandewalle, J., On the best rank-1 and rank-(R1,R2,…,RN) approximation of higher-order tensors, SIAM journal of matrix analysis and applications, 21, 4, 1324-1342, (2000) · Zbl 0958.15026 [32] Qi, L.; Sun, W.; Wang, Y., Numerical multilinear algebra and its applications, Frontiers of mathematics in China, 2, 4, 501-526, (2007) · Zbl 1134.65033 [33] Muti, D.; Bourennane, S., Survey on tensor signal algebraic filtering, Signal processing, 87, 2, 237-249, (2007) · Zbl 1186.94247 [34] Acar, E.; Yener, B., Unsupervised multiway data analysis: a literature survey, IEEE transactions on knowledge and data engineering, 21, 1, 6-20, (2009) [35] Kolda, T.G.; Bader, B.W., Tensor decompositions and applications, SIAM review, 51, 3, 455-500, (2009) · Zbl 1173.65029 [36] Zafeiriou, S., Algorithms for nonnegative tensor factorization, (), 105-124 · Zbl 1192.68840 [37] T. Hazan, S. Polak, A. Shashua, Sparse image coding using a 3D non-negative tensor factorization, in: Proceedings of IEEE Conference on Computer Vision, vol. 1, 2005, pp. 50-57. [38] A. Shashua, T. Hazan, Non-negative tensor factorization with applications to statistics and computer vision, in: Proceedings of International Conference on Machine Learning, 2005, pp. 792-799. [39] R. Bro, Multi-way analysis in the food industry – models, algorithms and applications, Ph.D. Thesis, University of Amsterdam, The Netherlands. URL $$\langle$$http://www.models.kvl.dk/sites/default/files/brothesis_0.pdf〉, 1998. [40] Smilde, A.K.; Bro, R.; Geladi, P., Multi-way analysis, (2004), John Wiley and Sons [41] Bader, B.W.; Kolda, T.G., Algorithm 862: Matlab tensor classes for fast algorithm prototyping, ACM transactions on mathematical software, 32, 4, 635-653, (2006) · Zbl 1230.65054 [42] H. Lu, Multilinear subspace learning for face and gait recognition, Ph.D. Thesis, University of Toronto, 2008. URL $$\langle$$https://tspace.library.utoronto.ca/handle/1807/16750〉. [43] X. He, D. Cai, P. Niyogi, Tensor subspace analysis, in: Advances in Neural Information Processing Systems 18 (NIPS), 2005. [44] Duda, R.O.; Hart, P.E.; Stork, D.G., Pattern classification, (2001), Wiley Interscience · Zbl 0968.68140 [45] Moon, T.K.; Stirling, W.C., Mathematical methods and algorithms for signal processing, (2000), Prentice Hall [46] Y. Wang, S. Gong, Tensor discriminant analysis for view-based object recognition, in: Proceedings of International Conference on Pattern Recognition, vol. 3, 2006, pp. 33-36. [47] D. Tao, X. Li, X. Wu, S.J. Maybank, Elapsed time in human gait recognition: a new approach, in: Proceedings of IEEE International Conference on Acoustics, Speech and Signal Processing, vol. 2, 2006, pp. 177-180. [48] G. Hua, P.A. Viola, S.M. Drucker, Face recognition using discriminatively trained orthogonal rank one tensor projections, in: Proceedings of IEEE Conference on Computer Vision and Pattern Recognition, 2007, pp. 1-8. [49] Turk, M.; Pentland, A., Eigenfaces for recognition, Journal of cognitive neurosicence, 3, 1, 71-86, (1991) [50] Belhumeur, P.N.; Hespanha, J.P.; Kriegman, D.J., Eigenfaces vs. fisherfaces: recognition using class specific linear projection, IEEE transactions on pattern analysis and machine intelligence, 19, 7, 711-720, (1997) [51] Comon, P., Independent component analysis a new concept?, Signal processing, 36, 287-314, (1994) · Zbl 0791.62004 [52] Thompson, B., Canonical correlation analysis: uses and interpretation, (1984), Sage Publications Thousand Oaks, CA [53] Carroll, J.D.; Chang, J.J., Analysis of individual differences in multidimensional scaling via an n-way generalization of “eckart-young” decomposition, Psychometrika, 35, 283-319, (1970) · Zbl 0202.19101 [54] Harshman, R.A., Foundations of the parafac procedure: models and conditions for an “explanatory” multi-modal factor analysis, UCLA working papers in phonetics, 16, 1-84, (1970) [55] Kroonenberg, P.; Leeuw, J., Principal component analysis of three-mode data by means of alternating least squares algorithms, Psychometrika, 45, 1, 69-97, (1980) · Zbl 0431.62035 [56] Yang, J.; Zhang, D.; Frangi, A.F.; Yang, J., Two-dimensional PCA: a new approach to appearance-based face representation and recognition, IEEE transactions on pattern analysis and machine intelligence, 26, 1, 131-137, (2004) [57] Ye, J., Generalized low rank approximations of matrices, Machine learning, 61, 1-3, 167-191, (2005) · Zbl 1087.65043 [58] Wang, H.; Ahuja, N., A tensor approximation approach to dimensionality reduction, International journal of computer vision, 76, 3, 217-229, (2008) [59] Tao, D.; Song, M.; Li, X.; Shen, J.; Sun, J.; Wu, X.; Faloutsos, C.; Maybank, S.J., Bayesian tensor approach for 3-D face modeling, IEEE transactions on circuits and systems for video technology, 18, 10, 1397-1410, (2008) [60] C.M. Bishop, Bayesian PCA, in: Advances in Neural Information Processing Systems (NIPS), 1999, pp. 382-388. [61] K. Inoue, K. Hara, K. Urahama, Robust multilinear principal component analysis, in: Proceedings of IEEE Conference on Computer Vision, 2009, pp. 591-597. [62] Panagakis, Y.; Kotropoulos, C.; Arce, G.R., Non-negative multilinear principal component analysis of auditory temporal modulations for music genre classification, IEEE transactions on audio, speech, and language processing, 18, 3, 576-588, (2010) [63] S. Papadimitriou, J. Sun, C. Faloutsos, Streaming pattern discovery in multiple time-series, in: Proceedings of 31st International Conference on Very Large Data Bases, 2005, pp. 697-708. [64] A. Shashua, A. Levin, Linear image coding for regression and classification using the tensor-rank principle, in: Proceedings of IEEE Conference on Computer Vision and Pattern Recognition, vol. I, 2001, pp. 42-49. [65] J. Ye, R. Janardan, Q. Li, Two-dimensional linear discriminant analysis, in: Advances in Neural Information Processing Systems (NIPS), 2004, pp. 1569-1576. [66] Liu, Q.; Tang, X.; Lu, H.; Ma, S., Face recognition using kernel scatter-difference-based discriminant analysis, IEEE transactions on neural networks, 17, 4, 1081-1085, (2006) [67] Fukunaga, K., Introduction to statistical pattern recognition, (1990), Academic Press Boston, MA · Zbl 0711.62052 [68] Kolda, T.G., Orthogonal tensor decompositions, SIAM journal of matrix analysis and applications, 23, 1, 243-255, (2001) · Zbl 1005.15020 [69] Jin, Z.; Yang, J.Y.; Tang, Z.M.; Hu, Z.S., A theorem on the uncorrelated optimal discriminant vectors, Pattern recognition, 34, 10, 2041-2047, (2001) · Zbl 0999.68189 [70] Ye, J.; Janardan, R.; Li, Q.; Park, H., Feature reduction via generalized uncorrelated linear discriminant analysis, IEEE transactions on knowledge and data engineering, 18, 10, 1312-1322, (2006) [71] Tucker, L.R., Some mathematical notes on three-mode factor analysis, Psychometrika, 31, 279-311, (1966) [72] Kapteyn, A.; Neudecker, H.; Wansbeek, T., An approach to n-mode components analysis, Psychometrika, 51, 269-275, (1986) · Zbl 0613.62078 [73] Carroll, J.D.; Pruzansky, S.; Kruskal, J.B., CANDELINC: a general approach to multidimensional analysis of many-way arrays with linear constraints on parameters, Psychometrika, 45, 3-24, (1980) · Zbl 0478.62050 [74] Comon, P.; Mourrain, B., Decomposition of quantics in sums of powers of linear forms, Signal processing, 53, 93-108, (1996) · Zbl 0875.94079 [75] Lathauwer, L.D.; Vandewalle, J., Dimensionality reduction in higher-order signal processing and rank-(R1,R2,…,RN) reduction in multilinear algebra, Linear algebra and its applications, 391, 31-55, (2004) · Zbl 1066.94001 [76] M.A.O. Vasilescu, D. Terzopoulos, Multilinear analysis of image ensembles: tensorfaces, in: Proceedings of Seventh European Conference on Computer Vision, 2002, pp. 447-460. · Zbl 1034.68693 [77] M.A.O. Vasilescu, D. Terzopoulos, Multilinear image analysis for facial recognition, in: Proceedings of International Conference on Pattern Recognition, vol. 2, 2002, pp. 511-514. [78] M.A.O. Vasilescu, Human motion signatures: analysis, synthesis, recognition, in: Proceedings of International Conference on Pattern Recognition, vol. 3, 2002, pp. 456-460. [79] C.S. Lee, A. Elgammal, Towards scalable view-invariant gait recognition: Multilinear analysis for gait, in: Proceedings of International Conference on Audio and Video-Based Biometric Person Authentication, 2005, pp. 395-405. [80] Kim, T.-K.; Cipolla, R., Canonical correlation analysis of video volume tensors for action categorization and detection, IEEE transactions on pattern analysis and machine intelligence, 31, 8, 1415-1428, (2009) [81] Xu, D.; Yan, S.; Lin, S.; Huang, T.S.; Chang, S.-F., Enhancing bilinear subspace learning by element rearrangement, IEEE transactions on pattern analysis and machine intelligence, 31, 10, 1913-1920, (2009) [82] S. Yan, D. Xu, S. Lin, T.S. Huang, S.-F. Chang, Element rearrangement for tensor-based subspace learning, in: Proceedings of IEEE Conference on Computer Vision and Pattern Recognition, 2007, pp. 1-8. [83] Zhang, J.; Pu, J.; Chen, C.; Fleischer, R., Low-resolution gait recognition, IEEE transactions on systems, man, and cybernetics—part B: cybernetics, 40, 4, 986-996, (2010) [84] G. Dai, D.Y. Yeung, Tensor embedding methods, in: Proceedings of Twenty-First National Conference on Artificial Intelligence, 2006, pp. 330-335. [85] Yan, S.; Xu, D.; Zhang, B.; Zhang, H.J.; Yang, Q.; Lin, S., Graph embedding and extensions: a general framework for dimensionality reduction, IEEE transactions on pattern analysis and machine intelligence, 29, 1, 40-51, (2007) [86] Xu, D.; Lin, S.; Yan, S.; Tang, X., Rank-one projections with adaptive margins for face recognition, IEEE transactions on systems, man, and cybernetics—part B: cybernetics, 37, 5, 1226-1236, (2007) [87] Gao, X.; Li, X.; Feng, J.; Tao, D., Shot-based video retrieval with optical flow tensor and hmms, Pattern recognition letters, 30, 2, 140-147, (2010) [88] S. Aja-Fernández, R.d.L. García, D. Tao, X. Li (Eds.), Tensors in Image Processing and Computer Vision, Springer, 2009. [89] Cichocki, A.; Zdunek, R.; Phan, A.H.; Amari, S., Nonnegative matrix and tensor factorizations: applications to exploratory multi-way data analysis and blind source separation, (2009), Wiley-Blackwell [90] Workshop on Algorithms for Modern Massive Data Sets, 2010, 2008, 2006. URL $$\langle$$http://www.stanford.edu/group/mmds/〉. [91] Jain, A.K.; Ross, A.; Prabhakar, S., An introduction to biometric recognition, IEEE transactions on circuits and systems for video technology, 14, 1, 4-20, (2004) [92] Wang, J.; Barreto, A.; Wang, L.; Chen, Y.; Rishe, N.; Andrian, J.; Adjouadi, M., Multilinear principal component analysis for face recognition with fewer features, Neurocomputing, 73, 10-12, 1550-1555, (2010) [93] H. Lu, K.N. Plataniotis, A.N. Venetsanopoulos, Uncorrelated multilinear principal component analysis through successive variance maximization, in: Proceedings of International Conference on Machine Learning, 2008, pp. 616-623. [94] H. Lu, K.N. Plataniotis, A.N. Venetsanopoulos, Uncorrelated multilinear discriminant analysis with regularization for gait recognition, in: Proceedings of Biometrics Symposium 2007, 2007, doi:10.1109/BCC.2007.4430540. [95] H. Lu, K.N. Plataniotis, A.N. Venetsanopoulos, Boosting LDA with regularization on MPCA features for gait recognition, in: Proceedings of Biometrics Symposium 2007, 2007, doi:10.1109/BCC.2007.4430542. [96] H. Lu, K.N. Plataniotis, A.N. Venetsanopoulos, Multilinear principal component analysis of tensor objects for recognition, in: Proceedings of International Conference on Pattern Recognition, vol. 2, 2006, pp. 776-779. [97] Lu, H.; Plataniotis, K.N.; Venetsanopoulos, A.N., Boosting discriminant learners for gait recognition using mpca features, EURASIP journal on image and video processing, 11, (2009), article ID 713183 [98] Blankertz, B.; Tomioka, R.; Lemm, S.; Kawanabe, M.; Müller, K.-R., Optimizing spatial filters for robust EEG single-trial analysis, IEEE signal processing magazine, 25, 1, 41-56, (2008) [99] Wen, J.; Gao, X.; Yuan, Y.; Tao, D.; Li, J., Incremental tensor biased discriminant analysis: a new color-based visual tracking method, Neurocomputing, 73, 4-6, 827-839, (2010) [100] H. Lu, H.-L. Eng, M. Thida, K.N. Plataniotis, Visualization and clustering of crowd video content in mpca subspace, in: Proceedings of 19th ACM Conference on Information and Knowledge Management, 2010, pp. 1777-1780. [101] Gao, X.; Yang, Y.; Tao, D.; Li, X., Discriminative optical flow tensor for video semantic analysis, Computer vision and image understanding, 113, 3, 372-383, (2009) [102] J. Wen, X. Li, X. Gao, D. Tao, Incremental learning of weighted tensor subspace for visual tracking, in: Proceedings of 2009 IEEE International Conference on Systems, Man and Cybernetics, 2009, pp. 3688-3693. [103] J.A. Ruiz-Hernandez, J.L. Crowley, A. Lux, “How old are you?”: age estimation with tensors of binary gaussian receptive maps, in: Proceedings of the British Machine Vision Conference, 2010, pp. 6.1-11. [104] Tenenbaum, J.B.; de Silva, V.; Langford, J., A global geometric framework for nonlinear dimensionality reduction, Science, 290, 22, 2319-2323, (2000) [105] Roweis, S.; Saul, L., Nonlinear dimensionality reduction by locally linear embedding, Science, 290, 22, 2323-2326, (2000) [106] He, X.; Yan, S.; Hu, Y.; Niyogi, P.; Zhang, H., Face recognition using laplacianfaces, IEEE transactions on pattern analysis and machine intelligence, 27, 3, 328-340, (2005) [107] Cai, D.; He, X.; Han, J.; Zhang, H.J., Orthogonal laplacianfaces for face recognition, IEEE transactions on image processing, 15, 11, 3608-3614, (2006) [108] Bowyer, K.W.; Chang, K.; Flynn, P., A survey of approaches and challenges in 3D and multi-modal 3D + 2D face recognition, Computer vision and image understanding, 101, 1, 1-15, (2006) [109] S.Z. Li, C. Zhao, M. Ao, Z. Lei, Learning to fuse 3D+2D based face recognition at both feature and decision levels, in: Proceedings IEEE International Workshop on Analysis and Modeling of Faces and Gestures, 2005, pp. 43-53. [110] Colombo, A.; Cusano, C.; Schettini, R., 3D face detection using curvature analysis, Pattern recognition, 39, 3, 444-455, (2006) · Zbl 1158.68464 [111] P.J. Phillips, P. Flynn, T. Scruggs, K. Bowyer, J. Chang, K. Hoffman, J. Marques, J. Min, W. Worek, Overview of the face recognition grand challenge, in: Proceedings IEEE Conference on Computer Vision and Pattern Recognition, vol. 1, 2005, pp. 947-954. [112] Xu, R.; II, D.W., Survey of clustering algorithms, IEEE transactions on neural networks, 16, 3, 645-678, (2005) [113] Nolker, C.; Ritter, H., Visual recognition of continuous hand postures, IEEE transactions on neural networks, 13, 4, 983-994, (2002)
This reference list is based on information provided by the publisher or from digital mathematics libraries. Its items are heuristically matched to zbMATH identifiers and may contain data conversion errors. It attempts to reflect the references listed in the original paper as accurately as possible without claiming the completeness or perfect precision of the matching.