Independent component analysis for tensor-valued data. (English) Zbl 1381.62107

Summary: In preprocessing tensor-valued data, e.g., images and videos, a common procedure is to vectorize the observations and subject the resulting vectors to one of the many methods used for independent component analysis (ICA). However, the tensor structure of the original data is lost in the vectorization and, as a more suitable alternative, we propose the matrix- and tensor fourth order blind identification (MFOBI and TFOBI). In these tensorial extensions of the classic fourth order blind identification (FOBI) we assume a Kronecker structure for the mixing and perform FOBI simultaneously on each direction of the observed tensors. We discuss the theory and assumptions behind MFOBI and TFOBI and provide two different algorithms and related estimates of the unmixing matrices along with their asymptotic properties. Finally, simulations are used to compare the method’s performance with that of classical FOBI for vectorized data and we end with a real data clustering example.


62H12 Estimation in multivariate analysis
62H25 Factor analysis and principal components; correspondence analysis
62G20 Asymptotic properties of nonparametric inference
Full Text: DOI arXiv


[1] Beckmann, C. F.; Smith, S. M., Tensorial extensions of independent component analysis for multisubject FMRI analysis, Neuroimage, 25, 294-311, (2005)
[2] Cardoso, J.-F., Source separation using higher order moments, (International Conference on Acoustics, Speech, and Signal Processing 1989, (1989), IEEE), 2109-2112
[3] Cardoso, J.-F.; Souloumiac, A., Blind beamforming for non-Gaussian signals, (IEE Proceedings F (Radar and Signal Processing), Vol. 140, (1993), IET), 362-370
[4] Ding, S.; Cook, R. D., Dimension folding PCA and PFC for matrix-valued predictors, Statist. Sinica, 24, 463-492, (2014) · Zbl 1416.62325
[5] Ding, S.; Cook, R. D., Higher-order sliced inverse regressions, Wiley Interdiscip. Rev. Comput. Statist., 7, 249-257, (2015)
[6] Ding, S.; Cook, R. D., Tensor sliced inverse regression, J. Multivariate Anal., 133, 216-231, (2015) · Zbl 1302.62127
[7] Greenewald, K.; Hero, A., Robust Kronecker product PCA for spatio-temporal covariance estimation, IEEE Trans. Signal Process., 63, 6368-6378, (2015) · Zbl 1395.94113
[8] Gupta, A.; Nagar, D., Matrix Variate Distributions, (2010), Chapman & Hall/CRC Boca Raton, FL · Zbl 0935.62064
[9] Hung, H.; Wang, C.-C., Matrix variate logistic regression model with application to EEG data, Biostatistics, 14, 189-202, (2013)
[10] Hung, H.; Wu, P.; Tu, I.; Huang, S., On multilinear principal component analysis, Biometrika, 99, 569-583, (2012) · Zbl 1437.62500
[11] Hyvärinen, A.; Karhunen, J.; Oja, E., Independent Component Analysis, (2001), Wiley New York
[12] Ilmonen, P.; Nevalainen, J.; Oja, H., Characteristics of multivariate distributions and the invariant coordinate system, Statist. Probab. Lett., 80, 1844-1853, (2010) · Zbl 1202.62068
[13] Ilmonen, P.; Nordhausen, K.; Oja, H.; Ollila, E., A new performance index for ICA: properties, computation and asymptotic analysis, (Latent Variable Analysis and Signal Separation, (2010), Springer New York), 229-236
[14] Ilmonen, P.; Oja, H.; Serfling, R., On invariant coordinate system (ICS) functionals, Internat. Statist. Rev., 80, 93-110, (2012)
[15] Kim, H.-J.; Ollila, E.; Koivunen, V.; Croux, C., Robust and sparse estimation of tensor decompositions, (Global Conference on Signal and Information Processing (GlobalSIP), 2013 IEEE, (2013), IEEE), 965-968
[16] Klaassen, C. A.; Mokveld, P. J.; Van Es, B., Squared skewness minus kurtosis bounded by 186/125 for unimodal distributions, Statist. Probab. Lett., 50, 131-135, (2000) · Zbl 0966.60006
[17] Kolda, T. G.; Bader, B. W., Tensor decompositions and applications, SIAM Rev., 51, 455-500, (2009) · Zbl 1173.65029
[18] Lathauwer, L. D.; Moor, B. D.; Vandewalle, J., A multilinear singular value decomposition, SIAM J. Matrix Anal. Appl., 21, 1253-1278, (2000) · Zbl 0962.15005
[19] Li, K.-C., Sliced inverse regression for dimension reduction, J. Amer. Statist. Assoc., 86, 316-327, (1991) · Zbl 0742.62044
[20] Li, B.; Kim, M. K.; Altman, N., On dimension folding of matrix- or array-valued statistical objects, Ann. Statist., 38, 1094-1121, (2010) · Zbl 1183.62091
[21] M. Lichman, UCI Machine Learning Repository, 2013.
[22] Lu, H.; Plataniotis, K. N.; Venetsanopoulos, A. N., A survey of multilinear subspace learning for tensor data, Pattern Recognit., 44, 1540-1551, (2011) · Zbl 1210.68083
[23] Manceur, A. M.; Dutilleul, P., Maximum likelihood estimation for the tensor normal distribution: algorithm, minimum sample size, and empirical bias and dispersion, J. Comput. Appl. Math., 239, 37-49, (2013) · Zbl 1255.65029
[24] McCullagh, P., Tensor Methods in Statistics, (1987), Chapman & Hall New York · Zbl 0732.62003
[25] Miettinen, J.; Nordhausen, K.; Oja, H.; Taskinen, S., Deflation-based fastica with adaptive choices of nonlinearities, IEEE Trans. Signal Process., 62, 5716-5724, (2014) · Zbl 1394.94394
[26] Miettinen, J.; Nordhausen, K.; Taskinen, S., Blind source separation based on joint diagonalization in R : the packages JADE and bssasymp, J. Stat. Softw., 76, (2017)
[27] Miettinen, J.; Taskinen, S.; Nordhausen, K.; Oja, H., Fourth moments and independent component analysis, Statist. Sci., 30, 372-390, (2015) · Zbl 1332.62196
[28] Nordhausen, K.; Oja, H.; Tyler, D. E., Tools for exploring multivariate data: the package ICS, J. Stat. Softw., 28, 1-31, (2008)
[29] K. Nordhausen, H. Oja, D.E. Tyler, Asymptotic and bootstrap tests for subspace dimension, 2016, arXiv preprint arxiv:1611.04908.
[30] Nordhausen, K.; Oja, H.; Tyler, D. E.; Virta, J., Asymptotic and bootstrap tests for the dimension of the non-Gaussian subspace, IEEE Signal Process. Lett., (2017)
[31] Ohlson, M.; Ahmad, M. R.; von Rosen, D., The multilinear normal distribution: introduction and some basic properties, J. Multivariate Anal., 113, 37-47, (2013) · Zbl 1354.60015
[32] Pan, V. Y.; Chen, Z.; Zheng, A., (The Complexity of the Algebraic Eigenproblem, Mathematical Sciences Research Institute, (1998)), MSRI Preprint 1998-071
[33] Peña, D.; Prieto, F.; Viladomat, J., Eigenvectors of a kurtosis matrix as interesting directions to reveal cluster structure, J. Multivariate Anal., 101, 1995-2007, (2010) · Zbl 1203.62114
[34] Pfeiffer, R. M.; Forzani, L.; Bura, E., Sufficient dimension reduction for longitudinally measured predictors, Stat. Med., 31, 2414-2427, (2012)
[35] T. Plate, R. Heiberger, ABIND: Combine multidimensional arrays, 2015, R Package Version 1.4-3.
[36] R Core Team. R : A language and environment for statistical computing, R Foundation for Statistical Computing, Vienna, Austria, 2014.
[37] Roman, S., Advanced Linear Algebra, Vol. 3, (2005), Springer · Zbl 1085.15001
[38] Ros, B.; Bijma, F.; de Munck, J. C.; de Gunst, M. C., Existence and uniqueness of the maximum likelihood estimator for models with a Kronecker product covariance structure, J. Multivariate Anal., 143, 345-361, (2016) · Zbl 1328.62135
[39] J. Rougier, TENSOR: Tensor product of arrays, R Package Version 1.5, 2012.
[40] Schott, J. R., Tests for Kronecker envelope models in multilinear principal component analysis, Biometrika, 101, 978-984, (2014) · Zbl 1306.62135
[41] Srivastava, M. S.; von Rosen, T.; von Rosen, D., Models with a Kronecker product covariance structure: estimation and testing, Math. Methods Statist., 17, 357-370, (2008) · Zbl 1231.62101
[42] Y. Sun, P. Babu, D. Palomar, Robust estimation of structured covariance matrix for heavy-tailed distributions, in: IEEE International Conference on Acoustics, Speech and Signal Processing, ICASSP, 2015, 2015, pp. 5693-5697.
[43] Tyler, D. E.; Critchley, F.; Dümbgen, L.; Oja, H., Invariant co-ordinate selection, J. R. Stat. Soc. Ser. B Stat. Methodol., 71, 549-592, (2009) · Zbl 1250.62032
[44] Vasilescu, M. A.O.; Terzopoulos, D., Multilinear independent components analysis, (IEEE Computer Society Conference on Computer Vision and Pattern Recognition, 2005, Vol. 1, (2005), IEEE), 547-553
[45] Venables, W. N.; Ripley, B. D., Modern Applied Statistics with S, (2002), Springer New York · Zbl 1006.62003
[46] J. Virta, B. Li, K. Nordhausen, H. Oja, JADE for tensor-valued observations, 2016, Preprint in arXiv:1603.05406. · Zbl 1381.62107
[47] Virta, J.; Nordhausen, K., Blind source separation of tensor-valued time series, Signal Process., 141, 204-216, (2017)
[48] J. Virta, K. Nordhausen, H. Oja, Joint use of third and fourth cumulants in independent component analysis, 2015, arXiv preprint arXiv:1505.02613.
[49] J. Virta, K. Nordhausen, H. Oja, B. Li, tensorBSS: Blind Source Separation Methods for Tensor-Valued Observations, R Package Version 0.3, 2016.
[50] Virta, J.; Taskinen, S.; Nordhausen, K., Applying fully tensorial ICA to fMRI data, (Signal Processing in Medicine and Biology Symposium (SPMB), 2016 IEEE, (2016), IEEE), 1-6
[51] Werner, K.; Jansson, M.; Stoica, P., On estimation of covariance matrices with Kronecker product structure, IEEE Trans. Signal Process., 56, 478-491, (2008) · Zbl 1390.94472
[52] Wiesel, A., Geodesic convexity and covariance estimation, IEEE Trans. Signal Process., 60, 6182-6189, (2012) · Zbl 1393.94489
[53] Xue, Y.; Yin, X., Sufficient dimension folding for regression mean function, J. Comput. Graph. Statist., 23, 1028-1043, (2014)
[54] Zeng, P.; Zhong, W., Dimension reduction for tensor classification, Topics Appl. Statist., 55, 213-227, (2013)
[55] Zhang, L.; Gao, Q.; Zhang, L., Directional independent component analysis with tensor representation, (IEEE Conference on Computer Vision and Pattern Recognition 2008, (2008), IEEE), 1-7
[56] Zhao, J.; Leng, C., Structured lasso for regression with matrix covariates, Statist. Sinica, 24, 799-814, (2014) · Zbl 1285.62064
[57] Zhong, W.; Xing, X.; Suslick, K., Tensor sufficient dimension reduction, Wiley Interdiscip. Rev. Comput. Statist., 7, 178-184, (2015)
[58] Zhou, H.; Li, L., Regularized matrix regression, J. R. Stat. Soc. Ser. B Stat. Methodol., 76, 463-483, (2014)
[59] Zhou, H.; Li, L.; Zhu, H., Tensor regression with applications in neuroimaging, J. Amer. Statist. Assoc., 108, 540-552, (2013) · Zbl 06195959
This reference list is based on information provided by the publisher or from digital mathematics libraries. Its items are heuristically matched to zbMATH identifiers and may contain data conversion errors. It attempts to reflect the references listed in the original paper as accurately as possible without claiming the completeness or perfect precision of the matching.