A general framework for tensor screening through smoothing. (English) Zbl 1493.62595

Summary: Screening is an important technique for analyzing high-dimensional data. Most screening tools have been developed for vectors and are marginal in the sense that each variable is evaluated individually at a time. Many multi-dimensional arrays (tensors) are generated nowadays. In addition to being high-dimensional, these data further have the tensor structure that should be exploited for more efficient analysis. Variables adjacent to each other in a tensor tend to be important or unimportant at the same time. Such information is ignored by marginal screening methods. In this article, we propose a general framework for tensor screening called smoothed tensor screening (STS). STS combines the strength of current marginal screening methods with tensor structural information by aggregating the information of its adjacent variables when evaluating one variable. STS is widely applicable since the statistical utility used in screening can be chosen based on the underlying model or data type of the responses and predictors. Moreover, we establish the SURE screening property for STS under mild conditions. Numerical studies demonstrate that STS has better performance than marginal screening methods.


62P10 Applications of statistics to biology and medical sciences; meta analysis
62F07 Statistical ranking and selection procedures
Full Text: DOI Link


[1] Brandi, G. and Di Matteo, T. (2021). Predicting multidimensional data via tensor learning. Journal of Computational Science 53 101372.
[2] Cai, T. and Liu, W. (2011). A direct estimation approach to sparse linear discriminant analysis. Journal of the American Statistical Association 106 1566-1577. · Zbl 1233.62129
[3] Chang, J., Tang, C. Y. and Wu, Y. (2013). Marginal empirical likelihood and sure independence feature screening. The Annals of Statistics 41 2123-2148. · Zbl 1277.62109 · doi:10.1214/13-AOS1139
[4] Chang, J., Tang, C. Y. and Wu, Y. (2016). Local independence feature screening for nonparametric and semiparametric models by marginal empirical likelihood. The Annals of Statistics 44 515-539. · Zbl 1486.62082 · doi:10.1214/15-AOS1374
[5] Chi, E. C. and Kolda, T. G. (2012). On tensors, sparsity, and nonnegative factorizations. SIAM Journal on Matrix Analysis and Applications 33 1272-1299. · Zbl 1262.15029
[6] Clemmensen, L., Hastie, T., Witten, D. and ErsbØll, B. (2011). Sparse discriminant analysis. Technometrics 53 406-413.
[7] Fan, J. and Fan, Y. (2008). High-dimensional classification using features annealed independence rules. The Annals of Statistics 36 2605-2637. · Zbl 1360.62327 · doi:10.1214/07-AOS504
[8] Fan, J., Feng, Y. and Song, R. (2011). Nonparametric independence screening in sparse ultra-high-dimensional additive models. Journal of the American Statistical Association 106 544-557. · Zbl 1232.62064
[9] Fan, J., Feng, Y. and Tong, X. (2012). A road to classification in high dimensional space: the regularized optimal affine discriminant. Journal of the Royal Statistical Society: Series B (Statistical Methodology) 74 745-771. · Zbl 1411.62167
[10] Fan, J. and Lv, J. (2008). Sure independence screening for ultrahigh dimensional feature space. Journal of the Royal Statistical Society: Series B (Statistical Methodology) 70 849-911. · Zbl 1411.62187
[11] Fan, J., Ma, Y. and Dai, W. (2014). Nonparametric independence screening in sparse ultra-high-dimensional varying coefficient models. Journal of the American Statistical Association 109 1270-1284. · Zbl 1368.62095
[12] Fan, J., Samworth, R. and Wu, Y. (2009). Ultrahigh dimensional feature selection: beyond the linear model. The Journal of Machine Learning Research 10 2013-2038. · Zbl 1235.62089
[13] Fan, J. and Song, R. (2010). Sure independence screening in generalized linear models with NP-dimensionality. The Annals of Statistics 38 3567-3604. · Zbl 1206.68157 · doi:10.1214/10-AOS798
[14] Friedman, J., Hastie, T. and Tibshirani, R. (2010). Regularization paths for generalized linear models via coordinate descent. Journal of Statistical Software 33 1-22.
[15] Gahrooei, M. R., Yan, H., Paynabar, K. and Shi, J. (2021). Multiple tensor-on-tensor regression: An approach for modeling processes with heterogeneous sources of data. Technometrics 63 147-159.
[16] Gonzalez, R. C. and Woods, R. E. (2007). Digital image processing (3rd Edition). Pearson.
[17] Gupta, A. K. and Nagar, D. K. (1999). Matrix variate distributions. Chapman and Hall/CRC.
[18] Hall, P. and Miller, H. (2009). Using generalized correlation to effect variable selection in very high dimensional problems. Journal of Computational and Graphical Statistics 18 533-550.
[19] He, X., Wang, L. and Hong, H. G. (2013). Quantile-adaptive model-free variable screening for high-dimensional heterogeneous data. The Annals of Statistics 41 342-369. · Zbl 1295.62053 · doi:10.1214/13-AOS1087
[20] Hoff, P. D. (2015). Multilinear tensor regression for longitudinal relational data. The Annals of Applied Statistics 9 1169-1193. · Zbl 1454.62481 · doi:10.1214/15-AOAS839
[21] Kolda, T. G. and Bader, B. W. (2009). Tensor decompositions and applications. SIAM Review 51 455-500. · Zbl 1173.65029 · doi:10.1137/07070111X
[22] Kong, D., An, B., Zhang, J. and Zhu, H. (2020). L2RM: Low-rank linear regression models for high-dimensional matrix responses. Journal of the American Statistical Association 115 403-424. · Zbl 1437.62357
[23] Li, B., Kim, M. K. and Altman, N. (2010). On dimension folding of matrix- or array-valued statistical objects. The Annals of Statistics 38 1094-1121. · Zbl 1183.62091 · doi:10.1214/09-AOS737
[24] Li, L. and Zhang, X. (2017). Parsimonious tensor response regression. Journal of the American Statistical Association 112 1131-1146.
[25] Li, R., Zhong, W. and Zhu, L. (2012). Feature screening via distance correlation learning. Journal of the American Statistical Association 107 1129-1139. · Zbl 1443.62184
[26] Liu, J., Zhu, C. and Liu, Y. (2020). Smooth compact tensor ring regression. IEEE Transactions on Knowledge and Data Engineering 1-1. · doi:10.1109/TKDE.2020.3037131
[27] Liu, J., Zhu, C., Long, Z., Huang, H. and Liu, Y. (2021). Low-rank tensor ring learning for multi-linear regression. Pattern Recognition 113 107753. · doi:10.1016/j.patcog.2020.107753
[28] Liu, T., Yuan, M. and Zhao, H. (2017). Characterizing spatiotemporal transcriptome of human brain via low rank tensor decomposition. arXiv preprint arXiv:1702.07449.
[29] Liu, Y., Liu, J. and Zhu, C. (2020). Low-rank tensor train coefficient array estimation for tensor-on-tensor regression. IEEE Transactions on Neural Networks and Learning Systems 31 5402-5411.
[30] Llosa-Vite, C. and Maitra, R. (2020). Reduced-rank tensor-on-tensor regression and tensor-variate analysis of variance. arXiv preprint arXiv:2012.10249.
[31] Lock, E. F. (2018). Tensor-on-tensor regression. Journal of Computational and Graphical Statistics 27 638-647. · Zbl 07498939
[32] Lyu, T., Lock, E. F. and Eberly, L. E. (2017). Discriminating sample groups with multi-way data. Biostatistics 18 434-450.
[33] Lyu, X., Sun, W. W., Wang, Z., Liu, H., Yang, J. and Cheng, G. (2020). Tensor graphical model: Non-convex optimization and statistical inference. IEEE Transactions on Pattern Analysis and Machine Intelligence 42 2024-2037. · doi:10.1109/TPAMI.2019.2907679
[34] Mai, Q., Yang, Y. and Zou, H. (2019). Multiclass sparse discriminant analysis. Statistica Sinica 29 97-111. · Zbl 1412.62081
[35] Mai, Q. and Zou, H. (2012). The Kolmogorov filter for variable screening in high-dimensional binary classification. Biometrika 100 229-234. · Zbl 1452.62456
[36] Mai, Q. and Zou, H. (2015). The fused Kolmogorov filter: A nonparametric model-free screening method. The Annals of Statistics 43 1471-1497. · Zbl 1431.62216
[37] Mai, Q., Zou, H. and Yuan, M. (2012). A direct approach to sparse discriminant analysis in ultra-high dimensions. Biometrika 99 29-42. · Zbl 1437.62550 · doi:10.1093/biomet/asr066
[38] Michel, V., Eger, E., Keribin, C., Poline, J.-B. and Thirion, B. (2010). A supervised clustering approach for extracting predictive information from brain activation images. In 2010 IEEE Computer Society Conference on Computer Vision and Pattern Recognition-Workshops 7-14. IEEE.
[39] Michel, V., Gramfort, A., Varoquaux, G., Eger, E. and Thirion, B. (2011). Total variation regularization for fMRI-based prediction of behavior. IEEE Transactions on Medical Imaging 30 1328-1340.
[40] Min, K., Mai, Q. and Zhang, X. (2021). Fast and separable estimation in high-dimensional tensor Gaussian graphical models. Journal of Computational and Graphical Statistics 0 1-7. · Zbl 07546477 · doi:10.1080/10618600.2021.1938086
[41] Pacifico, M. P., Genovese, C., Verdinelli, I. and Wasserman, L. (2007). Scan clustering: A false discovery approach. Journal of Multivariate Analysis 98 1441-1469. · Zbl 1116.62103
[42] Pan, Y., Mai, Q. and Zhang, X. (2019). Covariate-adjusted tensor classification in high dimensions. Journal of the American statistical association 114 1305-1319. · Zbl 1428.62291
[43] Raskutti, G. and Yuan, M. (2015). Convex regularization for high-dimensional tensor regression. arXiv preprint arXiv:1512.01215. · Zbl 1428.62324
[44] Rinaldo, A. (2009). Properties and refinements of the fused lasso. The Annals of Statistics 37 2922-2952. · Zbl 1173.62027 · doi:10.1214/08-AOS665
[45] Shao, J., Wang, Y., Deng, X. and Wang, S. (2011). Sparse linear discriminant analysis by thresholding for high dimensional data. The Annals of Statistics 39 1241-1265. · Zbl 1215.62062 · doi:10.1214/10-AOS870
[46] Sharbrough, F., Chatrian, G. E., Lesser, R., Luders, H., Nuwer, M. and Picton, T. (1991). American electroencephalographic society guidelines for standard electrode position nomenclature. Journal of Clinical Neurophysiology 8 200-202.
[47] Snodgrass, J. G. and Vanderwart, M. (1980). A standardized set of 260 pictures: norms for name agreement, image agreement, familiarity, and visual complexity. Journal of Experimental Psychology: Human Learning and Memory 6 174-215.
[48] Sun, W. W. and Li, L. (2017). STORE: Sparse tensor response regression and neuroimaging analysis. The Journal of Machine Learning Research 18 4908-4944. · Zbl 1442.62773
[49] Sun, W. W. and Li, L. (2019). Dynamic tensor clustering. Journal of the American Statistical Association 114 1894-1907. · Zbl 1428.62260
[50] Sun, W. W., Lu, J., Liu, H. and Cheng, G. (2017). Provable sparse tensor decomposition. Journal of the Royal Statistical Society: Series B (Statistical Methodology) 79 899-916. · Zbl 1411.62158
[51] Székely, G. J., Rizzo, M. L. and Bakirov, N. K. (2007). Measuring and testing dependence by correlation of distances. The Annals of Statistics 35 2769-2794. · Zbl 1129.62059
[52] Wainwright, M. J. (2019). High-dimensional statistics: A non-asymptotic viewpoint. Cambridge Series in Statistical and Probabilistic Mathematics. Cambridge University Press. · Zbl 1457.62011 · doi:10.1017/9781108627771
[53] Wang, X., Zhu, H. and Initiative, A. D. N. (2017). Generalized scalar-on-image regression models via total variation. Journal of the American Statistical Association 112 1156-1168.
[54] Witten, D. M. and Tibshirani, R. (2011). Penalized classification using Fisher’s linear discriminant. Journal of the Royal Statistical Society: Series B (Statistical Methodology) 73 753-772. · Zbl 1228.62079
[55] Xu, P., Zhu, J., Zhu, L. and Li, Y. (2015). Covariance-enhanced discriminant analysis. Biometrica 102 33-45. · Zbl 1345.62091
[56] Zhang, A. and Han, R. (2019). Optimal sparse singular value decomposition for high-dimensional high-order data. Journal of the American Statistical Association 114 1708-1725. · Zbl 1428.62262
[57] Zhang, X. and Li, L. (2017). Tensor envelope partial least-squares regression. Technometrics 59 426-436.
[58] Zhang, X. L., Begleiter, H., Porjesz, B., Wang, W. and Litke, A. (1995). Event related potentials during object recognition tasks. Brain Research Bulletin 38 531-538.
[59] Zhou, H., Li, L. and Zhu, H. (2013). Tensor regression with applications in neuroimaging data analysis. Journal of the American Statistical Association 108 540-552. · Zbl 06195959
[60] Zhu, L.-P., Li, L., Li, R. and Zhu, L.-X. (2011). Model-free feature screening for ultrahigh-dimensional data. Journal of the American Statistical Association 106 1464-1475. · Zbl 1233.62195
This reference list is based on information provided by the publisher or from digital mathematics libraries. Its items are heuristically matched to zbMATH identifiers and may contain data conversion errors. In some cases that data have been complemented/enhanced by data from zbMATH Open. This attempts to reflect the references listed in the original paper as accurately as possible without claiming completeness or a perfect matching.