Constrained dual graph regularized orthogonal nonnegative matrix tri-factorization for co-clustering.

*(English)*Zbl 1435.68259Summary: Coclustering approaches for grouping data points and features have recently been receiving extensive attention. In this paper, we propose a constrained dual graph regularized orthogonal nonnegative matrix trifactorization (CDONMTF) algorithm to solve the coclustering problems. The new method improves the clustering performance obviously by employing hard constraints to retain the priori label information of samples, establishing two nearest neighbor graphs to encode the geometric structure of data manifold and feature manifold, and combining with biorthogonal constraints as well. In addition, we have also derived the iterative optimization scheme of CDONMTF and proved its convergence. Clustering experiments on 5 UCI machine-learning data sets and 7 image benchmark data sets show that the achievement of the proposed algorithm is superior to that of some existing clustering algorithms.

##### MSC:

68T05 | Learning and adaptive systems in artificial intelligence |

15A23 | Factorization of matrices |

62H30 | Classification and discrimination; cluster analysis (statistical aspects) |

68R10 | Graph theory (including graph drawing) in computer science |

PDF
BibTeX
XML
Cite

\textit{S. Ge} et al., Math. Probl. Eng. 2019, Article ID 7565640, 17 p. (2019; Zbl 1435.68259)

Full Text:
DOI

##### References:

[1] | Zhao, K.; Shi, G.; Huang, S., Multi-graph fusion for multi-view spectral clustering, Knowledge-Based Systems (2019) |

[2] | Zhao, K.; Zhao, X.; Peng, C., Partition level multiview subspace clustering, Neural Networks, 122, 279-288 (2019) |

[3] | Kang, Z.; Wen, L.; Chen, W.; Xu, Z., Low-rank kernel learning for graph-based clustering, Knowledge-Based Systems, 163, 510-517 (2019) |

[4] | Karimzadeh, S.; Olafsson, S., Data clustering using proximity matrices with missing values, Expert Systems with Applications, 126, 265-276 (2019) |

[5] | Melendez-Melendez, G.; Cruz-Paz, D.; Carrasco-Ochoa, J. A.; Martínez-Trinidad, J. F., An improved algorithm for partial clustering, Expert Systems with Applications, 121, 282-291 (2019) |

[6] | Lu, N.; Miao, H., Structure constrained nonnegative matrix factorization for pattern clustering and classification, Neurocomputing, 171, 400-411 (2016) |

[7] | Wang, S.; Huang, A., Penalized nonnegative matrix tri-factorization for co-clustering, Expert Systems with Applications, 78, 64-73 (2017) |

[8] | Hussain, S. F.; Ramazan, M., Biclustering of human cancer microarray data using co-similarity based co-clustering, Expert Systems with Applications, 55, 520-531 (2016) |

[9] | Lee, D. D.; Seung, H. S., Learning the parts of objects by non-negative matrix factorization, Nature, 401, 6755, 788-791 (1999) · Zbl 1369.68285 |

[10] | Zhu, W.; Yan, Y.; Peng, Y., Topological structure regularized nonnegative matrix factorization for image clustering, Neural Computing and Applications, 31, 11, 7381-7399 (2019) |

[11] | Guan, N.; Liu, T.; Zhang, Y.; Tao, D.; Davis, L. S., Truncated cauchy non-negative matrix factorization, IEEE Transactions on Pattern Analysis and Machine Intelligence, 41, 1, 246-259 (2019) |

[12] | Lu, G.-F.; Wang, Y.; Zou, J., Low-rank matrix factorization with adaptive graph regularizer, IEEE Transactions on Image Processing, 25, 5, 2196-2205 (2016) · Zbl 1408.94443 |

[13] | Wang, T.; Liu, Y., Multiview community discovery algorithm via nonnegative factorization matrix in heterogeneous networks, Mathematical Problems in Engineering, 2017 (2017) |

[14] | Wang, Y.-X.; Zhang, Y.-J., Nonnegative matrix factorization: a comprehensive review, IEEE Transactions on Knowledge and Data Engineering, 25, 6, 1336-1353 (2013) |

[15] | Xie, Q.; Zhu, H., Image retrieval based on multiview constrained nonnegative matrix factorization and Gaussian mixture model spectral clustering method, Mathematical Problems in Engineering, 2016 (2016) · Zbl 1400.62139 |

[16] | Logothetis, N.; Sheinberg, D., Visual object recognition, Annual Review of Neuroscience, 19, 577-621 (1996) |

[17] | Wachsmuth, E.; Oram, M. W.; Perrett, D. I., Recognition of objects and their component parts: responses of single units in the temporal cortex of the macaque, Cerebral Cortex, 4, 5, 509-522 (1994) |

[18] | Deng, C.; He, X.; Han, J.; Huang, T. S., Graph regularized nonnegative matrix factorization for data representation, IEEE Transactions on Pattern Analysis and Machine Intelligence, 33, 8, 1548-1560 (2011) |

[19] | Salehani, Y. E.; Arabnejad, E.; Cheriet, M., Graph and sparse-based robust nonnegative block value decomposition for clustering, IEEE Journal of Selected Topics in Signal Processing, 12, 6, 1561-1574 (2018) |

[20] | Wang, C.; Song, X.; Zhang, J., Graph regularized nonnegative matrix factorization with sample diversity for image representation, Engineering Applications of Artificial Intelligence, 68, 32-39 (2018) |

[21] | Yang, M.; Shang, R.; Jiao, L.; Zhang, W.; Yuan, Y.; Yang, S., Feature selection based dual-graph sparse non-negative matrix factorization for local discriminative clustering, Neurocomputing, 290, 87-99 (2018) |

[22] | Peng, C.; Zhao, K.; Hu, Y.; Cheng, J.; Cheng, Q., Nonnegative matrix factorization with integrated graph and feature learning, ACM Transactions on Intelligent Systems and Technology, 8, 3, 1-29 (2017) |

[23] | Shang, R.; Meng, Y.; Wang, W.; Shang, F.; Jiao, L., Local discriminative based sparse subspace learning for feature selection, Pattern Recognition, 92, 219-230 (2019) |

[24] | Belkin, M.; Niyogi, P.; Sindhwani, V.; Bartlett, P., Manifold regularization: a geometric framework for learning from examples, Journal of Machine Learning Research, 7, 1, 2399-2434 (2006) · Zbl 1222.68144 |

[25] | Yang, L.; Wang, L.; Gao, Y.; Sun, Q.; Zhao, T., A convex relaxation framework for a class of semi-supervised learning methods and its application in pattern recognition, Engineering Applications of Artificial Intelligence, 35, 35, 335-344 (2014) |

[26] | Zhou, D.; Bousquet, O.; Lal, T. N.; Weston, J.; Schölkopf, B., Learning with local and global consistency, Proceedings of the 16th International Conference on Neural Information Processing Systems, NIPS’03, MIT Press |

[27] | Liu, H.; Wu, Z.; Cai, D.; Huang, T. S., Constrained nonnegative matrix factorization for image representation, IEEE Transactions on Pattern Analysis and Machine Intelligence, 34, 7, 1299-1311 (2012) |

[28] | Meng, Y.; Shang, R.; Shang, F.; Jiao, L.; Yang, S.; Stolkin, R., Semi-supervised graph regularized deep NMF with bi-orthogonal constraints for data representation, IEEE Transactions on Neural Networks and Learning Systems, 1-14 (2019) |

[29] | Bisson, G.; Grimal, C., Co-clustering of multi-view datasets: a parallelizable approach, Proceedings of the 2012 IEEE 12th International Conference on Data Mining |

[30] | Hussain, S. F.; Haris, M., A k-means based co-clustering (kCC) algorithm for sparse, high dimensional data, Expert Systems with Applications, 118, 20-34 (2019) |

[31] | Koҫ, S. Ş.; Özer, M.; Toroslu, İ. H.; Davulcu, H.; Jordan, J., Triadic co-clustering of users, issues and sentiments in political tweets, Expert Systems with Applications, 100, 79-94 (2018) |

[32] | Dhillon, I. S.; Mallela, S.; Modha, D. S., Information-theoretic co-clustering, Proceedings of the Ninth ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, KDD’03, ACM |

[33] | Labiod, L.; Mohamed, N., Co-clustering under nonnegative matrix tri-factorization, Proceedings of the 18th International Conference on Neural Information Processing—Volume Part II, ICONIP’11, Springer-Verlag |

[34] | Del Buono, N.; Pio, G., Non-negative matrix tri-factorization for co-clustering: an analysis of the block matrix, Information Sciences, 301, 13-26 (2015) |

[35] | Shang, F.; Jiao, L. C.; Wang, F., Graph dual regularization non-negative matrix factorization for co-clustering, Pattern Recognition, 45, 6, 2237-2250 (2012) · Zbl 1234.68356 |

[36] | Meng, Y.; Shang, R.; Jiao, L.; Zhang, W.; Yang, S., Dual-graph regularized non-negative matrix factorization with sparse and orthogonal constraints, Engineering Applications of Artificial Intelligence, 69, 24-35 (2018) |

[37] | Gu, Q.; Zhou, J., Co-clustering on manifolds, Proceedings of the 15th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, KDD’09, ACM |

[38] | Zhao, K.; Pan, H.; Hoi, S. C. H.; Xu, Z., Robust graph learning from noisy data, IEEE Transactions on Cybernetics, 1-11 (2019) |

[39] | Kang, Z.; Peng, C.; Cheng, Q., Kernel-driven similarity learning, Neurocomputing, 267, 210-219 (2017) |

[40] | Kang, Z.; Xu, H.; Wang, B.; Zhu, H.; Xu, Z., Clustering with similarity preserving, Neurocomputing, 365, 211-218 (2019) |

[41] | Kuhn, H. W.; Tucker, A. W., Nonlinear programming, Proceedings of the Second Berkeley Symposium on Mathematical Statistics and Probability |

[42] | Lee, D. D.; Sebastian Seung, H.; Leen, T. K.; Dietterich, T. G.; Tresp, V., Algorithms for non-negative matrix factorization, Advances in Neural Information Processing Systems, 13, 556-562 (2001), Cambridge, MA, USA: MIT Press, Cambridge, MA, USA |

[43] | Kivinen, J.; Warmuth, M. K., Exponentiated gradient versus gradient descent for linear predictors, Information and Computation, 132, 1, 1-63 (1997) · Zbl 0872.68158 |

[44] | Cai, D.; He, X.; Han, J., Document clustering using locality preserving indexing, IEEE Transactions on Knowledge and Data Engineering, 17, 12, 1624-1637 (2005) |

[45] | Forestier, G.; Wemmert, C.; Gançarski, P.; Bi, Y.; Williams, M.-A., Background knowledge integration in clustering using purity indexes, Knowledge Science, Engineering and Management, 28-38 (2010), Berlin, Germany: Springer Berlin Heidelberg, Berlin, Germany |

[46] | Cui, G.; Li, X.; Dong, Y., Subspace clustering guided convex nonnegative matrix factorization, Neurocomputing, 292, 38-48 (2018) |

[47] | Hong, S.; Choi, J.; Feyereisl, J.; Han, B.; Davis, L. S., Joint image clustering and labeling by matrix factorization, IEEE Transactions on Pattern Analysis and Machine Intelligence, 38, 7, 1411-1424 (2016) |

[48] | Yang, Z.; Zhang, Y.; Xiang, Y.; Yan, W.; Xie, S., Non-negative matrix factorization with dual constraints for image clustering, IEEE Transactions on Systems, Man, and Cybernetics: Systems, 1-10 (2018) |

[49] | Liu, J.; Cheng, Y.; Wang, X.; Cui, X.; Kong, Y.; Du, J., Low rank subspace clustering via discrete constraint and hypergraph regularization for tumor molecular pattern discovery, IEEE/ACM Transactions on Computational Biology and Bioinformatics, 15, 5, 1500-1512 (2018) |

[50] | Lyons, M. J.; Budynek, J.; Akamatsu, S., Automatic classification of single facial images, IEEE Transactions on Pattern Analysis and Machine Intelligence, 21, 12, 1357-1362 (1999) |

[51] | Nene, S. A.; Nayar, S. K.; Murase, H., Columbia object image library (coil-20), Technical Report CUCS-005-96, New York City, NY, USA: Department of Computer Science, Columbia University, New York City, NY, USA |

[52] | Hirzer, M.; Beleznai, C.; Roth, P. M.; Bischof, H., Person re-identification by descriptive and discriminative classification, Proceedings of the Scandinavian Conference on Image Analysis (SCIA) |

This reference list is based on information provided by the publisher or from digital mathematics libraries. Its items are heuristically matched to zbMATH identifiers and may contain data conversion errors. It attempts to reflect the references listed in the original paper as accurately as possible without claiming the completeness or perfect precision of the matching.