×

zbMATH — the first resource for mathematics

Coupling privileged kernel method for multi-view learning. (English) Zbl 1443.68157
Summary: Multi-view learning concentrates on fully using the data collected from diverse domains or obtained from various feature extractors to learn effectively. The consensus and complementarity principles provide significant guidance in multi-view modeling. Many support vector machine (SVM)-based multi-view learning models have been proposed, which mainly follow the consensus principle through exploiting the label correlation with regularization terms. In this paper, we propose a simple yet effective coupling privileged kernel method for multi-view learning, termed as MCPK. The coupling term included in the primal objective allows the combination of the errors from all views to be minimized, which guarantees the consensus principle. Similar to our previous work PSVM-2V, MCPK realizes the complementarity principle by applying the learning using privileged information (LUPI) paradigm. The proposed model not only fully integrates the information from all views in the learning process, but also maintains the characteristic of different views to some extent. We employ the standard quadratic programming solver to solve MCPK. Further more, we theoretically analyze the performance of MCPK from the viewpoints of the generalization capability and the PSVM-2V and SVM-2K models. Experimental results demonstrate that MCPK compares more favorably than other state-of-the-art multi-view algorithms in terms of classification accuracy and efficiency.
MSC:
68T05 Learning and adaptive systems in artificial intelligence
62H30 Classification and discrimination; cluster analysis (statistical aspects)
Software:
SimpleMKL; SHOGUN
PDF BibTeX XML Cite
Full Text: DOI
References:
[1] Bach, F. R.; Lanckriet, G. R.; Jordan, M. I., Multiple kernel learning, conic duality, and the smo algorithm, Proceedings of the International Conference on Machine Learning, 6-13 (2004), ACM
[2] Balcan, M.; Blum, A.; Yang, K., Co-training and expansion: towards bridging theory and practice, Proceedings of the Annual Conference on Neural Information Processing Systems, 29-58 (2004)
[3] Bang, S.; Kang, J.; Jhun, M.; Kim, E., Hierarchically penalized support vector machine with grouped variables, Int. J. Mach. Learn. Cybern., 8, 4, 1211-1221 (2017)
[4] Bartlett, P.; Mendelson, S., Rademacher and gaussian complexities: risk bounds and structural results, J. Mach. Learn. Res., 3, 463-482 (2003) · Zbl 1084.68549
[5] Blum, A.; Mitchell, T., Combining labeled and unlabeled data with co-training, Proceedings of the Annual Conference on Computational Learning Theory, 92-100 (1998)
[6] Chao, G.; Sun, S., Consensus and complementarity based maximum entropy discrimination for multi-view classification, Inf. Sci., 367-368, 296-310 (2016) · Zbl 1428.68232
[7] Chen, X.; Yin, H.; Jiang, F.; Wang, L., Multi-view dimensionality reduction based on universum learning, Neurocomputing, 275, 2279-2286 (2018)
[8] Das, S. P.; Padhy, S., A novel hybrid model using teaching-learning-based optimization and a support vector machine for commodity futures index forecasting, Int. J. Mach. Learn. Cybern., 9, 1, 97-111 (2018)
[9] Demšar, J., Statistical comparisons of classifiers over multiple data sets, J. Mach. Learn. Res., 7, 1-30 (2006) · Zbl 1222.68184
[10] Deng, N.; Tian, Y.; Zhang, C., Support Vector Machines: Optimization Based Theory, Algorithms, and Extensions (2012), CRC press
[11] Deshmukh, A. B.; Rani, N. U., Fractional-grey wolf optimizer-based kernel weighted regression model for multi-view face video super resolution, Int. J. Mach. Learn. Cybern., 1-19 (2017)
[12] Eidenberger, H., Statistical analysis of content-based mpeg-7 descriptors for image retrieval, Multimed. Syst., 10, 2, 84-97 (2004)
[13] Farquhar, J.; Hardoon, D.; Meng, H.; Shawe-taylor, J.; Szedmak, S., Two view learning: Svm-2k, theory and practice, Proceedings of the Annual Conference on Neural Information Processing Systems, 355-362 (2005)
[14] Fukumizu, K.; Bach, F.; Gretton, A., Statistical consistency of kernel canonical correlation analysis, J. Mach. Learn. Res., 8, 361-383 (2007) · Zbl 1222.62063
[15] Gao, X.; Fan, L.; Xu, H., Multiple rank multi-linear kernel support vector machine for matrix data classification, Int. J. Mach. Learn. Cybern., 9, 2, 251-261 (2018)
[16] Hardoon, D. R.; Shawe-Taylor, J., Convergence analysis of kernel canonical correlation analysis: theory and practice, Mach. Learn., 74, 1, 23-38 (2009)
[17] Houthuys, L.; Langone, R.; Suykens, J. A.K., Multi-view kernel spectral clustering, Inf. Fusion, 44, 46-56 (2018)
[18] Houthuys, L.; Langone, R.; Suykens, J. A.K., Multi-view least squares support vector machines classification, Neurocomputing, 282, 78-88 (2018)
[19] Kumar, A.; Daumé, H., A co-training approach for multi-view spectral clustering, Proceedings of the International Conference on Machine Learning, 393-400 (2011)
[20] Li, B.; Yuan, C.; Xiong, W.; Hu, W.; Peng, H.; Ding, X.; Maybank, S., Multi-view multi-instance learning based on joint sparse representation and multi-view dictionary learning, IEEE Trans. Pattern Anal. Mach. Intell., 39, 12, 2554-2560 (2017)
[21] Li, J.; Nigel, A.; Tao, D.; Li, X., Multitraining support vector machine for image retrieval, IEEE Trans. Image Process., 15, 11, 3597-3601 (2006)
[22] Li, Y.; Shawe-Taylor, J., Advanced learning algorithms for cross-language patent retrieval and classification, Information Processing and Management, 43, 5, 1183-1199 (2007)
[23] Liu, A. A.; Xu, N.; Nie, W. Z.; Su, Y. T.; Wong, Y.; Kankanhalli, M., Benchmarking a multimodal and multiview and interactive dataset for human action recognition, IEEE Trans. Cybern., 47, 7, 1781-1794 (2017)
[24] Mao, L.; Sun, S., Soft margin consistency based scalable multi-view maximum entropy discrimination, Proceedings of the International Joint Conference on Artificial Intelligence, 1839-1845 (2016)
[25] Ménard, O.; Frezza-Buet, H., Model of multi-modal cortical processing: coherent learning in self-organizing modules, Neural Networks, 18, 5, 646-655 (2005)
[26] Peng, J.; Aved, A. J.; Seetharaman, G.; Palaniappan, K., Multiview boosting with information propagation for classification, IEEE Trans. Neural Netw. Learn. Syst., 29, 3, 657-669 (2018)
[27] Peng, J.; Luo, P.; Guan, Z.; Fan, J., Graph-regularized multi-view semantic subspace learning, Int. J. Mach. Learn. Cybern., 1-17 (2017)
[28] Rakotomamonjy, A.; Bach, F. R.; Canu, S.; Grandvalet, Y., Simplemkl, J. Mach. Learn. Res., 9, 3, 2491-2521 (2008) · Zbl 1225.68208
[29] Shen, X.; Niu, L.; Qi, Z.; Tian, Y., Support vector machine classifier with truncated pinball loss, Pattern Recognit., 68, 199-210 (2017)
[30] Sonnenburg, S.; Rätsch, G.; Schäfer, C.; Schölkopf, B., Large scale multiple kernel learning, J. Mach. Learn. Res., 7, 1531-1565 (2006) · Zbl 1222.90072
[31] Sun, J.; Keates, S., Canonical correlation analysis on data with censoring and error information, IEEE Trans. Neural Netw. Learn. Syst., 24, 12, 1909-1919 (2013)
[32] Sun, S., Multi-view Laplacian support vector machines, International Conference on Advanced Data Mining and Applications, 209-222 (2011), Springer
[33] Sun, S.; Chao, G., Multi-view maximum entropy discrimination, Proceedings of the International Joint Conference on Artificial Intelligence, 1706-1712 (2013), AAAI Press
[34] Sun, S.; Shawe-Taylor, J., Sparse semi-supervised learning using conjugate functions, J. Mach. Learn. Res., 11, 2423-2455 (2010) · Zbl 1242.68251
[35] Sun, S.; Xie, X.; Yang, M., Multiview uncorrelated discriminant analysis, IEEE Trans. Cybern., 46, 12, 3272-3284 (2016)
[36] Tang, J.; Tian, Y., A multi-kernel framework with nonparallel support vector machine, Neurocomputing, 266, 226-238 (2017)
[37] Tang, J.; Tian, Y.; Zhang, P.; Liu, X., Multiview privileged support vector machines, IEEE Trans. Neural Netw. Learn. Syst., 29, 8, 3463-3477 (2018)
[38] Tian, Y.; Shi, Y.; Liu, X., Recent advances on support vector machines research, Technol. Econ. Dev. Econ., 18, 1, 5-33 (2012)
[39] Wang, W.; Zhou, Z., A new analysis of co-training, Proceedings of the International Conference on Machine Learning, 1135-1142 (2010)
[40] Wang, Y.; Zhang, W.; Wu, L.; Lin, X.; Zhao, X., Unsupervised metric fusion over multiview data by graph random walk-based cross-view diffusion, IEEE Trans. Neural Netw. Learn. Syst., 28, 1, 57-70 (2017)
[42] Xu, C.; Tao, D.; Xu, C., Multi-view intact space learning, IEEE Trans. Pattern Anal. Mach. Intell., 37, 12, 2531-2544 (2015)
[43] Xue, Z.; Li, G.; Huang, Q., Joint multi-view representation and image annotation via optimal predictive subspace learning, Inf. Sci., 451-452, 180-194 (2018)
[44] Yang, X.; Liu, W.; Tao, D.; Cheng, J., Canonical correlation analysis networks for two-view image recognition, Inf. Sci., 385-386, 338-352 (2017)
[45] Yang, Z.-M.; Wu, H.-J.; Li, C.-N.; Shao, Y.-H., Least squares recursive projection twin support vector machine for multi-class classification, Int. J. Mach. Learn. Cybern., 7, 3, 411-426 (2016)
[46] Zheng, W.; Zhou, X.; Zou, C.; Zhao, L., Facial expression recognition using kernel canonical correlation analysis, IEEE Trans. Neural Netw., 17, 1, 233-238 (2006)
[47] Zhu, P.; Zhu, W.; Hu, Q.; Zhang, C.; Zuo, W., Subspace clustering guided unsupervised feature selection, Pattern Recognit., 66, C, 364-374 (2017)
[48] Zhuang, F.; Karypis, G.; Ning, X.; He, Q.; Shi, Z., Multi-view learning via probabilistic latent semantic analysis, Inf. Sci., 199, 20-30 (2012)
This reference list is based on information provided by the publisher or from digital mathematics libraries. Its items are heuristically matched to zbMATH identifiers and may contain data conversion errors. It attempts to reflect the references listed in the original paper as accurately as possible without claiming the completeness or perfect precision of the matching.