Improved multi-view privileged support vector machine.

*(English)*Zbl 1434.68458Summary: Multi-view learning (MVL) concentrates on the problem of learning from the data represented by multiple distinct feature sets. The consensus and complementarity principles play key roles in multi-view modeling. By exploiting the consensus principle or the complementarity principle among different views, various successful support vector machine (SVM)-based multi-view learning models have been proposed for performance improvement. Recently, a framework of learning using privileged information (LUPI) has been proposed to model data with complementary information. By bridging connections between the LUPI paradigm and multi-view learning, we have presented a privileged SVM-based two-view classification model, named PSVM-2V, satisfying both principles simultaneously. However, it can be further improved in these three aspects: (1) fully unleash the power of the complementary information among different views; (2) extend to multi-view case; (3) construct a more efficient optimization solver. Therefore, in this paper, we propose an improved privileged SVM-based model for multi-view learning, termed as IPSVM-MV. It directly follows the standard LUPI model to fully utilize the multi-view complementary information; also it is a general model for multi-view scenario, and an alternating direction method of multipliers (ADMM) is employed to solve the corresponding optimization problem efficiently. Further more, we theoretically analyze the performance of IPSVM-MV from the viewpoints of the consensus principle and the generalization error bound. Experimental results on 75 binary data sets demonstrate the effectiveness of the proposed method; here we mainly concentrate on two-view case to compare with state-of-the-art methods.

##### MSC:

68T05 | Learning and adaptive systems in artificial intelligence |

##### Keywords:

multi-view learning; support vector machine; privileged information; consensus; complementarity
Full Text:
DOI

##### References:

[1] | Anguita, D.; Ghio, A.; Oneto, L.; Ridella, S., A deep connection between the Vapnik-Chervonenkis entropy and the Rademacher complexity, IEEE Transactions on Neural Networks and Learning Systems, 25, 12, 2202-2211 (2014) |

[2] | Bach, F. R.; Lanckriet, G. R.; Jordan, M. I., Multiple kernel learning, conic duality, and the SMO algorithm, (Proceedings of the international conference on machine learning (2004), ACM), 6-13 |

[4] | Bartlett, P.; Mendelson, S., Rademacher and Gaussian complexities: Risk bounds and structural results, Journal of Machine Learning Research (JMLR), 3, 463-482 (2003) · Zbl 1084.68549 |

[6] | Chao, G.; Sun, S., Consensus and complementarity based maximum entropy discrimination for multi-view classification, Information Sciences, 367-368, 296-310 (2016) |

[7] | Chen, X.; Yin, H.; Jiang, F.; Wang, L., Multi-view dimensionality reduction based on Universum learning, Neurocomputing, 275, 2279-2286 (2018) |

[8] | Demšar, J., Statistical comparisons of classifiers over multiple data sets, Journal of Machine Learning Research (JMLR), 7, 1-30 (2006) · Zbl 1222.68184 |

[9] | Deng, N.; Tian, Y.; Zhang, C., Support vector machines: Optimization based theory, algorithms, and extensions (2012), CRC press |

[11] | Eidenberger, H., Statistical analysis of content-based MPEG-7 descriptors for image retrieva, Multimedia Systems, 10, 2, 84-97 (2004) |

[13] | Fukumizu, K.; Bach, F.; Gretton, A., Statistical consistency of kernel canonical correlation analysis, Journal of Machine Learning Research (JMLR), 8, 2007, 361-383 (2007) · Zbl 1222.62063 |

[14] | Hsieh, W. W., Nonlinear canonical correlation analysis by neural networks, Neural Networks, 13, 10, 1095-1105 (2000) |

[15] | Huang, C.; Chung, F.-l.; Wang, S., Multi-view L2-SVM and its multi-view core vector machine, Neural Networks, 75, 110-125 (2016) |

[18] | Lapin, M.; Hein, M.; Schiele, B., Learning using privileged information: SVM+ and weighted SVM, Neural Networks, 53, 95-108 (2014) · Zbl 1308.68098 |

[19] | Li, J.; Nigel, A.; Tao, D.; Li, X., Multitraining support vector machine for image retrieval, IEEE Transactions on Image Processing, 15, 11, 3597-3601 (2006) |

[21] | Liu, A. A.; Xu, N.; Nie, W. Z.; Su, Y. T.; Wong, Y.; Kankanhalli, M., Benchmarking a multimodal and multiview and interactive dataset for human action recognition, IEEE Transactions on Cybernetics, 47, 7, 1781-1794 (2017) |

[23] | Ménard, O.; Frezza-Buet, H., Model of multi-modal cortical processing: Coherent learning in self-organizing modules, Neural Networks, 18, 5, 646-655 (2005) |

[26] | Peng, J.; Aved, A. J.; Seetharaman, G.; Palaniappan, K., Multiview boosting with information propagation for classification, IEEE Transactions on Neural Networks and Learning Systems, 29, 3, 657-669 (2018) |

[27] | Rakotomamonjy, A.; Bach, F. R.; Canu, S.; Grandvalet, Y., Simplemkl, Journal of Machine Learning Research (JMLR), 9, 3, 2491-2521 (2008) · Zbl 1225.68208 |

[28] | Sonnenburg, S.; Rätsch, G.; Schäfer, C.; Schölkopf, B., Large scale multiple kernel learning, Journal of Machine Learning Research (JMLR), 7, 1531-1565 (2006) · Zbl 1222.90072 |

[29] | Sun, J.; Keates, S., Canonical correlation analysis on data with censoring and error information, IEEE Transactions on Neural Networks and Learning Systems, 24, 12, 1909-1919 (2013) |

[30] | Sun, S., A survey of multi-view machine learning, Neural Computing and Applications, 23, 7-8, 2031-2038 (2013) |

[31] | Sun, S.; Xie, X.; Yang, M., Multiview uncorrelated discriminant analysis, IEEE Transactions on Cybernetics, 46, 12, 3272-3284 (2016) |

[32] | Tang, J.; Tian, Y., A multi-kernel framework with nonparallel support vector machine, Neurocomputing, 266, 226-238 (2017) |

[33] | Tang, J.; Tian, Y.; Zhang, P.; Liu, X., Multiview privileged support vector machines, IEEE Transactions on Neural Networks and Learning Systems (2017) |

[34] | Vapnik, V.; Izmailov, R., Learning using privileged information: Similarity control and knowledge transfer, Journal of Machine Learning Research (JMLR), 16, 2023-2049 (2015) · Zbl 1351.68240 |

[35] | Vapnik, V.; Vashist, A., A new learning paradigm: Learning using privileged information, Neural Networks, 22, 5, 544-557 (2009) · Zbl 1335.68212 |

[36] | Vía, J.; Santamaría, I.; Pérez, J., A learning algorithm for adaptive canonical correlation analysis of several data sets, Neural Networks, 20, 1, 139-152 (2007) · Zbl 1158.68459 |

[38] | Wang, Y.; Zhang, W.; Wu, L.; Lin, X.; Zhao, X., Unsupervised metric fusion over multiview data by graph random walk-based cross-view diffusion, IEEE Transactions on Neural Networks and Learning Systems, 28, 1, 57-70 (2017) |

[40] | Zhao, J.; Xie, X.; Xu, X.; Sun, S., Multi-view learning overview: Recent progress and new challenges, Information Fusion, 38, 43-54 (2017) |

[41] | Zong, L.; Zhang, X.; Zhao, L.; Yu, H.; Zhao, Q., Multi-view clustering via multi-manifold regularized non-negative matrix factorization, Neural Networks, 88, 74-89 (2017) |

This reference list is based on information provided by the publisher or from digital mathematics libraries. Its items are heuristically matched to zbMATH identifiers and may contain data conversion errors. It attempts to reflect the references listed in the original paper as accurately as possible without claiming the completeness or perfect precision of the matching.