×

Knowledge transfer in SVM and neural networks. (English) Zbl 1422.68207

Summary: The paper considers general machine learning models, where knowledge transfer is positioned as the main method to improve their convergence properties. Previous research was focused on mechanisms of knowledge transfer in the context of SVM framework; the paper shows that this mechanism is applicable to neural network framework as well. The paper describes several general approaches for knowledge transfer in both SVM and ANN frameworks and illustrates algorithmic implementations and performance of one of these approaches for several synthetic examples.

MSC:

68T05 Learning and adaptive systems in artificial intelligence
68T30 Knowledge representation
PDFBibTeX XMLCite
Full Text: DOI

References:

[1] Brachman, R., Levesque, H.: Knowledge Representation and Reasoning. Morgan Kaufmann, San Francisco (2004) · Zbl 1341.68228
[2] Fouad, S., Tino, P., Raychaudhury, S., Schneider, P.: Incorporating privileged information through metric learning. IEEE Transactions on Neural Networks and Learning Systems 24, 1086-1098 (2013) · doi:10.1109/TNNLS.2013.2251470
[3] Demuth, H., Beale, M.: Neural Network Toolbox for Use with MATLAB. Mathworks, Inc., Natick, MA (2000)
[4] Hagan, M., Demuth, H., Beale, M.: Neural Network Design. PWS Publishing Co, Boston, MA (1996)
[5] Heaton, J.: Introduction to Neural Networks for Java. Heaton Research Inc (2008) · Zbl 1351.68240
[6] Ilin, R., Streltsov, S., Izmailov, R.: Learning with privileged information for improved target classification. International Journal of Monitoring and Surveillance Technologies Research 2(3), 5-66 (2014) · doi:10.4018/IJMSTR.2014070103
[7] Kolmogorov, A.: Mathematics as a Profession. Moscow, Nauka (1988). (in Russian) · Zbl 0649.00029
[8] LeCun, Y., Bottou, L., Orr, G., Müller, K.: Efficient BackProp. In: Orr, G., Müller, K. (eds.) Neural Networks: Tricks of the Trade, pp 5-50. Springer, Heidelberg (1998)
[9] Lichman, M.: UCI Machine Learning Repository [http://archive.ics.uci.edu/ml]. Irvine, CA: University of California School of Information and Computer Science (2013)
[10] Pechyony, D., Izmailov, R., Vashist, A., Vapnik, V.: SMO-style algorithms for learning using privileged information. In: 2010 International Conference on Data Mining, pp 235-241 (2010)
[11] Ribeiro, B., Silva, C., Chen, N., Vieirac, A., Carvalho das Nevesd, J.: Enhanced default risk models with SVM + . Expert Syst. Appl. 39, 10140-10152 (2012) · doi:10.1016/j.eswa.2012.02.142
[12] Sharmanska, V., Lampert, C.: Learning to rank using privileged information. In: 2013 IEEE International Conference on Computer Vision (ICCV), pp 825-832 (2013)
[13] Vapnik, V.: The Nature of Statistical Learning Theory. Springer, New York (1995) · Zbl 0833.62008 · doi:10.1007/978-1-4757-2440-0
[14] Vapnik, V.: Statistical Learning Theory. Wiley, New York (1998) · Zbl 0935.62007
[15] Vapnik, V.: Estimation of Dependencies Based on Empirical Data, 2nd edn. Springer, New York (2006) · Zbl 1118.62002
[16] Vapnik, V., Vashist, A.: A new learning paradigm: learning using privileged information. Neural Netw. 22, 546-557 (2009) · Zbl 1335.68212 · doi:10.1016/j.neunet.2009.06.042
[17] Vapnik, V., Izmailov, R.: Learning with intelligent teacher: similarity control and knowledge transfer. In: Statistical Learning and Data Sciences. LNAI 9047, pp 3-32. Springer, London (2015) · Zbl 1351.68240
[18] Vapnik, V., Izmailov, R.: Learning using privileged information: similarity control and knowledge transfer. J. Mach. Learn. Res. 16, 2023-2049 (2015) · Zbl 1351.68240
[19] Yang, H., Patras, I.: Privileged information-based conditional regression forest for facial feature detection. In: 2013 IEEE International Conference on Automatic Face and Gesture Recognition, pp 1-6 (2013)
This reference list is based on information provided by the publisher or from digital mathematics libraries. Its items are heuristically matched to zbMATH identifiers and may contain data conversion errors. In some cases that data have been complemented/enhanced by data from zbMATH Open. This attempts to reflect the references listed in the original paper as accurately as possible without claiming completeness or a perfect matching.