×

Improving the solution of least squares support vector machines with application to a blast furnace system. (English) Zbl 1264.68129

Summary: The solution of least squares support vector machines (LS-SVMs) is characterized by a specific linear system, that is, a saddle point system. Approaches for its numerical solutions such as conjugate methods and null space methods have been proposed. To speed up the solution of LS-SVM, this paper employs the minimal residual (MINRES) method to solve the above saddle point system directly. Theoretical analysis indicates that the MINRES method is more efficient than the conjugate gradient method and the null space method for solving the saddle point system. Experiments on benchmark data sets show that compared with mainstream algorithms for LS-SVM, the proposed approach significantly reduces the training time and keeps comparable accuracy. To heel, the LS-SVM based on MINRES method is used to track a practical problem originated from blast furnace iron-making process: changing trend prediction of silicon content in hot metal. The MINRES method-based LS-SVM can effectively perform feature reduction and model selection simultaneously, so it is a practical tool for the silicon trend prediction task.

MSC:

68T05 Learning and adaptive systems in artificial intelligence

Software:

UCI-ml; PRMLT
PDFBibTeX XMLCite
Full Text: DOI

References:

[1] V. N. Vapnik, The Nature of Statistical Learning Theory, Springer-Verlag, New York, NY, USA, 2nd edition, 2000. · Zbl 0934.62009
[2] B. Schölkopf and A. Smola, Learning with Kernels: Support Vector Machines, Regularization, Optimization, and Beyond, MIT Press, Cambridge, Mass, USA, 2002.
[3] T. Evgeniou, M. Pontil, and T. Poggio, “Regularization networks and support vector machines,” Advances in Computational Mathematics, vol. 13, no. 1, pp. 1-50, 2000. · Zbl 0939.68098 · doi:10.1023/A:1018946025316
[4] N. Cristianini and J. Shawe-Taylor, An Introduction to Support Vector Machines, Cambridge University, Cambridge, UK, 2000. · Zbl 0994.68074
[5] C. M. Bishop, Pattern Recognition and Machine Learning, vol. 4, Springer, New York, NY, USA, 2006. · Zbl 1107.68072 · doi:10.1007/978-0-387-45528-0
[6] J. A. K. Suykens and J. Vandewalle, “Least squares support vector machine classifiers,” Neural Processing Letters, vol. 9, no. 3, pp. 293-300, 1999. · Zbl 0958.93042
[7] T. Van Gestel, J. A. K. Suykens, B. Baesens et al., “Benchmarking least squares support vector machine classifiers,” Machine Learning, vol. 54, no. 1, pp. 5-32, 2004. · Zbl 1078.68737 · doi:10.1023/B:MACH.0000008082.80494.e0
[8] W. Chu, C. J. Ong, and S. S. Keerthi, “An improved conjugate gradient scheme to the solution of least squares SVM,” IEEE Transactions on Neural Networks, vol. 16, no. 2, pp. 498-501, 2005. · doi:10.1109/TNN.2004.841785
[9] C. C. Paige and M. A. Saunders, “Solutions of sparse indefinite systems of linear equations,” SIAM Journal on Numerical Analysis, vol. 12, no. 4, pp. 617-629, 1975. · Zbl 0319.65025 · doi:10.1137/0712047
[10] L. Jian, C. Gao, L. Li, and J. Zeng, “Application of least squares support vector machines to predict the silicon content in blast furnace hot metal,” ISIJ International, vol. 48, no. 11, pp. 1659-1661, 2008. · doi:10.2355/isijinternational.48.1659
[11] C. Gao, L. Jian, and S. Luo, “Modeling of the thermal state change of blast furnace hearth with support vector machines,” IEEE Transactions on Industrial Electronics, vol. 59, no. 2, pp. 1134-1145, 2012. · doi:10.1109/TIE.2011.2159693
[12] H. A. van der Vorst, Iterative Krylov Methods for Large Linear Systems, vol. 13, Cambridge University Press, Cambridge, UK, 2003. · Zbl 1023.65027 · doi:10.1017/CBO9780511615115
[13] C. Blake and C. Merz, “Uci repository of machine learning databases,” 1998.
[14] C. Gao, J. L. Chen, J. Zeng, X. Liu, and Y. Sun, “A chaos-based iterated multistep predictor for blast furnace ironmaking process,” AIChE Journal, vol. 55, no. 4, pp. 947-962, 2009. · doi:10.1002/aic.11724
[15] L. Jian, C. Gao, and Z. Xia, “A sliding-window smooth support vector regression model for nonlinear blast furnace system,” Steel Research International, vol. 82, no. 3, pp. 169-179, 2011. · doi:10.1002/srin.201000082
[16] Y. W. Chen and C. J. Lin, “Combining SVMs with various feature selection strategies,” Studies in Fuzziness and Soft Computing, vol. 207, pp. 315-324, 2006.
This reference list is based on information provided by the publisher or from digital mathematics libraries. Its items are heuristically matched to zbMATH identifiers and may contain data conversion errors. In some cases that data have been complemented/enhanced by data from zbMATH Open. This attempts to reflect the references listed in the original paper as accurately as possible without claiming completeness or a perfect matching.