×

zbMATH — the first resource for mathematics

\(K\)-plane regression. (English) Zbl 1357.62232
Summary: In this paper, we present a novel algorithm for piecewise linear regression which can learn continuous as well as discontinuous piecewise linear functions. The main idea is to repeatedly partition the data and learn a linear model in each partition. The proposed algorithm is similar in spirit to \(k\)-means clustering algorithm. We show that our algorithm can also be viewed as a special case of an EM algorithm for maximum likelihood estimation under a reasonable probability model. We empirically demonstrate the effectiveness of our approach by comparing its performance with that of the state of art algorithms on various datasets.
MSC:
62J02 General nonlinear regression
62H30 Classification and discrimination; cluster analysis (statistical aspects)
PDF BibTeX XML Cite
Full Text: DOI
References:
[1] Amaldi, E.; Mattavelli, M., The MIN PFS problem and piecewise linear model estimation, Discrete Appl. Math., 118, 1-2, 115-143, (2002) · Zbl 0995.90076
[2] S. An, H. Shi, Q. Hu, X. Li, J. Dang, Fuzzy rough regression with application to wind speed prediction, Inform. Sci. (0), 16 April 2014.
[3] A. Asuncion, D. Newman, UCI Machine Learning Repository, University of California, Irvine, School of Information and Computer Sciences, 2007. <http://www.ics.uci.edu/∼mlearn/MLRepository.html>.
[4] Bagirov, A. M.; Clausen, C.; Kohler, M., Estimation of a regression function by maxima of minima of linear functions, IEEE Trans. Inf. Theory, 55, 2, 833-845, (2009) · Zbl 1367.62085
[5] Bagirov, A. M.; Ugon, J.; Mirzayeva, H., Nonsmooth nonconvex optimization approach to clusterwise linear regression problems, Eur. J. Oper. Res., 229, 1, 132-142, (2013) · Zbl 1317.90242
[6] Bazaraa, M. S.; Sherali, H. D.; Shetty, C. M., Nonlinear programming theory and algorithms, (2004), John Wiley & Sons Inc. · Zbl 1140.90040
[7] A. Bemporad, A. Garulli, S. Paoletti, A. Vicino, A greedy approach to identification of piecewise affine models, in: Proceedings of the 6th International Conference on Hybrid Systems: Computation and Control (HSCC), Prague, Czech Republic, 2003, pp. 97-112. · Zbl 1032.93500
[8] Bemporad, A.; Garulli, A.; Paoletti, S.; Vicino, A., A bounded error approach to piecewise affine system identification, IEEE Trans. Autom. Control, 50, 10, 1567-1580, (2005) · Zbl 1365.93100
[9] Bishop, C. M., Pattern recognition and machine learning (information science and statistics), (2006), Springer
[10] Breiman, L., Hinging hyperplane for regression, classification and function approximation, IEEE Trans. Inform. Theory, 39, 3, 999-1013, (1993) · Zbl 0793.62031
[11] Breiman, L.; Friedman, J.; Olshen, R.; Stone, C., Classification and Regression Trees, Statistics/Probability Series, (1984), Wadsworth Publishing Company Belmont, California, USA · Zbl 0541.62042
[12] Carbonneau, R. A.; Caporossi, G.; Hansen, P., Globally optimal clusterwise regression by mixed logical-quadratic programming, Eur. J. Oper. Res., 212, 1, 213-222, (2011)
[13] C.-C. Chang, C.-J. Lin, LIBSVM: A Library for Support Vector Machines, Software, 2001. <http://www.csie.ntu.edu.tw/∼cjlin/libsvm/>.
[14] DeSarbo, W. S.; Cron, W. L., A maximum likelihood methodology for clusterwise linear regression, J. Classif., 5, 2, 249-282, (1988) · Zbl 0692.62052
[15] D’Urso, P.; Massari, R.; Santoro, A., A class of fuzzy clusterwise regression models, Inf. Sci., 180, 24, 4737-4762, (2010) · Zbl 1204.62112
[16] Hastie, T.; Tibshirani, R.; Friedman, J., Elements of statistical learning theory, (2001), Springer
[17] Haykin, S., Neural networks: A comprehensive foundation, (2006), Pearson Education · Zbl 0828.68103
[18] Hung, K.-C.; Lin, K.-P., Long-term business cycle forecasting through a potential intuitionistic fuzzy least-squares support vector regression approach, Inform. Sci., 224, 0, 37-48, (2013)
[19] Jacobs, R.; Jordan, M.; Nowlan, S.; Hinton, G., Adaptive mixtures of local experts, Neural Comput., 3, 1, 79-87, (1991)
[20] Jayadeva; Deb, A. K.; Chandra, S., Algorithm for building a neural network for function approximation, IEE Proc.-Circ. Devices Syst., 149, 516, 301-307, (2002)
[21] Jeng, J.-T., Hybrid approach of selecting hyperparameters of support vector machine for regression, IEEE Trans. Syst. Man Cybern.-Part B: Cybern., 36, 3, 699-709, (2005)
[22] Jordan, M. I.; Jacobs, R. A., Hierarchical mixture of experts and EM algorithm, Neural Comput., 6, 2, 181-214, (1994)
[23] Lin, K.-P.; Pai, P.-F.; Lu, Y.-M.; Chang, P.-T., Revenue forecasting using a least-squares support vector regression model in a fuzzy environment, Inform. Sci., 220, 0, 196-209, (2013), online Fuzzy Machine Learning and Data Mining
[24] Musa, A. B., Comparative study on classification performance between support vector machine and logistic regression, Int. J. Mach. Learn. Cybern., 4, 13-24, (2013)
[25] Paoletti, S.; Juloski, A. L.; Ferrari-Trecate; Vidal, R., Identification of hybrid systems: a tutorial, Euro. J. Control, 13, 2/3, 242-260, (2007) · Zbl 1293.93219
[26] Pucar, P.; Sjöberg, J., On the hinge-finding algorithm for hinging hyperplanes, IEEE Trans. Inform. Theory, 44, 3, 1310-1319, (1998) · Zbl 1105.65308
[27] A.J. Smola, B. Schölkopf, A Tutorial on Support Vector Regression, NeuroCOLT2 Technical Report Series NC2-TR-1998-030, GMD, October 1998.
[28] Spath, H., A fast algorithm for clusterwise linear regression, Computing, 29, 2, 175-181, (1982) · Zbl 0485.65030
[29] S.R. Waterhouse, Classification and Regression using Mixtures of Experts, Ph.D. thesis, Department of Engineering, University of Cambridge, 1997.
[30] Wu, Q.; Law, R.; Wu, E.; Lin, J., A hybrid-forecasting model reducing gaussian noise based on the Gaussian support vector regression machine and chaotic particle swarm optimization, Inform. Sci., 238, 0, 96-110, (2013) · Zbl 1320.68151
[31] Zhang, Y.; Cao, F.; Yan, C., Learning rates of least-square regularized regression with strongly mixing observations, Int. J. Mach. Learn. Cybern., 3, 277-283, (2012)
[32] Zhao, Y.-P.; Sun, J.-G.; Du, Z.-H.; Zhang, Z.-A.; Li, Y.-B., Online independent reduced least squares support vector regression, Inform. Sci., 201, 0, 37-52, (2012) · Zbl 1248.68427
[33] Zhu, J.; Hoi, S. C.H.; Lyu, M. R.-T., Robust regularized kernel regression, IEEE Trans. Syst. Man Cybern.-Part B: Cybern., 38, 6, 1639-1644, (2008)
This reference list is based on information provided by the publisher or from digital mathematics libraries. Its items are heuristically matched to zbMATH identifiers and may contain data conversion errors. It attempts to reflect the references listed in the original paper as accurately as possible without claiming the completeness or perfect precision of the matching.