zbMATH — the first resource for mathematics

Robust weighted Gaussian processes. (English) Zbl 07315559
Summary: This paper presents robust weighted variants of batch and online standard Gaussian processes (GPs) to effectively reduce the negative impact of outliers in the corresponding GP models. This is done by introducing robust data weighers that rely on robust and quasi-robust weight functions that come from robust M-estimators. Our robust GPs are compared to various GP models on four datasets. It is shown that our batch and online robust weighted GPs are indeed robust to outliers, significantly outperforming the corresponding standard GPs and the recently proposed heteroscedastic GP method GPz. Our experiments also show that our methods are comparable to and sometimes better than a state-of-the-art robust GP that uses a Student-\(t\) likelihood.
65C60 Computational problems in statistics (MSC2010)
LIBRA; robustbase
Full Text: DOI
[1] Agostinelli, C.; Greco, L., A weighted strategy to handle likelihood uncertainty in Bayesian inference, Comput Stat, 28, 319-339 (2013) · Zbl 1305.65018
[2] Almosallam, IA; Jarvis, MJ; Roberts, SJ, GPz: non-stationary sparse Gaussian processes for heteroscedastic uncertainty estimation in photometric redshifts, Mon Not R Astron Soc, 462, 1, 726-739 (2016)
[3] Bentley, JL, Multidimensional divide and conquer, Commun ACM, 23, 4, 214-229 (1980) · Zbl 0434.68049
[4] Bernholt, T.; Fischer, P., The complexity of computing the MCD-estimator, Theor Comput Sci, 326, 383-398 (2004) · Zbl 1054.62028
[5] Bishop, CM, Pattern recognition and machine learning (2006), New York: Springer, New York · Zbl 1107.68072
[6] Buta, R., The structure and dynamics of ringed galaxies, III: surface photometry and kinematics of the ringed nonbarred spiral NGC7531, Astrophys J Suppl Ser, 64, 1-37 (1987)
[7] Csató L (2002) Gaussian processes—iterative sparse approximations. PhD thesis. Aston University, Birmingham, UK. http://publications.aston.ac.uk/id/eprint/1327/ · Zbl 0987.62060
[8] Csató, L.; Opper, M., Sparse on-line Gaussian processes, Neural Comput, 14, 3, 641-668 (2002) · Zbl 0987.62060
[9] de Boor, CA, Practical guide to splines. Applied mathematical sciences (2001), New York: Springer, New York · Zbl 0987.65015
[10] Dennis, JE Jr; Welsch, RE, Techniques for nonlinear least squares and robust regression, Commun Stat Simul Comput, 7, 4, 345-359 (1978) · Zbl 0395.62046
[11] Drucker H, Burges CJ, Kaufman L, Smola AJ, Vapnik V (1997) Support vector regression machines. Advances in neural information processing systems, pp 155-161
[12] Geweke, J., Bayesian treatment of the independent Student-t linear model, J Appl Econom, 8, S1, S19-S40 (1993)
[13] Girden ER (1992) ANOVA: repeated measures. Sage University Paper Series on Quantitative Applications in the Social Sciences 07-084, Sage University, Newbury Park, CA
[14] Greco, L.; Racugno, W.; Ventura, L., Robust likelihood functions in Bayesian analysis, J Stat Plan Inference, 138, 5, 1258-1270 (2008) · Zbl 1133.62017
[15] Hampel, FR; Ronchetti, EM; Rousseeuw, PJ; Stahel, WA, Robust statistics. The approach based on influence functions (1986), New York: Wiley, New York · Zbl 0593.62027
[16] Huber, PJ, Robust estimation of a location parameter, Ann Stat, 53, 1, 73-101 (1964) · Zbl 0136.39805
[17] Huber, PJ; Ronchetti, EM, Robust statistics (1981), New York: Wiley, New York · Zbl 0536.62025
[18] Jylänki, P.; Vanhatalo, J.; Vehtari, A., Robust Gaussian process regression with a student-t likelihood, J Mach Learn Res, 12, 3227-3257 (2011) · Zbl 1280.60025
[19] Kemmler M, Rodner E, Denzler J (2010) One-class classification with Gaussian processes. In: Proceedings of the Asian conference on computer vision. Lecture notes in computer science, vol 6493. Springer, pp 489-500
[20] Kuss M (2006) Gaussian process models for robust regression, classification, and reinforcement learning. Doctoral dissertation, Technische Universität Darmstadt, Germany. http://hdl.handle.net/11858/00-001M-0000-0013-D2CD-C
[21] Kuss M, Pfingsten T, CsatóL, Rasmussen CE (2005) Approximate inference for robust Gaussian process regression. Max Planck Institute for Biological Cybernetics, Tübingen, Germany, Technical Report 136. http://hdl.handle.net/11858/00-001M-0000-0013-D703-4
[22] Le QV, Smola AJ, Canu S (2005) Heteroscedastic Gaussian process regression. In: Proceedings of the 22nd international conference on machine learning. ACM, pp 489-496
[23] MacKay, DJC; Bishop, CM, Introduction to Gaussian processes, Neural networks and machine learning. NATO ASI series, 133-165 (1998), Berlin: Springer, Berlin
[24] Mahalanobis, PC, On the generalised distance in statistics, Proc Natl Inst Sci India, 2, 1, 49-55 (1936) · Zbl 0015.03302
[25] Maronna, RA; Martin, DR; Yohai, VJ, Robust statistics: theory and methods (2006), Chichester: Wiley, Chichester · Zbl 1094.62040
[26] Mattos CLC, Santos JDA, Barreto GA (2015) An empirical evaluation of robust Gaussian process models for system identification. In: International conference on intelligent data engineering and automated learning. Springer, Cham, pp 172-180
[27] Minka TP (2001) Expectation propagation for approximate Bayesian Inference. In: Proceedings of the seventeenth conference on uncertainty in artificial intelligence. Morgan Kaufmann Publishers Inc., pp 362-369
[28] Murphy L, Martin S, Corke P (2012) Creating and using probabilistic cost maps from vehicle experience. In: Proceedings of IEEE/RSJ international conference on intelligent robots and systems, intelligent robots and systems (IROS). IEEE, pp 4689-4694
[29] Neal RM (1997) Monte Carlo implementation of Gaussian process models for Bayesian regression and classification. Technical Report 9702, Department of Statistics and Department of Computer Science, University of Toronto. arXiv:physics/9701026
[30] Opper, M.; Saad, D., A Bayesian approach to on-line learning, On-line learning in neural networks (1998), Cambridge: Cambridge University Press, Cambridge · Zbl 0966.68178
[31] Ramirez-Padron R (2015) Batch and online implicit weighted Gaussian processes for robust Novelty detection. Doctoral dissertation, University of Central Florida. http://purl.fcla.edu/fcla/etd/CFE0005869
[32] Ramirez-Padron R, Mederos B, Gonzalez AJ (2013) Novelty detection using sparse online Gaussian processes for visual object recognition. In: International FLAIRS conference. St. Pete Beach, FL, USA, pp 124-129
[33] Ranjan, R.; Huang, B.; Fatehi, A., Robust Gaussian process modeling using EM algorithm, J Process Control, 42, 125-136 (2016)
[34] Rasmussen, CE; Williams, C., Gaussian processes for machine learning (2006), Cambridge: MIT Press, Cambridge · Zbl 1177.68165
[35] Rey, WJJ, Introduction to robust and quasi-robust statistical methods (1983), Berlin: Springer, Berlin
[36] Rottmann A, Burgard W (2010) Learning non-stationary system dynamics online using Gaussian processes. In: Proceedings of 32nd DAGM symposium, Darmstadt, Germany, pp 192-201
[37] Rousseeuw, PJ, Least median of squares regression, J Am Stat Assoc, 79, 388, 871-880 (1984) · Zbl 0547.62046
[38] Schmidt G, Mattern R, Schüler F (1981) Biomechanical investigation to determine physical and traumatological differentiation criteria for the maximum load capacity of head and vertebral column with and without protective helmet under effects of impact. EEC Research Program on Biomechanics of Impacts, Final Report Phase III, Project 65, Institut für Rechtsmedizin, Universität Heidelberg, Germany
[39] Seeger, M., Gaussian processes for machine learning, Int J Neural Syst, 14, 2, 69-106 (2004)
[40] Shawe-Taylor, J.; Cristianini, N., Kernel methods for pattern analysis (2004), Cambridge: Cambridge University Press, Cambridge
[41] Silverman, BW, Some aspects of the spline smoothing approach to non-parametric curve fitting, J R Stat Soc Ser B (Methodol), 47, 1, 1-52 (1985) · Zbl 0606.62038
[42] Sugiyama, M.; Krauledat, M.; Müller, KR, Covariate shift adaptation by importance weighted cross validation, J Mach Learn Res, 8, 985-1005 (2007) · Zbl 1222.68313
[43] Tipping ME (2000) The relevance vector machine. In: Advances in neural information processing systems, pp 652-658
[44] Tipping, ME; Lawrence, ND, Variational inference for Student-t models: robust Bayesian interpolation and generalised component analysis, Neurocomputing, 69, 1-3, 123-141 (2005)
[45] Verboven, S.; Hubert, M., LIBRA: a MATLAB Library for robust analysis, Chemometr Intell Lab Syst, 75, 127-136 (2005)
[46] Wald I, Havran V (2006) On building fast kd-trees for ray tracing, and on doing that in O(N log N). In: Proceedings of the 2006 IEEE symposium on interactive ray tracing, pp 61-69
[47] Wang, B.; Mao, Z., Outlier detection based on Gaussian process with application to industrial processes, Appl Soft Comput J, 76, 505-516 (2019)
[48] West, M., Outlier models and prior distributions in Bayesian linear regression, J R Stat Soc (Ser B), 46, 3, 431-439 (1984) · Zbl 0567.62022
[49] Williams, CKI; Barber, D., Bayesian classification with Gaussian processes, IEEE Trans Pattern Anal Mach Intell, 12, 20, 1342-1351 (1998)
[50] Yeh, C., Modeling of strength of high performance concrete using artificial neural networks, Cem Concr Res, 28, 12, 1797-1808 (1998)
This reference list is based on information provided by the publisher or from digital mathematics libraries. Its items are heuristically matched to zbMATH identifiers and may contain data conversion errors. It attempts to reflect the references listed in the original paper as accurately as possible without claiming the completeness or perfect precision of the matching.