×

zbMATH — the first resource for mathematics

Metamodel construction for sensitivity analysis. (English. French summary) Zbl 1384.62137
Summary: We propose to estimate a metamodel and the sensitivity indices of a complex model \(m\) in the Gaussian regression framework. Our approach combines methods for sensitivity analysis of complex models and statistical tools for sparse non-parametric estimation in multivariate Gaussian regression model. It rests on the construction of a metamodel for aproximating the Hoeffding-Sobol decomposition of \(m\). This metamodel belongs to a reproducing kernel Hilbert space constructed as a direct sum of Hilbert spaces leading to a functional ANOVA decomposition. The estimation of the metamodel is carried out via a penalized least-squares minimization allowing to select the subsets of variables that contribute to predict the output. It allows to estimate the sensitivity indices of \(m\). We establish an oracle-type inequality for the risk of the estimator, describe the procedure for estimating the metamodel and the sensitivity indices, and assess the performances of the procedure via a simulation study.
MSC:
62G08 Nonparametric regression and quantile regression
62H12 Estimation in multivariate analysis
62J10 Analysis of variance and covariance (ANOVA)
60E15 Inequalities; stochastic orderings
Software:
hgam; R
PDF BibTeX XML Cite
Full Text: DOI
References:
[1] Sylvain Arlot and Alain Celisse. A survey of cross-validation procedures for model selection. Statistics surveys, 4:40-79, 2010. · Zbl 1190.62080
[2] Francis Bach. High-Dimensional Non-Linear Variable Selection through Hierarchical Kernel Learning. working paper orpreprint, September 2009.
[3] Peter L. Bartlett, Olivier Bousquet, and Shahar Mendelson. Local Rademacher complexities. Ann. Statist., 33(4):1497-1537,2005. · Zbl 1083.62034
[4] A. Berlinet and C. Thomas-Agnan. Reproducing Kernel Hilbert Spaces in Probability and Statistics. Springer US, 2003. · Zbl 1145.62002
[5] G. Blatman and B. Sudret. Adaptive sparse polynomial chaos expansion based on least angle regression. Journal of computa-tional Physics, 230:2345-2367, 2011. · Zbl 1210.65019
[6] Christer Borell. The Brunn-Minkowski inequality in Gauss space. Invent. Math., 30:207-216, 1975. · Zbl 0292.60004
[7] Stéphane Boucheron, Gabor Lugosi, and Pascal Massart. A sharp concentration inequality with applications. Random Struc-tures & Algorithms, 16(3):277-292, 2000. · Zbl 0954.60008
[8] Stephen Boyd, Neal Parikh, Eric Chu, Borja Peleato, and Jonathan Eckstein. Distributed optimization and statistical learningvia the alternating direction method of multipliers. Found. Trends Mach. Learn., 3(1):1-122, 2011. · Zbl 1229.90122
[9] S. Bubeck. Convex optimization: Algorithms and complexity. volume 8 of Foundation and trends in Machine Learning, pages231-357. now, 2015.68S. HUET AND M.L. TAUPIN · Zbl 1365.90196
[10] Fan Chung and Linyuan Lu. Concentration inequalities and martingale inequalities: a survey. Internet Math., 3(1):79-127,2006. · Zbl 1111.60010
[11] B.S. Cirel’son, I.A. Ibragimov, and V.N. Sudakov. Norms of Gaussian sample functions. Proc. 3rd Japan-USSR Symp. Probab.Theory, Taschkent 1975, Lect. Notes Math. 550, 20-41 (1976)., 1976.
[12] N. Durrande, D. Ginsbourger, O. Roustant, and L. Carraro. ANOVA kernels and RKHS of zero mean functions formodel-based sensitivity analysis. Journal of Multivariate Analysis, 115(0):57 - 67, 2013. · Zbl 1259.49066
[13] Jerome H. Friedman. Multivariate adaptive regression splines. Ann. Statist., 19(1):1-141, 1991. With discussion and a rejoinderby the author. · Zbl 0765.62064
[14] Roger G. Ghanem and Pol D. Spanos. Stochastic Finite Elements: A Spectral Approach. Springer-Verlag New York, Inc., NewYork, NY, USA, 1991. · Zbl 0722.73080
[15] L. Gu and F.J. Wu. A unified framework for uncertainty and sensitivity analysis of computational models with many input.In SIMUL 2014 : The Sixth International Conference on Advances in System Simulation. IARIA XPS Press, 2014.
[16] SR Gunn and JS Kandola. Structural modelling with sparse kernels. MACHINE LEARNING, 48(1-3):137-163, 2002. · Zbl 0998.68119
[17] Bertrand Iooss and Paul Lemaître. A review on global sensitivity analysis methods. In C. Meloni and G. Dellino, editors,Uncertainty management in Simulation-Optimization of Complex Systems: Algorithms and Applications. Springer, 2015.
[18] Adel Javanmard and Andrea Montanari. Model selection for high-dimensional regression under the generalized irrepresentabilitycondition. In C. J. C. Burges, L. Bottou, M. Welling, Z. Ghahramani, and K. Q. Weinberger, editors, Advances in NeuralInformation Processing Systems 26, pages 3012-3020. Curran Associates, Inc., 2013. · Zbl 1335.94015
[19] K. Kandasamy and Y. Yu. Additive approximations in high dimensional nonparametric regression via the salsa. In Proceedingsof the 33rd International Conference on Machine Learning, New-York, volume 48. JMLR: W & CP, 2016.
[20] George S. Kimeldorf and Grace Wahba. A correspondence between bayesian estimation on stochastic processes and smoothingby splines. Ann. Math. Statist., 41(2):495-502, 04 1970. · Zbl 0193.45201
[21] V. Koltchinskii and M. Yuan. Sparse recovery in large ensembles of kernel machines. In Proceeding of the 21st Annual Con-ference on Learning Theory, Helsinki, pages 229-238. 2008.
[22] Vladimir Koltchinskii and Ming Yuan. Sparsity in multiple kernel learning. The Annals of Statistics, 38(6):3660-3695, 12 2010. · Zbl 1204.62086
[23] M. Ledoux and M. Talagrand. Probability in Banach Spaces: Isoperimetry and Processes. Ergebnisse der Mathematik undihrer Grenzgebiete. U.S. Government Printing Office, 1991. · Zbl 0748.60004
[24] Chenlei Leng, Yi Lin, and Grace Wahba. A note on the Lasso and related procedures in model selection. STATISTICASINICA, 16(4):1273-1284, OCT 2006. · Zbl 1109.62056
[25] Yi Lin and Hao Helen Zhang. Component selection and smoothing in multivariate nonparametric regression. Ann. Statist.,34(5):2272-2297, 2006. · Zbl 1106.62041
[26] P. Massart. Concentration inequalities and model selection, volume 1896 of Lecture Notes in Mathematics. Springer, Berlin,2007. Lectures from the 33rd Summer School on Probability Theory held in Saint-Flour, July 6-23, 2003, With a foreword byJean Picard.
[27] Lukas Meier, Sara van de Geer, and Peter Bühlmann. High-dimensional additive modeling. The Annals of Statistics,37(6B):3779-3821, 12 2009. · Zbl 1360.62186
[28] Shahar Mendelson. Geometric parameters of kernel machines. In Computational learning theory (Sydney, 2002), volume 2375of Lecture Notes in Comput. Sci., pages 29-43. Springer, Berlin, 2002. · Zbl 1050.68070
[29] Gilles Pisier. The volume of convex bodies and Banach space geometry, volume 94 of Cambridge Tracts in Mathematics.Cambridge University Press, Cambridge, 1989. · Zbl 0698.46008
[30] R Development Core Team. R: A Language and Environment for Statistical Computing. R Foundation for Statistical Com-puting, Vienna, Austria, 2008. ISBN 3-900051-07-0.
[31] Garvesh Raskutti, Martin J. Wainwright, and Bin Yu. Minimax-optimal rates for sparse additive models over kernel classesvia convex programming. J. Mach. Learn. Res., 13:389-427, 2012. · Zbl 1283.62071
[32] Pradeep Ravikumar, John Lafferty, Han Liu, and Larry Wasserman. Sparse additive models. J. R. Stat. Soc. Ser. B Stat.Methodol., 71(5):1009-1030, 2009. · Zbl 1411.62107
[33] Philippe Rigollet and Alexandre B. Tsybakov. Sparse Estimation by Exponential Weighting. STATISTICAL SCIENCE,27(4):558-575, NOV 2012. · Zbl 1331.62351
[34] A. Saltelli, K. Chan, and E.M. (eds) Scott. Sensitivity analysis. J. Wiley & Sons, 2000.
[35] Andrea Saltelli, Marco Ratto, Terry Andres, Francesca Campolongo, Jessica Cariboni, Debora Gatelli, Michaela Saisana, andStefano Tarantola. Global sensitivity analysis: The primer. Chichester: John Wiley & Sons. , 2008. · Zbl 1161.00304
[36] I. M. Sobol. Sensitivity estimates for nonlinear mathematical models. Math. Model. Comput. Exp., 1(4):407-414, 1993. · Zbl 1039.65505
[37] I.M Sobol. Global sensitivity indices for nonlinear mathematical models and their monte carlo estimates. Mathematics andComputers in Simulation, 55(1âĂŞ3):271 - 280, 2001. The Second IMACS Seminar on Monte Carlo Methods. · Zbl 1005.65004
[38] Christian Soize and Roger Ghanem. Physical systems with random uncertainties: chaos representations with arbitrary proba-bility measure. SIAM Journal on Scientific Computing, 26(2):395-410, 2004. · Zbl 1075.60084
[39] Samir Touzani. Response surface methods based on analysis of variance expansion for sensitivity analysis. PhD thesis, Uni-versité Grenoble, 2011.METAMODEL CONSTRUCTION FOR SENSITIVITY ANALYSIS69
[40] Samir Touzani and Daniel Busby. Smoothing spline analysis of variance approach for global sensitivity analysis of computercodes. RELIABILITY ENGINEERING & SYSTEM SAFETY, 112:67-81, APR 2013.
[41] A. W. van der Vaart. Asymptotic statistics. Cambridge series in statistical and probabilistic mathematics. Cambridge UniversityPress, Cambridge (UK), New York (N.Y.), 1998. Autre tirage : 2000 (Ãľdition brochÃľe), 2005, 2006, 2007.
[42] Grace Wahba. Spline models for observational data, volume 59 of CBMS-NSF Regional Conference Series in Applied Mathe-matics. Society for Industrial and Applied Mathematics (SIAM), Philadelphia, PA, 1990. · Zbl 0813.62001
[43] Grace Wahba, Yuedong Wang, Chong Gu, Ronald Klein, and Barbara Klein. Smoothing spline ANOVA for exponential families,with application to the Wisconsin Epidemiological Study of Diabetic Retinopathy. Ann. Statist., 23(6):1865-1895, 1995. · Zbl 0854.62042
This reference list is based on information provided by the publisher or from digital mathematics libraries. Its items are heuristically matched to zbMATH identifiers and may contain data conversion errors. It attempts to reflect the references listed in the original paper as accurately as possible without claiming the completeness or perfect precision of the matching.