×

Bayesian nonparametric modelling of the link function in the single-index model using a Bernstein-Dirichlet process prior. (English) Zbl 07193897

Summary: This paper proposes the use of the Bernstein-Dirichlet process prior for a new nonparametric approach to estimating the link function in the single-index model (SIM). The Bernstein-Dirichlet process prior has so far mainly been used for nonparametric density estimation. Here we modify this approach to allow for an approximation of the unknown link function. Instead of the usual Gaussian distribution, the error term is assumed to be asymmetric Laplace distributed which increases the flexibility and robustness of the SIM. To automatically identify truly active predictors, spike-and-slab priors are used for Bayesian variable selection. Posterior computations are performed via a Metropolis-Hastings-within-Gibbs sampler using a truncation-based algorithm for stick-breaking priors. We compare the efficiency of the proposed approach with well-established techniques in an extensive simulation study and illustrate its practical performance by an application to nonparametric modelling of the power consumption in a sewage treatment plant.

MSC:

62-XX Statistics
PDFBibTeX XMLCite
Full Text: DOI

References:

[1] Hardle W, Stoker T.Investigating Smooth Multiple Regression by the Method of Average Derivatives. J Am Stat Assoc. 1989;84(408):986-995. [Taylor & Francis Online], [Web of Science ®], [Google Scholar] · Zbl 0703.62052
[2] Hardle W, Hall P, Ichimura H.Optimal smoothing in single-index models. Ann Stat. 1993;21(1):157-178. doi: 10.1214/aos/1176349020[Crossref], [Web of Science ®], [Google Scholar] · Zbl 0770.62049
[3] Ichimura H.Semiparametric least squares (SLS) and weighted SLS estimation of single-index models. J Econom. 1991;58:71-120. doi: 10.1016/0304-4076(93)90114-K[Crossref], [Web of Science ®], [Google Scholar] · Zbl 0816.62079
[4] Carroll RJ, Fan J, Gijbels I, et al. Generalized partially linear single-index models. J Am Stat Assoc. 1997;92(438):477-489. doi: 10.1080/01621459.1997.10474001[Taylor & Francis Online], [Web of Science ®], [Google Scholar] · Zbl 0890.62053
[5] Huh J, Park BU.Likelihood-based local polynomial fitting for single-index models. J Multivariate Anal. 2002;80:302-321. doi: 10.1006/jmva.2000.1984[Crossref], [Web of Science ®], [Google Scholar] · Zbl 0992.62042
[6] Kong E, Xia Y.Quantile estimation of a general single-index model. Statistics. 2008;16(1):33-39. [Google Scholar]
[7] Kuchibhotla AK, Patra RK. Efficient estimation in single index models through smoothing splines; 2017. arXiv preprint arXiv:1612.00068v2. [Google Scholar] · Zbl 1466.62295
[8] Yu Y, Ruppert D.Penalized spline estimation for partially linear single-index models. J Am Stat Assoc. 2002;97(460):1042-1054. doi: 10.1198/016214502388618861[Taylor & Francis Online], [Web of Science ®], [Google Scholar] · Zbl 1045.62035
[9] Wang X, Roy V, Zhu Z.A new algorithm to estimate monotone nonparametric link functions and a comparison with parametric approach. Stat Comput. 2018;28:1083-1094. doi: 10.1007/s11222-017-9781-3[Crossref], [Web of Science ®], [Google Scholar] · Zbl 1406.62082
[10] Antoniadis A, Gregoire G, Mckeague IW.Bayesian estimation in single-index models. Stat Sin. 2004;14(4):1147-1164. [Web of Science ®], [Google Scholar] · Zbl 1060.62031
[11] Wang HB.Bayesian estimation and variable selection for single index models. Comput Stat Data Anal. 2009;53(7):2617-2627. doi: 10.1016/j.csda.2008.12.010[Crossref], [Web of Science ®], [Google Scholar] · Zbl 1453.62237
[12] Choi T, Shi JQ, Wang B.A Gaussian process regression approach to a single-index model. J Nonparametr Stat. 2011;23(1):21-36. doi: 10.1080/10485251003768019[Taylor & Francis Online], [Web of Science ®], [Google Scholar] · Zbl 1359.62121
[13] Gramacy RB, Lian H.Gaussian process single-index models as emulators for computer experiments. Technometrics. 2012;54(1):30-41. doi: 10.1080/00401706.2012.650527[Taylor & Francis Online], [Web of Science ®], [Google Scholar]
[14] Hu Y, Gramacy RB, Lian H.Bayesian quantile regression for single-index models. Stat Comput. 2013;23(4):437-454. doi: 10.1007/s11222-012-9321-0[Crossref], [Web of Science ®], [Google Scholar] · Zbl 1325.62089
[15] Petrone S.Random Bernstein polynomials. Scand J Stat. 1999;26(3):373-393. Available from: https://onlinelibrary.wiley.com/doi/abs/10.1111/1467-9469.00155. [Crossref], [Web of Science ®], [Google Scholar] · Zbl 0939.62046
[16] Petrone S.Bayesian density estimation using Bernstein polynomials. Can J Stat. 1999;27(1):105-126. Available from: https://onlinelibrary.wiley.com/doi/abs/10.2307/3315494. [Crossref], [Web of Science ®], [Google Scholar] · Zbl 0929.62044
[17] Petrone S, Wasserman L.Consistency of Bernstein polynomial posteriors. J R Stat Soc. 2002;64(1):79-100. doi: 10.1111/1467-9868.00326[Crossref], [Google Scholar] · Zbl 1015.62033
[18] Choudhuri N, Ghosal S, Roy A.Bayesian estimation of the spectral density of a time series. J Am Stat Assoc. 2004;99(468):1050-1059. doi: 10.1198/016214504000000557[Taylor & Francis Online], [Web of Science ®], [Google Scholar] · Zbl 1055.62100
[19] Kirch C, Edwards MC, Meier A, et al. Beyond whittle: nonparametric correction of a parametric likelihood with a focus on Bayesian time series analysis. Bayesian Anal. 2018. Available from: https://projecteuclid.org/euclid.ba/1540865702. [Crossref], [Google Scholar] · Zbl 1435.62332
[20] Sethuraman J.A constructive definition of Dirichlet priors. Stat Sin. 1994;4(2):639-650. [Web of Science ®], [Google Scholar] · Zbl 0823.62007
[21] Edwards M, Meyer R, Christensen N.Bayesian nonparametric spectral density estimation using b-spline priors. Stat Comput. 2019;29(1):67-78. doi: 10.1007/s11222-017-9796-9[Crossref], [Web of Science ®], [Google Scholar] · Zbl 1430.62072
[22] Powell JL, Stock JH, Stoker TM.Semiparametric estimation of index coefficients. Econometrica. 1989;57(6):1403-1430. doi: 10.2307/1913713[Crossref], [Web of Science ®], [Google Scholar] · Zbl 0683.62070
[23] Hristache M, Juditsky A, Spokoiny V.Direct estimation of the index coefficient in a single-index model. Ann Stat. 2001;29(3):595-623. doi: 10.1214/aos/1009210681[Crossref], [Web of Science ®], [Google Scholar] · Zbl 1012.62043
[24] Naik PA, Tsai CL.Single-index model selections. Biometrika. 2001;88(3):821-832. doi: 10.1093/biomet/88.3.821[Crossref], [Web of Science ®], [Google Scholar] · Zbl 0988.62042
[25] Peng H, Huang T.Penalized least squares for single index models. J Stat Plan Inference. 2011;141(4):1362-1379. doi: 10.1016/j.jspi.2010.10.003[Crossref], [Web of Science ®], [Google Scholar] · Zbl 1204.62070
[26] Lv Y, Zhang R, Zhao W, et al. Quantile regression and variable selection for the single-index model. J Appl Stat. 2014;41(7):1565-1577. doi: 10.1080/02664763.2014.881786[Taylor & Francis Online], [Web of Science ®], [Google Scholar] · Zbl 1514.62727
[27] O’Hara RB, Sillanpää MJ.A review of Bayesian variable selection methods: what, how and which. Bayesian Anal. 2009;4(1):85-117. doi: 10.1214/09-BA403[Crossref], [Web of Science ®], [Google Scholar] · Zbl 1330.62291
[28] Kuo L, Mallick B.Variable selection for regression models. Sankhya. 1998;60(1):65-81. [Google Scholar] · Zbl 0972.62016
[29] Koenker R, Machado JA.Goodness of fit and related inference processes for quantile regression. J Am Stat Assoc. 1999;94(448):1296-1310. doi: 10.1080/01621459.1999.10473882[Taylor & Francis Online], [Web of Science ®], [Google Scholar] · Zbl 0998.62041
[30] Yu K, Moyeed RA.Bayesian quantile regression. Stat Probab Lett. 2001;54(4):437-447. doi: 10.1016/S0167-7152(01)00124-9[Crossref], [Web of Science ®], [Google Scholar] · Zbl 0983.62017
[31] Kottas A, Gelfand AE.Bayesian semiparametric median regression modeling. Publ Am Stat Assoc. 2001;96(456):1458-1468. doi: 10.1198/016214501753382363[Taylor & Francis Online], [Web of Science ®], [Google Scholar] · Zbl 1051.62038
[32] Kottas A, Krnjajic M.Bayesian semiparametric modelling in quantile regression. Scand J Stat. 2009;36(2):297-319. doi: 10.1111/j.1467-9469.2008.00626.x[Crossref], [Web of Science ®], [Google Scholar] · Zbl 1190.62053
[33] Ghosal S. Fundamentals of nonparametric bayesian inference; 2016. Cambridge Series on Statistical and Probabilistic Mathematics; 44. [Google Scholar] · Zbl 1376.62004
[34] Edwards MC, Meyer R, Christensen N.Bayesian semiparametric power spectral density estimation with applications in gravitational wave data analysis. Phys Rev D. 2015 Sep;92:064011. Available from: https://link.aps.org/doi/10.1103/PhysRevD.92.064011. [Crossref], [Web of Science ®], [Google Scholar]
[35] Kozubowski TJ, Podgorski K.A multivariate and asymmetric generalization of Laplace distribution. Comput Stat. 2000;15(4):531-540. [Web of Science ®], [Google Scholar] · Zbl 1035.60010
[36] Lum K, Gelfand AE.Spatial quantile multiple regression using the asymmetric Laplace process. Bayesian Anal. 2012;7(2):235-258. doi: 10.1214/12-BA708[Crossref], [Web of Science ®], [Google Scholar] · Zbl 1330.62197
[37] Kuchibhotla AK, Patra RK, Sen B. Efficient estimation in convex single index models; 2016. R package version 06. [Google Scholar]
[38] Gramacy RB.tgp: an r package for Bayesian nonstationary, semiparametric nonlinear regression and design by treed Gaussian process models. J Stat Softw. 2007;19(i09). [Google Scholar]
[39] Gramacy RB, Taddy MA, Leeuw JD.Categorical inputs, sensitivity analysis, optimization and importance tempering with tgp version 2, an r package for treed Gaussian process models. J Stat Softw. 2010;33(i06):1-48. [PubMed], [Google Scholar]
[40] Wu TZ, Yu K, Yu Y.Single-index quantile regression. J Multivar Anal. 2010;101(7):1607-1621. doi: 10.1016/j.jmva.2010.02.003[Crossref], [Web of Science ®], [Google Scholar] · Zbl 1189.62075
[41] Kong E, Xia Y.A single-index quantile regression model and its estimation. Econ Theory. 2012;28(4):730-768. doi: 10.1017/S0266466611000788[Crossref], [Google Scholar] · Zbl 1419.62090
[42] Feng Z, Zhu L.An alternating determination-optimization approach for an additive multi-index model. Comput Stat Data Anal. 2012;56(6):1981-1993. doi: 10.1016/j.csda.2011.12.004[Crossref], [Web of Science ®], [Google Scholar] · Zbl 1243.62041
[43] Liang R. Urban sewage treatment plant energy consumption influence factors research [Master’s thesis]. Beijing Jiaotong University; 2014. [Google Scholar]
[44] Ren F, Mao L, Fu W, et al. Study of influent factors on energy consumption of municipal wastewater treatment plant operation in China. Water Wastewater Eng. 2015;41(1):42-47. [Google Scholar]
[45] Yu Y, Zou Z, Wang S.Statistical regression modeling for energy consumption in wastewater treatment. J Environ Sci. 2018;75:201-208. doi: 10.1016/j.jes.2018.03.023[Crossref], [Google Scholar]
This reference list is based on information provided by the publisher or from digital mathematics libraries. Its items are heuristically matched to zbMATH identifiers and may contain data conversion errors. In some cases that data have been complemented/enhanced by data from zbMATH Open. This attempts to reflect the references listed in the original paper as accurately as possible without claiming completeness or a perfect matching.