Belitser, Eduard; Ghosal, Subhashis; van Zanten, Harry Optimal two-stage procedures for estimating location and size of the maximum of a multivariate regression function. (English) Zbl 1296.62158 Ann. Stat. 40, No. 6, 2850-2876 (2012); correction ibid. 49, No. 1, 612-613 (2021). Summary: We propose a two-stage procedure for estimating the location \(\mathbf{\mu}\) and size \(M\) of the maximum of a smooth \(d\)-variate regression function \(f\). In the first stage, a preliminary estimator of \(\mathbf{\mu}\) obtained from a standard nonparametric smoothing method is used. At the second stage, we “zoom-in” near the vicinity of the preliminary estimator and make further observations at some design points in that vicinity. We fit an appropriate polynomial regression model to estimate the location and size of the maximum. We establish that, under suitable smoothness conditions and appropriate choice of the zooming, the second stage estimators have better convergence rates than the corresponding first stage estimators of \(\mathbf{\mu}\) and \(M\). More specifically, for \(\alpha\)-smooth regression functions, the optimal nonparametric rates \(n^{-(\alpha-1)/(2\alpha+d)}\) and \(n^{-\alpha/(2\alpha+d)}\) at the first stage can be improved to \(n^{-(\alpha-1)/(2\alpha)}\) and \(n^{-1/2}\), respectively, for \(\alpha>1+\sqrt{1+d/2}\). These rates are optimal in the class of all possible sequential estimators. Interestingly, the two-stage procedure resolves “the curse of the dimensionality” problem to some extent, as the dimension \(d\) does not control the second stage convergence rates, provided that the function class is sufficiently smooth. We consider a multi-stage generalization of our procedure that attains the optimal rate for any smoothness level \(\alpha>2\) starting with a preliminary estimator with any power-law rate at the first stage. Cited in 1 ReviewCited in 4 Documents MSC: 62L12 Sequential estimation 62G05 Nonparametric estimation 62H12 Estimation in multivariate analysis 62L05 Sequential statistical design Keywords:two-stage procedure; optimal rate; sequential design; multi-stage procedure; adaptive estimation × Cite Format Result Cite Review PDF Full Text: DOI arXiv Euclid References: [1] Blum, J. R. (1954). Multidimensional stochastic approximation methods. Ann. Math. Statistics 25 737-744. · Zbl 0056.38305 · doi:10.1214/aoms/1177728659 [2] Chen, H. (1988). Lower rate of convergence for locating a maximum of a function. Ann. Statist. 16 1330-1334. · Zbl 0651.62034 · doi:10.1214/aos/1176350965 [3] Dippon, J. (2003). Accelerated randomized stochastic optimization. Ann. Statist. 31 1260-1281. · Zbl 1105.62370 · doi:10.1214/aos/1059655913 [4] Facer, M. R. and Müller, H.-G. (2003). Nonparametric estimation of the location of a maximum in a response surface. J. Multivariate Anal. 87 191-217. · Zbl 1039.62033 · doi:10.1016/S0047-259X(03)00030-7 [5] Fan, J. and Gijbels, I. (1996). Local Polynomial Modelling and Its Applications. Monographs on Statistics and Applied Probability 66 . Chapman & Hall, London. · Zbl 0873.62037 [6] Hasminskiĭ, R. Z. (1979). Lower bound for the risks of nonparametric estimates of the mode. In Contributions to Statistics (J. Jureckova, ed.) 91-97. Reidel, Dordrecht. · Zbl 0419.62037 · doi:10.1007/978-94-009-9362-4_11 [7] Kiefer, J. and Wolfowitz, J. (1952). Stochastic estimation of the maximum of a regression function. Ann. Math. Statistics 23 462-466. · Zbl 0049.36601 · doi:10.1214/aoms/1177729392 [8] Klemelä, J. (2005). Adaptive estimation of the mode of a multivariate density. J. Nonparametr. Stat. 17 83-105. · Zbl 1055.62036 · doi:10.1080/10485250410001723151 [9] Lan, Y., Banerjee, M. and Michailidis, G. (2009). Change-point estimation under adaptive sampling. Ann. Statist. 37 1752-1791. · Zbl 1168.62018 · doi:10.1214/08-AOS602 [10] Mokkadem, A. and Pelletier, M. (2007). A companion for the Kiefer-Wolfowitz-Blum stochastic approximation algorithm. Ann. Statist. 35 1749-1772. · Zbl 1209.62191 · doi:10.1214/009053606000001451 [11] Müller, H.-G. (1985). Kernel estimators of zeros and of location and size of extrema of regression functions. Scand. J. Stat. 12 221-232. · Zbl 0571.62032 [12] Müller, H.-G. (1989). Adaptive nonparametric peak estimation. Ann. Statist. 17 1053-1069. · Zbl 0683.62019 · doi:10.1214/aos/1176347255 [13] Polyak, B. T. and Tsybakov, A. B. (1990). Optimal orders of accuracy for search algorithms of stochastic optimization. Problems Inform. Transmission 26 126-133. · Zbl 0712.90073 [14] Shoung, J.-M. and Zhang, C.-H. (2001). Least squares estimators of the mode of a unimodal regression function. Ann. Statist. 29 648-665. · Zbl 1012.62044 · doi:10.1214/aos/1009210684 [15] Tang, R., Banerjee, M. and Michailidis, G. (2011). A two-stage hybrid procedure for estimating an inverse regression function. Ann. Statist. 39 956-989. · Zbl 1215.62044 · doi:10.1214/10-AOS820 This reference list is based on information provided by the publisher or from digital mathematics libraries. Its items are heuristically matched to zbMATH identifiers and may contain data conversion errors. In some cases that data have been complemented/enhanced by data from zbMATH Open. This attempts to reflect the references listed in the original paper as accurately as possible without claiming completeness or a perfect matching.