# zbMATH — the first resource for mathematics

Conditional density estimation in a regression setting. (English) Zbl 1129.62025
Summary: Regression problems are traditionally analyzed via univariate characteristics like the regression functions, scale functions and marginal density of regression errors. These characteristics are useful and informative whenever the association between the predictor and the response is relatively simple. More detailed information about the association can be provided by the conditional density of the response given the predictor. For the first time in the literature, this article develops a theory of minimax estimation of the conditional density for regression settings with fixed and random designs of predictors, bounded and unbounded responses and a vast set of anisotropic classes of conditional densities.
The study of fixed design regression is of special interest and novelty because the known literature is devoted to the case of random predictors. For the aforementioned models, the paper suggests a universal adaptive estimator which (i) matches performance of an oracle that knows both an underlying model and an estimated conditional density; (ii) is sharp minimax over a vast class of anisotropic conditional densities; (iii) is at least rate minimax when the response is independent of the predictor and thus a bivariate conditional density becomes a univariate density; (iv) is adaptive to an underlying design (fixed or random) of predictors.

##### MSC:
 62G07 Density estimation 62C20 Minimax procedures in statistical decision theory 62E20 Asymptotic distribution theory in statistics
Full Text:
##### References:
  Arnold, B. C., Castillo, E. and Sarabia, J. M. (1999). Conditional Specification of Statistical Models. Springer, New York. · Zbl 0932.62001  Bashtannyk, D. M. and Hyndman, R. J. (2001). Bandwidth selection for kernel conditional density estimation. Comput. Statist. Data Anal. 36 279-298. · Zbl 1038.62034  Efromovich, S. (1985). Nonparametric estimation of a density with unknown smoothness. Theory Probab. Appl. 30 557-568. · Zbl 0593.62034  Efromovich, S. (1989). On sequential nonparametric estimation of a density. Theory Probab. Appl. 34 228-239. · Zbl 0716.62077  Efromovich, S. (1999). Nonparametric Curve Estimation : Methods , Theory and Applications. Springer, New York. · Zbl 0935.62039  Efromovich, S. (2000). On sharp adaptive estimation of multivariate curves. Math. Methods Statist. 9 117-139. · Zbl 1006.62033  Efromovich, S. (2001). Density estimation under random censorship and order restrictions: From asymptotic to small samples. J. Amer. Statist. Assoc. 96 667-684. · Zbl 1017.62029  Efromovich, S. (2005). Estimation of the density of regression errors. Ann. Statist. 33 2194-2227. · Zbl 1086.62053  Efromovich, S. (2005). Conditional density estimation in a regression setting: Small sample sizes and proofs. Technical report, Univ. New Mexico.  Efromovich, S. (2006). Dimension reduction, optimality and oracle approach in conditional density estimation. Technical report, Univ. Texas at Dallas.  Efromovich, S. (2007). Sequential design and estimation in heteroscedastic nonparametric regression; with discussion. Sequential Anal. 26 3-25. · Zbl 1112.62080  Efromovich, S. and Pinsker, M. S. (1982). Estimation of a square integrable probability density of a random variable. Problems Inform. Transmission 18 175-189. · Zbl 0533.62038  Efromovich, S. and Pinsker M. (1996). Sharp-optimal and adaptive estimation for heteroscedastic nonparametric regression. Statist. Sinica 6 925-942. · Zbl 0857.62037  Fan, J. (1992). Design-adaptive nonparametric regression. J. Amer. Statist. Assoc. 87 998-1004. · Zbl 0850.62354  Fan, J. and Yao, Q. (2003). Nonlinear Time Series : Nonparametric and Parametric Methods . Springer, New York. · Zbl 1014.62103  Fan, J., Yao, Q. and Tong, H. (1996). Estimation of conditional densities and sensitivity measures in nonlinear dynamical systems. Biometrika 83 189-206. · Zbl 0865.62026  Fan, J. and Yim, T. H. (2004). A cross-validation method for estimating conditional densities. Biometrika 91 819-834. · Zbl 1078.62032  Golubev, G. K. (1991). Local asymptotic normality in problems of nonparametric estimation of functions, and lower bounds for quadratic risks. Theory Probab. Appl. 36 152-157. · Zbl 0738.62043  Golubev, G. K. (1992). Nonparametric estimation of smooth densities of a distribution in $$L_2$$. Problems Inform. Transmission 28 44-54. · Zbl 0785.62039  Golubev, G. K. and Levit, B. Y. (1996). Asymptotically efficient estimation for analytic distributions. Math. Methods Statist. 5 357-368. · Zbl 0872.62042  Hall, P., Racine, J. and Li, Q. (2004). Cross-validation and the estimation of conditional probability densities. J. Amer. Statist. Assoc. 99 1015-1026. · Zbl 1055.62035  Hall, P., Wolff, R. C. L. and Yao, Q. (1999). Methods for estimating a conditional distribution function. J. Amer. Statist. Assoc. 94 154-163. · Zbl 1072.62558  Hall, P. and Yao, Q. (2005). Approximating conditional distribution functions using dimension reduction. Ann. Statist. 33 1404-1421. · Zbl 1072.62008  Hasminskii, R. and Ibragimov, I. (1990). On density estimation in the view of Kolmogorov’s ideas in approximation theory. Ann. Statist. 18 999-1010. · Zbl 0705.62039  Hoffmann, M. and Lepski, O. (2002). Random rates in anisotropic regression (with discussion). Ann. Statist. 30 325-396. · Zbl 1012.62042  Hyndman, R. J., Bashtannyk, D. M. and Grunwald, G. K. (1996). Estimating and visualizing conditional densities. J. Comput. Graph. Statist. 5 315-336.  Hyndman, R. J. and Yao, Q. (2002). Nonparametric estimation and symmetry tests for conditional density functions. J. Nonparametr. Statist. 14 259-278. · Zbl 1013.62040  Ibragimov, I. A. and Hasminskii, R. Z. (1983). Estimation of distribution density belonging to a class of entire functions. Theory Probab. Appl. 27 551-562. · Zbl 0516.62043  Kahane, J.-P. (1985). Some Random Series of Functions , 2nd ed. Cambridge Univ. Press. · Zbl 0571.60002  Kawata, T. (1972). Fourier Analysis in Probability Theory. Academic Press, New York. · Zbl 0271.60022  Neter, J., Kutner, M., Nachtsheim, C. and Wasserman, W. (1996). Applied Linear Statistical Models , 4th ed. McGraw-Hill, Boston. · Zbl 0347.62043  Nikolskii, S. M. (1975). Approximation of Functions of Several Variables and Imbedding Theorems . Springer, New York.  Pinsker, M. S. (1980). Optimal filtering of square integrable signals in Gaussian white noise. Problems Inform. Transmission 16 52-68. · Zbl 0452.94003  Prakasa Rao, B. L. S. (1983). Nonparametric Functional Estimation . Academic Press, New York. · Zbl 0542.62025  Rosenblatt, M. (1969). Conditional probability density and regression estimators. In Multivariate Analysis II (P. R. Krishnaiah, ed.) 25-31. Academic Press, New York.  Schipper, M. (1996). Optimal rates and constants in $$L_2$$-minimax estimation of probability density functions. Math. Methods Statist. 5 253-274. · Zbl 0872.62043
This reference list is based on information provided by the publisher or from digital mathematics libraries. Its items are heuristically matched to zbMATH identifiers and may contain data conversion errors. It attempts to reflect the references listed in the original paper as accurately as possible without claiming the completeness or perfect precision of the matching.