×

Nonparametric modal regression. (English) Zbl 1338.62113

Summary: Modal regression estimates the local modes of the distribution of \(Y\) given \(X=x\), instead of the mean, as in the usual regression sense, and can hence reveal important structure missed by usual regression methods. We study a simple nonparametric method for modal regression, based on a kernel density estimate (KDE) of the joint distribution of \(Y\) and \(X\). We derive asymptotic error bounds for this method, and propose techniques for constructing confidence sets and prediction sets. The latter is used to select the smoothing bandwidth of the underlying KDE. The idea behind modal regression is connected to many others, such as mixture regression and density ridge estimation, and we discuss these ties as well.

MSC:

62G08 Nonparametric regression and quantile regression
62G20 Asymptotic properties of nonparametric inference
62G05 Nonparametric estimation

Software:

PRMLT

References:

[1] Arias-Castro, E., Mason, D. and Pelletier, B. (2013). On the estimation of the gradient lines of a density and the consistency of the mean-shift algorithm. Unpublished Manuscript. · Zbl 1360.62150
[2] Bishop, C. M. (2006). Pattern Recognition and Machine Learning . Springer, New York. · Zbl 1107.68072
[3] Carreira-Perpiñán, M. Á. (2007). Gaussian mean-shift is an em algorithm. IEEE Trans. Pattern Anal. Mach. Intell. 29 0767-0776.
[4] Chaganty, A. T. and Liang, P. (2013). Spectral experts for estimating mixtures of linear regressions. In Proceedings of the 30 th International Conference on Machine Learning ( ICML- 13) 1040-1048. ACM, New York.
[5] Chen, Y.-C., Genovese, C. R. and Wasserman, L. (2014a). Enhanced mode clustering. Available at . arXiv:1406.1780
[6] Chen, Y.-C., Genovese, C. R. and Wasserman, L. (2014b). Generalized mode and ridge estimation. Available at . arXiv:1406.1803
[7] Chen, Y.-C., Genovese, C. R. and Wasserman, L. (2015). Asymptotic theory for density ridges. Ann. Statist. 43 1896-1928. · Zbl 1327.62303 · doi:10.1214/15-AOS1329
[8] Chen, Y.-C., Genovese, C. R., Tibshirani, R. J. and Wasserman, L. (2015). Supplement to “Nonparametric modal regression.” .
[9] Cheng, Y. (1995). Mean shift, mode seeking, and clustering. IEEE Trans. Pattern Anal. Mach. Intell. 17 790-799.
[10] Chernozhukov, V., Chetverikov, D. and Kato, K. (2014a). Anti-concentration and honest, adaptive confidence bands. Ann. Statist. 42 1787-1818. · Zbl 1305.62161 · doi:10.1214/14-AOS1235
[11] Chernozhukov, V., Chetverikov, D. and Kato, K. (2014b). Gaussian approximation of suprema of empirical processes. Ann. Statist. 42 1564-1597. · Zbl 1317.60038 · doi:10.1214/14-AOS1230
[12] Comaniciu, D. and Meer, P. (2002). Mean shift: A robust approach toward feature space analysis. IEEE Trans. Pattern Anal. Mach. Intell. 24 603-619.
[13] Eberly, D. (1996). Ridges in Image and Data Analysis . Springer, Berlin. · Zbl 0917.68218
[14] Efron, B. (1979). Bootstrap methods: Another look at the jackknife. Ann. Statist. 7 1-26. · Zbl 0406.62024 · doi:10.1214/aos/1176344552
[15] Einbeck, J. and Tutz, G. (2006). Modelling beyond regression functions: An application of multimodal regression to speed-flow data. J. Roy. Statist. Soc. Ser. C 55 461-475. · Zbl 1490.62098 · doi:10.1111/j.1467-9876.2006.00547.x
[16] Einmahl, U. and Mason, D. M. (2005). Uniform in bandwidth consistency of kernel-type function estimators. Ann. Statist. 33 1380-1403. · Zbl 1079.62040 · doi:10.1214/009053605000000129
[17] Genovese, C. R., Perone-Pacifico, M., Verdinelli, I. and Wasserman, L. (2014). Nonparametric ridge estimation. Ann. Statist. 42 1511-1545. · Zbl 1310.62045 · doi:10.1214/14-AOS1218
[18] Giné, E. and Guillou, A. (2002). Rates of strong uniform consistency for multivariate kernel density estimators. Ann. Inst. Henri Poincaré Probab. Stat. 38 907-921. · Zbl 1011.62034 · doi:10.1016/S0246-0203(02)01128-7
[19] Huang, M., Li, R. and Wang, S. (2013). Nonparametric mixture of regression models. J. Amer. Statist. Assoc. 108 929-941. · Zbl 06224977 · doi:10.1080/01621459.2013.772897
[20] Huang, M. and Yao, W. (2012). Mixture of regression models with varying mixing proportions: A semiparametric approach. J. Amer. Statist. Assoc. 107 711-724. · Zbl 1261.62036 · doi:10.1080/01621459.2012.682541
[21] Hunter, D. R. and Young, D. S. (2012). Semiparametric mixtures of regressions. J. Nonparametr. Stat. 24 19-38. · Zbl 1241.62055 · doi:10.1080/10485252.2011.608430
[22] Hyndman, R. J., Bashtannyk, D. M. and Grunwald, G. K. (1996). Estimating and visualizing conditional densities. J. Comput. Graph. Statist. 5 315-336.
[23] Jacobs, R. A., Jordan, M. I., Nowlan, S. J. and Hinton, G. E. (1991). Adaptive mixtures of local experts. Neural Comput. 3 79-87. ISSN 0899-7667. Available at .
[24] Jiang, W. and Tanner, M. A. (1999). Hierarchical mixtures-of-experts for exponential family regression models: Approximation and maximum likelihood estimation. Ann. Statist. 27 987-1011. · Zbl 0957.62032 · doi:10.1214/aos/1018031265
[25] Khalili, A. and Chen, J. (2007). Variable selection in finite mixture of regression models. J. Amer. Statist. Assoc. 102 1025-1038. · Zbl 1469.62306 · doi:10.1198/016214507000000590
[26] Lee, M.-j. (1989). Mode regression. J. Econometrics 42 337-349. · Zbl 0692.62092 · doi:10.1016/0304-4076(89)90057-2
[27] Li, J., Ray, S. and Lindsay, B. G. (2007). A nonparametric statistical approach to clustering via mode identification. J. Mach. Learn. Res. 8 1687-1723. · Zbl 1222.62076
[28] Rojas, A. (2005). Nonparametric mixture regression. Ph.D. thesis, Carnegie Mellon Univ., Pittsburgh, PA.
[29] Romano, J. P. (1988). On weak convergence and optimality of kernel density estimates of the mode. Ann. Statist. 16 629-647. · Zbl 0658.62053 · doi:10.1214/aos/1176350824
[30] Sager, T. W. and Thisted, R. A. (1982). Maximum likelihood estimation of isotonic modal regression. Ann. Statist. 10 690-707. · Zbl 0491.62035 · doi:10.1214/aos/1176345865
[31] Scott, D. W. (1992). Multivariate Density Estimation : Theory , Practice , and Visualization . Wiley, New York. · Zbl 0850.62006
[32] Viele, K. and Tong, B. (2002). Modeling with mixtures of linear regressions. Stat. Comput. 12 315-330. · doi:10.1023/A:1020779827503
[33] Yao, W. (2013). A note on EM algorithm for mixture models. Statist. Probab. Lett. 83 519-526. · Zbl 1352.62044 · doi:10.1016/j.spl.2012.10.017
[34] Yao, W. and Li, L. (2014). A new regression model: Modal linear regression. Scand. J. Stat. 41 656-671. · Zbl 1309.62119 · doi:10.1111/sjos.12054
[35] Yao, W. and Lindsay, B. G. (2009). Bayesian mixture labeling by highest posterior density. J. Amer. Statist. Assoc. 104 758-767. · Zbl 1388.62007 · doi:10.1198/jasa.2009.0237
[36] Yao, W., Lindsay, B. G. and Li, R. (2012). Local modal regression. J. Nonparametr. Stat. 24 647-663. · Zbl 1254.62059 · doi:10.1080/10485252.2012.678848
This reference list is based on information provided by the publisher or from digital mathematics libraries. Its items are heuristically matched to zbMATH identifiers and may contain data conversion errors. In some cases that data have been complemented/enhanced by data from zbMATH Open. This attempts to reflect the references listed in the original paper as accurately as possible without claiming completeness or a perfect matching.