×

A general projection framework for constrained smoothing. (English) Zbl 1059.62535

Summary: There is a wide array of smoothing methods available for finding structure in data. A general framework is developed which shows that many of these can be viewed as a projection of the data, with respect to appropriate norms. The underlying vector space is an unusually large product space, which allows inclusion of a wide range of smoothers in our setup (including many methods not typically considered to be projections). We give several applications of this simple geometric interpretation of smoothing. A major payoff is the natural and computationally frugal incorporation of constraints. Our point of view also motivates new estimates and helps understand the finite sample and asymptotic behavior of these estimates.

MSC:

62G07 Density estimation
65C60 Computational problems in statistics (MSC2010)

Software:

KernSmooth; fda (R); COBS
PDFBibTeX XMLCite
Full Text: DOI

References:

[1] Barlow, R. E. and van Zwet, W. R. (1970). Asy mptotic properties of isotonic estimators for generalized failure rate function I. Strong consistency. In Nonparametric Techniques in Statistical Inference (M. L. Puri, ed.) 159-173. Cambridge Univ. Press.
[2] Barlow, R. E., Bartholomew, D. J., Bremner, J. M. and Brunk, H. D. (1972). Statistical Inference under Order Restrictions. Wiley, New York Bowman, A. W. and Azzalini, A. (1997). Applied Smoothing Techniques for Data Analy sis: The Kernel Approach with S-Plus Illustrations. Oxford Univ. Press. · Zbl 0246.62038
[3] Cheng, K. F. and Lin, P. E. (1981). Nonparametric estimation of a regression function. Z. Wahrsch. Verw. Gebiete 57 223-233. · Zbl 0518.62017 · doi:10.1111/j.1467-842X.1981.tb00776.x
[4] Chu, C. K. and Marron, J. S. (1991). Choosing a kernel regression estimator (with discussion). Statist. Sci. 6 404-436. · Zbl 0955.62561 · doi:10.1214/ss/1177011586
[5] Cox, D. D. (1988). Approximation of method of regularization estimators. Ann. Statist. 16 694-712. · Zbl 0671.62044 · doi:10.1214/aos/1176350829
[6] Dechevsky, L. and MacGibbon (1999). Asy mptotically minimax nonparametric function estimation with positivity constraints i. Unpublished manuscript.
[7] Dechevsky, L., MacGibbon, B. and Penev, S. (2001). Numerical methods for asy mptotically minimax nonparametric function estimation with positivity constraints i. Sankhy\?a. · Zbl 1192.62103
[8] Delecroix, M., Simioni, M. and Thomas-Agnan, C. (1995). A shape constrained smoother: simulation study. Comput. Statist. 10 155-175. · Zbl 0936.62047
[9] Delecroix, M., Simioni, M. and Thomas-Agnan, C. (1996). Functional estimation under shape constraints. J. Nonparametr. Statist. 6 69-89. · Zbl 0862.62034 · doi:10.1080/10485259608832664
[10] Delecroix, M. and Thomas-Agnan, C. (2000). Spline and kernel smoothing under shape restrictions. In Smoothing and Regression: Approaches, Computation and Application (M. Schimek, ed.) 109-134. Wiley, New York. · Zbl 0986.62025
[11] den Hertog, D. (1994). Interior Point Approach to Linear, Quadratic and Convex Programming. Kluwer, Dordrecht. · Zbl 0808.90107
[12] Dierckx, P. (1980). An algorithm for cubic spline fitting with convexity constraints. Computing 24 349-371. · Zbl 0427.65008 · doi:10.1007/BF02237820
[13] Dole, D. (1999). C0Sm0: A constrained scatterplot smoother for estimating convex monotone transformations. J. Bus. Econom. Statist. 17 444.
[14] Dy kstra, R. L. (1983). An algorithm for restricted least squares regression. J. Amer. Statist. Assoc. 77 621-628. JSTOR: · Zbl 0535.62063 · doi:10.2307/2288193
[15] Efromovich, S. (1999). Nonparametric Curve Estimation: Methods, Theory, and Applications. Springer, New York. · Zbl 0935.62039
[16] Elfing, T. and Andersson, L. E. (1988). An algorithm for computing constrained smoothing spline functions. Numer. Math. 52 583-595. · Zbl 0622.41006 · doi:10.1007/BF01400893
[17] Eubank, R. L. (1999). Smoothing Splines and Nonparametric Regression, 2nd ed. Dekker, New York. · Zbl 0936.62044
[18] Fan, J. and Gijbels, I. (1996). Local Poly nomial Modelling and Its Application. Chapman and Hall/CRC, New York. · Zbl 0873.62037
[19] Fisher, N. I., Hall, P., Turlach, B. A. and Watson, G. S. (1997). On the estimation of a convex set from noisy data on its support function. J. Amer. Statist. Assoc. 92 84-91. JSTOR: · Zbl 0890.62028 · doi:10.2307/2291452
[20] Fletcher, R. (1987). Practical Methods of Optimization, 2nd ed. Wiley, New York. · Zbl 0905.65002
[21] Földes, A. and Révész, P. (1974). A general method for density estimation, Studia Sci. Math. Hungar. 9 81-92. · Zbl 0303.62035
[22] Friedman, J. H. and Tibshirani, R. (1984). The monotone smoothing of scatterplots. Technometrics 26 243-250.
[23] Gay lord, C. K. and Ramirez, D. E. (1991). Monotone regression splines for smoothed bootstrapping. Comput. Statist. Quarterly 6 85-97.
[24] Goldfarb, D. and Idnani, A. (1983). A numerically stable dual method for solving strictly convex quadratic programs. Math. Programming 27 1-33. · Zbl 0537.90081 · doi:10.1007/BF02591962
[25] Green, P. J. and Silverman, B. W. (1994). Nonparametric Regression and Generalized Linear Models: A Roughness Penalty Approach. Chapman and Hall/CRC, London. · Zbl 0832.62032
[26] Hall, P. and Tajvidi, N. (2000). Distribution and dependence function estimation for bivariate extreme-value distributions. Bernoulli 6 835-844. · Zbl 1067.62540 · doi:10.2307/3318758
[27] Härdle, W. (1990). Applied Non-parametric Regression. Cambridge Univ. Press.
[28] Härdle W. and Gasser, T. (1984). Robust nonparametric function fitting, J. Roy. Statist. Soc. Ser. B 46 42-51. · Zbl 0543.62034
[29] Härdle, W. and Mammen, E. (1993). Comparing nonparametric versus parametric regression fits. Ann. Statist. 21 1926- 1947. · Zbl 0795.62036 · doi:10.1214/aos/1176349403
[30] Hart, J. D. (1997). Nonparametric Smoothing and Lack-of-Fit Tests. Springer, New York. · Zbl 0886.62043
[31] Hastie, T. J. and Tibshirani, R. J. (1990). Generalized Additive Models. Chapman and Hall/CRC, London. · Zbl 0747.62061
[32] He, X. and Ng, P. (1999). COBS: qualitatively constrained smoothing via linear programming. Comput. Statist. 14 315-337. · Zbl 0941.62037 · doi:10.1007/s001800050019
[33] Irvine, L. D., Marin, S. P. and Smith, P. W. (1986). Constrained interpolation and smoothing. Constr. Approx. 2 129-151. · Zbl 0596.41012 · doi:10.1007/BF01893421
[34] Kelly, C. and Rice, J. (1990). Monotone smoothing with application to dose-response curves and the assessment of sy nergism. Biometrics 46 1071-1085.
[35] Loader, C. (1999). Local Regression and Likelihood, Statistics and Computing. Springer, New York. · Zbl 0929.62046 · doi:10.1007/b98858
[36] Mammen, E., Linton, O. and Nielsen, J. (1999). The existence and asy mptotic properties of a backfitting projection algorithm under weak conditions. Ann. Statist. 27 1443-1490. · Zbl 0986.62028
[37] Mammen, E. and Marron, J. S. (1997). Mass recentered kernel smoothers. Biometrika 84 765-778. Mammen, E., Marron, J. S., Turlach, B. A. and Wand, M. P. JSTOR: · Zbl 1090.62530 · doi:10.1093/biomet/84.4.765
[38] . Monotone local poly nomial smoothers. Unpublished manuscript.
[39] Mammen, E. and Thomas-Agnan, C. (1999). Smoothing splines and shape restrictions. Scand. J. Statist. 26 239-252. · Zbl 0932.62051 · doi:10.1111/1467-9469.00147
[40] McCormick, G. P. (1983). Nonlinear Programming: Theory, Algorithms and Applications. Wiley, New York. · Zbl 0563.90068
[41] Micchelli, C. A. and Utreras, F. I. (1988). Smoothing and interpolation in a convex subset of a Hilbert space. SIAM J. Sci. Statist. Comput. 9 728-746. · Zbl 0651.65046 · doi:10.1137/0909048
[42] Mukerjee, H. (1988). Monotone nonparametric regression. Ann. Statist. 16 741-750. · Zbl 0647.62042 · doi:10.1214/aos/1176350832
[43] M üller, H. G. (1988). Nonparametric Regression Analy sis of Longitudinal Data. Springer, New York.
[44] Nash, S. G. and Sofer, A. (1996). Linear and Nonlinear Programming. McGraw-Hill, New York.
[45] Opsomer, J. D. (2000). Ay smptotic properties of backfitting estimators. J. Multivariate Anal. 73 166-179. · Zbl 1065.62506 · doi:10.1006/jmva.1999.1868
[46] Opsomer, J. D. and Ruppert, D. (1997). Fitting a bivariate additive model by local poly nomial regression. Ann. Statist. 25 186-211. · Zbl 0869.62026 · doi:10.1214/aos/1034276626
[47] Prince, J. L. and Willsky, A. S. (1990). Reconstructing convex sets from support line measurements. IEEE Trans. Pattern Anal. Machine Intelligence 12 377-389.
[48] Ramsay, J. O. (1988). Monotone regression splines in action (with discussion). Statist. Sci. 3 425-461.
[49] Ramsay, J. O. (1998). Estimating smooth monotone functions. J. Roy. Statist. Soc. Ser. B 60 365-375. JSTOR: · Zbl 0909.62041 · doi:10.1111/1467-9868.00130
[50] Ramsay, J. O. and Silverman, B. W. (1997). Functional Data Analy sis. Springer, New York. · Zbl 0882.62002
[51] Ratkowsky, D. A. (1983). Nonlinear Regression Modeling. Dekker, New York. · Zbl 0572.62054
[52] Robertson, T., Wright, F. T. and Dy kstra, R. L. (1988). Order Restricted Statistical Inference. Wiley, New York. · Zbl 0645.62028
[53] Rudin, W. (1987). Real and Complex Analy sis. McGraw-Hill, New York. · Zbl 0925.00005
[54] Santal ó, L. A. (1976). Integral Geometry and Geometric Probability. Addison-Wesley, Reading, MA. · Zbl 0342.53049
[55] Schmidt, J. W. (1987). An unconstrained dual program for computing convex C1-spline approximants. Computing 39 133- 140. · Zbl 0633.65008 · doi:10.1007/BF02310102
[56] Schmidt, J. W. and Scholz, I. (1990). A dual algorithm for convex-concave data smoothing by cubic C2-splines. Numer. Math. 57 333-350. · Zbl 0702.65017 · doi:10.1007/BF01386414
[57] Schwetlick, H. and Kunert, V. (1993). Spline smoothing under constraints on derivatives. BIT 33 512-528. · Zbl 0796.65005 · doi:10.1007/BF01990532
[58] Silverman, B. W. (1986). Density Estimation for Statistics and Data Analy sis. Chapman and Hall/CRC, London. · Zbl 0617.62042
[59] Silverman, B. W. and Wood, J. T. (1987). The nonparametric estimation of branching curves. J. Amer. Statist. Assoc. 82 551-558. JSTOR: · Zbl 0652.62107 · doi:10.2307/2289464
[60] Simonoff, J. S. (1996). Smoothing Methods in Statistics. Springer, New York. · Zbl 0859.62035
[61] Speckman, P. L. (1988). Kernel smoothing in partial linear models. J. Roy. Statist. Soc. Ser. B 50 413-436. JSTOR: · Zbl 0671.62045
[62] Steer, B. T. and Hocking, R. A. (1985). The optimum timing of nitrogen application to irrigated sunflowers. In Proceedings of the Eleventh International Sunflower Conference 221-226. Asociación Argetina de Girasol, Buenos Aires.
[63] Stone, C. J., Hansen, M. H., Kooperberg, C. and Truong, Y. K. (1997). Poly nomial splines and their tensor products in extended linear modeling (with discussion). Ann. Statist. 25 1371-1424. · Zbl 0924.62036 · doi:10.1214/aos/1031594728
[64] Tantiy aswasdikul, C. and Woodroofe, M. B. (1994). Isotonic smoothing splines under sequential designs. J. Statist. Plann. Inference 38 75-88. · Zbl 0814.62042 · doi:10.1016/0378-3758(92)00151-S
[65] Tsy bakov, A. B. (1986). Robust reconstruction of functions by the local-approximation method. Problems Inform. Transmission 22 133-146. · Zbl 0622.62047
[66] Turlach, B. A. (1997). Constrained smoothing splines revisited. Statistics Research Report SRR 008-97, Center for Math. and Its Applications, Australian National Univ. Canberra. · Zbl 1088.62055
[67] Utreras, F. I. (1985). Smoothing noisy data under monotonicity constraints: Existence, characterization and convergence rates, Numer. Math. 47 611-625. · Zbl 0606.65006 · doi:10.1007/BF01389460
[68] van de Geer, S. (1990). Estimating a regression function. Ann. Statist. 18 907-924. · Zbl 0709.62040 · doi:10.1214/aos/1176347632
[69] Villalobos, M. and Wahba, G. (1987). Inequality-constrained multivariate smoothing splines with application to the estimation of posterior probabilities. J. Amer. Statist. Assoc. 82 239-248. JSTOR: · Zbl 0614.62047 · doi:10.2307/2289160
[70] Wahba, G. (1990). Spline Functions for Observational Data. SIAM, Philadelphia. · Zbl 0813.62001
[71] Walter, G. G. and Blum, J. (1979). Probability density estimation using delta sequences. Ann. Statist. 7 328-340. · Zbl 0403.62025 · doi:10.1214/aos/1176344617
[72] Wand, M. P. and Jones, M. C. (1995). Kernel smoothing. Chapman and Hall/CRC, London. · Zbl 0854.62043
[73] Wright, F. T. (1982). Monotone regression estimates for grouped observations. Ann. Statist. 10 278-286. · Zbl 0494.62031 · doi:10.1214/aos/1176345710
This reference list is based on information provided by the publisher or from digital mathematics libraries. Its items are heuristically matched to zbMATH identifiers and may contain data conversion errors. In some cases that data have been complemented/enhanced by data from zbMATH Open. This attempts to reflect the references listed in the original paper as accurately as possible without claiming completeness or a perfect matching.