×
Candès, Emmanuel J.

Ridgelets: estimating with ridge functions. (English) Zbl 1046.62037

Ann. Stat. 31, No. 5, 1561-1599 (2003).
Summary: Feedforward neural networks, projection pursuit regression, and more generally, estimation via ridge functions have been proposed as an approach to bypass the curse of dimensionality and are now becoming widely applied to approximation or prediction in applied sciences. To address problems inherent to these methods – ranging from the construction of neural networks to their efficiency and capability – the author [Appl. Comput. Harmon. Anal. 6, 197–218 (1999; Zbl 0931.68104)] developed a new system that allows the representation of arbitrary functions as superpositions of specific ridge functions, the ridgelets.
In a nonparametric regression setting, this article suggests expanding noisy data into a ridgelet series and applying a scalar nonlinearity to the coefficients (damping); this is unlike existing approaches based on stepwise additions of elements. The procedure is simple, constructive, stable and spatially adaptive – and fast algorithms have been developed to implement it.
The ridgelet estimator is nearly optimal for estimating functions with certain kinds of spatial inhomogeneities. In addition, ridgelets help to identify new classes of estimands – corresponding to a new notion of smoothness – that are well suited for ridge functions estimation. While the results are stated in a decision theoretic framework, numerical experiments are also presented to illustrate the practical performance of the methodology.

Cited in 23 Documents

MSC:

62G08 Nonparametric regression and quantile regression
62G07 Density estimation
41A30 Approximation by other special function classes
62C20 Minimax procedures in statistical decision theory
42C40 Nontrigonometric harmonic analysis involving wavelets and other special systems

Keywords:

ridgelets; ridge functions; projection pursuit regression; Radon transform; spatial inhomogeneities; edges; thresholding of ridgelet coefficients

Citations:

Zbl 0931.68104
Cite Review PDF
Full Text: DOI

References:

[1] Barron, A. R. (1991). Complexity regularization with application to artificial neural networks. In Nonparametric Functional Estimation and Related Topics (G. Roussas, ed.) 561–576. Kluwer, Dordrecht. · Zbl 0739.62001
[2] Candès, E. J. (1998). Ridgelets: Theory and applications. Ph.D. dissertation, Dept. Statistics, Stanford Univ.
[3] Candès, E. J. (1999a). Harmonic analysis of neural netwoks. Appl. Comput. Harmon. Anal. 6 197–218. · Zbl 0931.68104 · doi:10.1006/acha.1998.0248
[4] Candès, E. J. (1999b). Monoscale ridgelets for the representation of images with edges. Technical report, Dept. Statistics, Stanford Univ.
[5] Candès, E. J. (2001). Ridgelets and the representation of mutilated Sobolev functions. SIAM J. Math. Anal. 33 347–368. · Zbl 0989.42019 · doi:10.1137/S003614109936364X
[6] Candès, E. J. (2002). New ties between computational harmonic analysis and approximation theory. In Approximation Theory X (C. K. Chui, L. L. Schumaker and J. Stöckler, eds.) 87–153. Vanderbilt Univ. Press, Nashville, TN. · Zbl 1026.41017
[7] Candès, E. J. and Donoho, D. L. (2000). Curvelets—A surprisingly effective nonadaptive representation of objects with edges. In Curve and Surface Fitting (A. Cohen, C. Rabut and L. L. Schumaker, eds.) 105–120. Vanderbilt Univ. Press, Nashville, TN.
[8] Cheng, B. and Titterington, D. M. (1994). Neural networks: A review from a statistical perspective (with discussion). Statist. Sci. 9 2–54. JSTOR: · Zbl 0955.62589 · doi:10.1214/ss/1177010638
[9] Conway, J. H. and Sloane, N. J. A. (1988). Sphere Packings , Lattices and Groups . Springer, New York. · Zbl 0634.52002
[10] Cybenko, G. (1989). Approximation by superpositions of a sigmoidal function. Math. Control Signals Systems 2 303–314. · Zbl 0679.94019 · doi:10.1007/BF02551274
[11] Deans, S. R. (1983). The Radon Transform and Some of Its Applications . Wiley, New York. · Zbl 0561.44001
[12] Donoho, D. L. (1993). Unconditional bases are optimal bases for data compression and for statistical estimation. Appl. Comput. Harmon. Anal. 1 100–115. · Zbl 0796.62083 · doi:10.1006/acha.1993.1008
[13] Donoho, D. L. (1998). Digital ridgelet transform via rectopolar coordinate transform. Technical report, Dept. Statistics, Stanford Univ.
[14] Donoho, D. L. and Johnstone, I. M. (1989). Projection-based approximation and a duality with kernel methods. Ann. Statist. 17 58–106. JSTOR: · Zbl 0699.62067 · doi:10.1214/aos/1176347004
[15] Donoho, D. L. and Johnstone, I. M. (1994). Ideal spatial adaptation via wavelet shrinkage. Biometrika 81 425–455. · Zbl 0815.62019 · doi:10.1093/biomet/81.3.425
[16] Donoho, D. L. and Johnstone, I. M. (1995). Empirical atomic decomposition. Unpublished manuscript.
[17] Donoho, D. L. and Johnstone, I. M. (1998). Minimax estimation via wavelet shrinkage. Ann. Statist. 26 879–921. · Zbl 0935.62041 · doi:10.1214/aos/1024691081
[18] Efroimovich, S. and Pinsker, M. (1982). Estimation of square-integrable density on the basis of a sequence of observations. Problems Inform. Transmission 17 182–196. · Zbl 0524.62095
[19] Friedman, J. H. and Stuetzle, W. (1981). Projection pursuit regression. J. Amer. Statist. Assoc. 76 817–823.
[20] Härdle, W. (1990). Applied Nonparametric Regression. Cambridge Univ. Press. · Zbl 0714.62030
[21] Härdle, W., Kerkyacharian, G., Picard, D. and Tsybakov, A. (1998). Wavelets , Approximation , and Statistical Applications. Lecture Notes in Statist. 129 . Springer, New York. · Zbl 0899.62002
[22] Ibragimov, I. A. and Hasminskii, R. Z. (1981). Statistical Estimation. Asymptotic Theory . Springer, New York. · Zbl 0467.62026
[23] Johnstone, I. M. (1999). Wavelets and the theory of nonparametric function estimation. Available at www-stat.stanford.edu/ imj.
[24] Johnstone, I. M. and Silverman, B. W. (1997). Wavelet threshold estimators for data with correlated noise. J. Roy. Statist. Soc. Ser. B 59 319–351. · Zbl 0886.62044 · doi:10.1111/1467-9868.00071
[25] Jones, L. K. (1997). The computational intractability of training sigmoidal neural networks. IEEE Trans. Inform. Theory 43 167–173. · Zbl 0874.68255 · doi:10.1109/18.567673
[26] Korostelev, A. P. and Tsybakov, A. B. (1993). Minimax Theory of Image Reconstruction. Lecture Notes in Statist . 82 . Springer, New York. · Zbl 0833.62039
[27] Meyer, Y. (1992). Wavelets and Operators . Cambridge Univ. Press. · Zbl 0776.42019
[28] Pinsker, M. (1980). Optimal filtering of square integrable signals in Gaussian white noise. Problems Inform. Transm. 16 120–133. · Zbl 0452.94003
[29] Silverman, B. (1999). Wavelets in statistics: Beyond the standard assumptions. R. Soc. Lond. Philos. Trans. Ser. A Math. Phys. Eng. Sci. 357 2459–2473. · Zbl 1054.62538 · doi:10.1098/rsta.1999.0442
[30] Starck, J., Candès, E. and Donoho, D. (2002). The curvelet transform for image denoising. IEEE Trans. Image Process. 11 670–684. · Zbl 1288.94011 · doi:10.1109/TIP.2002.1014998
[31] Stone, C. J. (1977). Consistent nonparametric regression (with discussion). Ann. Statist. 5 595–645. JSTOR: · Zbl 0366.62051 · doi:10.1214/aos/1176343886
[32] Vu, V. H. (1998). On the infeasibility of training neural networks with small mean squared error. IEEE Trans. Inform. Theory 44 2892–2900. · Zbl 0981.68138 · doi:10.1109/18.737520
This reference list is based on information provided by the publisher or from digital mathematics libraries. Its items are heuristically matched to zbMATH identifiers and may contain data conversion errors. In some cases that data have been complemented/enhanced by data from zbMATH Open. This attempts to reflect the references listed in the original paper as accurately as possible without claiming completeness or a perfect matching.