zbMATH — the first resource for mathematics

Penalized logspline density estimation using total variation penalty. (English) Zbl 07345906
Summary: We study a penalized logspline density estimation method using a total variation penalty. The B-spline basis is adopted to approximate the logarithm of density functions. Total variation of derivatives of splines is penalized to impart a data-driven knot selection. The proposed estimator is a bona fide density function in the sense that it is positive and integrates to one. We devise an efficient coordinate descent algorithm for implementation and study its convergence property. An oracle inequality of the proposed estimator is established when the quality of fit is measured by the Kullback-Leibler divergence. Based on the oracle inequality, it is proved that the estimator achieves an optimal rate of convergence in the minimax sense. We also propose a logspline method for the bivariate case by adopting the tensor-product B-spline basis and a two-dimensional total variation type penalty. Numerical studies show that the proposed method captures local features without compromising the global smoothness.
62-XX Statistics
ash; R; MASS (R); glmnet
Full Text: DOI
[1] Barron, A. R.; Sheu, C.-H., Approximation of density functions by sequences of exponential families, Ann. Statist., 19, 3, 1347-1369 (1991) · Zbl 0739.62027
[2] Bertin, K.; Pennec, E. L.; Rivoirard, V., Adaptive dantzig density estimation, Ann. Inst. Henri Poincare B, 47, 1, 43-74 (2011) · Zbl 1207.62077
[3] Bigot, J.; Bellegem, S. V., Log-density deconvolution by wavelet thresholding, Scand. J. Stat., 36, 749-763 (2009) · Zbl 1223.62028
[4] Bunea, F.; Tsybakov, A. B.; Wegkamp, M. H., Sparsity oracle inequalities for the lasso, Electron. J. Stat., 1, 169-194 (2007) · Zbl 1146.62028
[5] Bunea, F.; Tsybakov, A. B.; Wegkamp, M. H.; Barbu, A., SPADES and mixture models, Ann. Statist., 38, 4, 2525-2558 (2010) · Zbl 1198.62025
[6] Friedman, J.; Hastie, T.; Tibshirani, R., Regularization paths for generalized linear models via coordinate descent, J. Stat. Softw., 33, 1 (2010)
[7] Golub, G. H.; Van Loan, C. F., Matrix Computations (1996), The Johns Hopkins University Press: The Johns Hopkins University Press London · Zbl 0865.65009
[8] Jhong, J.-H.; Koo, J.-Y.; Lee, S.-W., Penalized B-spline estimator for regression function using total variation penalty, J. Statist. Plann. Inference, 184, 77-93 (2017) · Zbl 1395.62080
[9] Koenker, R.; Mizera, I., Density estimation by total variation regularization, (Advances in Statistical Modeling and Inference. Essays in honor of Kjell A. Doksum. Advances in Statistical Modeling and Inference. Essays in honor of Kjell A. Doksum, Series in Biostatistics, vol. 3 (2007), World Scientific: World Scientific Singapore), 613-633
[10] Koo, J.-Y.; Chung, H.-Y., Log-density estimation in linear inverse problem, Ann. Statist., 26, 1, 335-362 (1998) · Zbl 0930.62036
[11] Koo, J.-Y.; Kim, W.-C., Wavelet density estimation by approximation of log-densities, Statist. Probab. Lett., 26, 271-278 (1996) · Zbl 0843.62040
[12] Koo, J.-Y.; Kooperberg, C., Logspline density estimation for binned data, Statist. Probab. Lett., 46, 133-147 (2000) · Zbl 0974.62033
[13] Koo, J.-Y.; Kooperberg, C.; Park, J., Logspline density estimation under censoring and truncation, Scand. J. Stat., 26, 87-105 (1999) · Zbl 0924.65151
[14] Koo, J.-Y.; Park, B. U., B-spline deconvolution based on the EM algorithm, J. Stat. Comput. Simul., 54, 4, 275-288 (1996) · Zbl 0903.62031
[15] Kooperberg, C.; Stone, C. J., A study of logspline density estimation, Comput. Statist. Data Anal., 12, 327-347 (1991) · Zbl 0825.62442
[16] Kooperberg, C.; Stone, C. J., Logspline density estimation for censored data, J. Comput. Graph. Statist., 1, 301-328 (1992)
[17] Mammen, E.; de Geer, S. V., Locally adaptive regression splines, Ann. Statist., 25, 1, 387-413 (1997) · Zbl 0871.62040
[18] Marron, J. S.; Wand, M. P., Exact mean integrated squared error, Ann. Statist., 20, 2, 712-736 (1992) · Zbl 0746.62040
[19] Mohler, G. O.; Bertozzi, A. L.; Goldstein, T. A.; Osher, S. J., Fast TV regularization for 2D maximum penalized likelihood estimation, J. Comput. Graph. Statist., 20, 2, 479-491 (2011)
[20] Postman, M.; Huchra, J. P.; Geller, M. J., Probes of large-scale structure in the corona Borealis region, Astron. J., 92, 6, 1238-1247 (1986)
[21] Qu, L.; Qian, Y.; Xie, H., Copula density estimation by total variation penalized likelihood, Comm. Statist. Simulation Comput., 38, 1891-1908 (2009) · Zbl 1182.62075
[22] Qu, L.; Yin, W., Copula density estimation by total variation penalized likelihood with linear equality constraints, Comput. Statist. Data Anal., 56, 384-398 (2012) · Zbl 1239.62038
[23] Roeder, K., Density estimation with confidence sets exemplified by superclusters and voids in the galaxies, J. Amer. Statist. Assoc., 85, 411, 617-624 (1990) · Zbl 0704.62103
[24] Sardy, S.; Tseng, P., Density estimation by total variation penalized likelihood driven by the sparsity \(l_1\) information criterion, Scand. J. Stat., 37, 321-337 (2010) · Zbl 1223.62035
[25] Schumaker, L., Spline Functions: Basic Theory (2007), Cambridge University Press · Zbl 1123.41008
[26] Scott, D. W., ash: David Scott’s ASH routines (2015), R package version 1.0-15. https://CRAN.R-project.org/package=ash
[27] Stone, C. J., Uniform error bounds involving logspline models, (Anderson, T. W.; Athreya, K. B.; Iglehart, D. L., Probability, Statistics and Mathematics: Papers in Honor of Samuel Karlin (1989), Academic, University of California at Berkeley: Academic, University of California at Berkeley Boston), 335-355 · Zbl 0729.62035
[28] Stone, C. J., Large-sample inference for log-spline models, Ann. Statist., 18, 2, 717-741 (1990) · Zbl 0712.62036
[29] Stone, C. J.; Hansen, M. H.; Kooperberg, C.; Truong, Y. K., Polynomial splines and their tensor products in extended linear modeling with discussion, Ann. Statist., 25, 4, 1371-1470 (1997) · Zbl 0924.62036
[30] Stone, C.; Koo, C.-Y., Logspline density estimation, Contemp. Math., 59, 1-15 (1986)
[31] Tibshirani, R. J.; Taylor, J., The solution path of the generalized lasso, Ann. Statist., 39, 3, 1335-1371 (2011) · Zbl 1234.62107
[32] Tseng, P., Convergence of a block coordinate descent method for nondifferentiable minimization, J. Optim. Theory Appl., 109, 3, 475-494 (2001) · Zbl 1006.65062
[33] Van de Geer, S., High-dimensional generalized linear models and the lasso, Ann. Statist., 36, 2, 614-645 (2008) · Zbl 1138.62323
[34] Venables, W. N.; Ripley, B. D., Modern Applied Statistics with S (2002), Springer: Springer New York, ISBN 0-387-95457-0, http://www.stats.ox.ac.uk/pub/MASS4 · Zbl 1006.62003
[35] Wainwright, M. J., Sharp thresholds for high-dimensional and noisy sparsity recovery using \(l_1\)-constrained quadratic programming (Lasso), IEEE Trans. Inform. Theory, 55, 5, 2183-2202 (2009) · Zbl 1367.62220
[36] Zhang, C.-H.; Huang, J., The sparsity and bias of the lasso selection in high-dimensional linear regression, Ann. Statist., 36, 4, 1567-1594 (2008) · Zbl 1142.62044
[37] Zhao, P.; Yu, B., On model selection consistency of lasso, J. Mach. Learn. Res., 7, 2541-2563 (2006) · Zbl 1222.62008
This reference list is based on information provided by the publisher or from digital mathematics libraries. Its items are heuristically matched to zbMATH identifiers and may contain data conversion errors. It attempts to reflect the references listed in the original paper as accurately as possible without claiming the completeness or perfect precision of the matching.