Approximation by log-concave distributions, with applications to regression. (English) Zbl 1216.62023

Summary: We study the approximation of arbitrary distributions \(P\) on \(d\)-dimensional space by distributions with log-concave density. Approximation means minimizing a Kullback-Leibler-type functional. We show that such an approximation exists if and only if \(P\) has finite first moments and is not supported by some hyperplane. Furthermore we show that this approximation depends continuously on \(P\) with respect to C. L. Mallows distance \(D_{1}(\cdot , \cdot )\) [Ann. Math. Stat. 43, 508–515 (1972; Zbl 0238.60017)]. This result implies consistency of the maximum likelihood estimator of a log-concave density under fairly general conditions. It also allows us to prove existence and consistency of estimators in regression models with a response \(Y=\mu (X)+\varepsilon \), where \(X\) and \(\varepsilon \) are independent, \(\mu (\cdot )\) belongs to a certain class of regression functions while \(\varepsilon \) is a random error with log-concave density and mean zero.


62E17 Approximations to statistical distributions (nonasymptotic)
62G07 Density estimation
62G08 Nonparametric regression and quantile regression
62H12 Estimation in multivariate analysis
62J05 Linear regression; mixed models


Zbl 0238.60017
Full Text: DOI arXiv


[1] Anderson, T. W. (1955). The integral of a symmetric unimodal function over a symmetric convex set and some probability inequalities. Proc. Amer. Math. Soc. 6 170-176. JSTOR: · Zbl 0066.37402
[2] Bagnoli, M. and Bergstrom, T. (2005). Log-concave probability and its applications. Econometric Theory 26 445-469. · Zbl 1077.60012
[3] Balabdaoui, F., Rufibach, K. and Wellner, J. A. (2009). Limit distribution theory for maximum likelihood estimation of a log-concave density. Ann. Statist. 37 1299-1331. · Zbl 1160.62008
[4] Barlow, R. E., Bartholomew, D. J., Bremner, J. M. and Brunk, H. D. (1972). Statistical Inference Under Order Restrictions. The Theory and Application of Isotonic Regression . Wiley, London. · Zbl 0246.62038
[5] Bickel, P. J. and Freedman, D. A. (1981). Some asymptotic theory for the bootstrap. Ann. Statist. 9 1196-1217. · Zbl 0449.62034
[6] Cule, M. L. and Samworth, R. J. (2010). Theoretical properties of the log-concave maximum likelihood estimator of a multidimensional density. Electronic J. Statist. 4 254-270. · Zbl 1329.62183
[7] Cule, M. L., Samworth, R. J. and Stewart, M. I. (2010). Maximum likelihood estimation of a multi-dimensional log-concave density (with discussion). J. Roy. Statist. Soc. Ser. B 72 545-607.
[8] Doksum, K., Ozeki, A., Kim, J. and Neto, E. C. (2007). Thinking outside the box: Statistical inference based on Kullback-Leibler empirical projections. Statist. Probab. Lett. 77 1201-1213. · Zbl 1116.62001
[9] Donoho, D. L. and Gasko, M. (1992). Breakdown properties of location estimates based on halfspace depth and projected outlyingness. Ann. Statist. 20 1803-1827. · Zbl 0776.62031
[10] Dümbgen, L., Hüsler, A. and Rufibach, K. (2007). Active set and EM algorithms for log-concave densities based on complete and censored data. Technical Report 61, IMSV, Univ. Bern. Available at .
[11] Dümbgen, L. and Rufibach, K. (2009). Maximum likelihood estimation of a log-concave density and its distribution function: Basic properties and uniform consistency. Bernoulli 15 40-68. · Zbl 1200.62030
[12] Dümbgen, L., Samworth, R. and Schuhmacher, D. (2010). Approximation by log-concave distributions with applications to regression. Technical Report 75, IMSV, Univ. Bern. Available at . · Zbl 1216.62023
[13] Grenander, U. (1956). On the theory of mortality measurement. II. Skand. Aktuarietidskr. 39 125-153. · Zbl 0077.33715
[14] Kantorovič, L. V. and Rubinšteĭn, G. Š. (1958). On a space of completely additive functions. Vestnik Leningrad. Univ. 13 52-59. · Zbl 0082.11001
[15] Koenker, R. and Mizera, I. (2010). Quasi-convex density estimation. Ann. Statist. 38 2998-3027. · Zbl 1200.62031
[16] Mallows, C. L. (1972). A note on asymptotic joint normality. Ann. Math. Statist. 43 508-515. · Zbl 0238.60017
[17] Pal, J., Woodroofe, M. and Meyer, M. (2007). Estimating a Polya frequency function 2 . In Complex Datasets and Inverse Problems: Tomography, Networks and Beyond ( R. Liu, W. Strawderman and C. H. Zhang, eds.). IMS Lecture Notes and Monograph Series 54 239-249. IMS, Beachwood, OH.
[18] Patilea, V. (2001). Convex models, MLE and misspecification. Ann. Statist. 29 94-123. · Zbl 1029.62020
[19] Pfanzagl, J. (1990). Large deviation probabilities for certain nonparametric maximum likelihood estimators. Ann. Statist. 18 1868-1877. · Zbl 0721.62048
[20] Pollard, D. (1990). Empirical Processes: Theory and Applications. NSF-CBMS Regional Conference Series in Probability and Statistics 2 . IMS, Hayward, CA. · Zbl 0741.60001
[21] Price, K., Storn, R. and Lampinen, J. (2005). Differential Evolution: A Practical Approach to Global Optimization . Springer, Berlin. · Zbl 1186.90004
[22] Rufibach, K. (2006). Log-concave density estimation and bump hunting for i.i.d. observations. Ph.D. thesis, Dept. Mathematics and Statistics, Univ. Bern.
[23] Schuhmacher, D. and Dümbgen, L. (2010). Consistency of multivariate log-concave density estimators. Statist. Probab. Lett. 80 376-380. · Zbl 1181.62048
[24] Schuhmacher, D., Hüsler, A. and Dümbgen, L. (2009). Multivariate log-concave distributions as a nearly parametric model. Technical Report 74, IMSV, Univ. Bern. Available at . · Zbl 1245.62060
[25] Seregin, A. and Wellner, J. A. (2010). Nonparametric estimation of multivariate convex-transformed densities. Ann. Statist. 38 3751-3781. · Zbl 1204.62058
[26] van der Vaart, A. W. and Wellner, J. A. (1996). Weak Convergence and Empirical Processes, with Applications to Statistics . Springer, New York. · Zbl 0862.60002
[27] Villani, C. (2003). Topics in Optimal Transportation. Graduate Studies in Mathematics 58 . Amer. Math. Soc., Providence, RI. · Zbl 1106.90001
[28] Walther, G. (2009). Inference and modeling with log-concave distributions. Statist. Sci. 24 319-327. · Zbl 1329.62192
This reference list is based on information provided by the publisher or from digital mathematics libraries. Its items are heuristically matched to zbMATH identifiers and may contain data conversion errors. It attempts to reflect the references listed in the original paper as accurately as possible without claiming the completeness or perfect precision of the matching.