Consistency of logistic classifier in abstract Hilbert spaces. (English) Zbl 1418.62075

Let \(E\) be an infinite-dimensional separable Hilbert space. Let \(X\) denote an \(E\)-valued random variable and \(Y\) a further random variable, taking values in \(\{-1, +1\}\), where \(X\) and \(Y\) are defined on the same probability space \((\Omega, \mathcal{F}, \mathbb{P})\). Assume a logistic model of the form \[ \mathbb{P}(Y = +1 | X = x) = p_{\theta_0}(x) = \frac{1}{1 + e^{-\langle \theta_0, x \rangle}}, \] where \(\langle \cdot, \cdot \rangle\) denotes the inner product in \(E\) and \(\theta_0\) is an unknown element of \(E\). Furthermore, let \((X_1, Y_1), \ldots, (X_n, Y_n)\) be a sample of independent \(E \times \{-1, +1\}\)-valued observables, such that \((X_1, Y_1)\) is distributed as \((X, Y)\).
The authors are concerned with estimating \(\theta_0\) by means of the quasi-maximum likelihood method, along some fixed sequence \((E_k)_{k}\) of linear subspaces of \(E\) with \(\dim E_k = k\), where \(k = k_n\) depends on the sample size \(n\). Two sets of conditions are derived for the consistency of the resulting estimator \(\hat{\theta}_{k_n, n}\) as \(n\) tends to infinity. The first set of conditions refers to the distribution of \(X\), and the second set of conditions refers to the growth rate of \(k_n\) as a function of \(n\). Finally, a simulation study is presented to illustrate the necessity of the conditions.


62F12 Asymptotic properties of parametric estimators
62H30 Classification and discrimination; cluster analysis (statistical aspects)
60B12 Limit theorems for vector-valued random variables (infinite-dimensional case)
62J12 Generalized linear models (logistic models)


fda (R)
Full Text: DOI Euclid


[1] Albert, A. & Anderson, J. A. (1984). On the existence of maximum likelihood estimates in logistic regression models., Biometrika71(1), 1-10. · Zbl 0543.62020 · doi:10.1093/biomet/71.1.1
[2] Bertsekas, D. P., Nedic, A. & Ozdaglar, A. E. (2003)., Convex Analysis and Optimization. Athena Scientific. · Zbl 1140.90001
[3] Chen, K., Hu, I. & Ying, Z. (1999). Strong consistency of maximum quasi-likelihood estimators in generalized linear models with fixed and adaptive designs., Ann. Statist.27(4), 1155-1163. · Zbl 0957.62056 · doi:10.1214/aos/1018031098
[4] Escabias, M., Aguilera, A. M. & Valderrama, M. J. (2007). Functional PLS logit regression model., Comput. Statist. Data Anal.51(10), 4891-4902. · Zbl 1162.62392 · doi:10.1016/j.csda.2006.08.011
[5] Fan, J. & Song, R. (2010). Sure independence screening in generalized linear models with NP-dimensionality., Ann. Statist.38(6), 3567-3604. · Zbl 1206.68157 · doi:10.1214/10-AOS798
[6] Van de Geer, S.A. (2008). High-dimensional generalized linear models and the Lasso., Ann. Statist.38(2), 614-645. · Zbl 1138.62323 · doi:10.1214/009053607000000929
[7] Kallenberg, O. (2001)., Foundations of Modern Probability, 2nd edn. Springer. · Zbl 0996.60001
[8] Kazakeviciute, A. & Olivo, M. (2016). A study of logistic classifier: uniform consistency in finite-dimensional linear spaces., Journal of Mathematics, Statistics and Operations Research3(2), 1-7.
[9] Kazakeviciute, A. & Olivo, M. (2017). Point separation in logistic regression on Hilbert space-valued variables., Statist. Probab. Lett., 128, 84-88. · Zbl 1384.62268 · doi:10.1016/j.spl.2017.04.019
[10] Kazakeviciute, A., Kazakevicius, V. & Olivo, M. (2017). Conditions for existence of uniformly consistent classifiers., IEEE Trans. Inform. Theory, 63(6), 3425-3432. · Zbl 1369.94191 · doi:10.1109/TIT.2017.2696961
[11] Liang, H. & Du, P. (2012). Maximum likelihood estimation in logistic regression models with a diverging number of covariates., Electron. J. Stat.6, 1838-1846. · Zbl 1295.62021 · doi:10.1214/12-EJS731
[12] Müller, H. G. & Stadtmüller, S. (2005). Generalized Functional Linear Models., Ann. Statist.32(2), 774-805. · Zbl 1068.62048
[13] Ramsay, J. O. & Silverman, B. W. (2002)., Applied Functional Data Analysis: Methods and Case Studies. Springer. · Zbl 1011.62002
[14] Ramsay, J. O. & Silverman, B. W. (2005)., Functional data analysis. Springer. · Zbl 1079.62006
[15] van der Vaart, A.W. (2000)., Asymptotic Statistics. Cambridge University Press. · Zbl 0910.62001
[16] van Ryzin, J. (1966). Bayes risk consistency of classification procedures using density estimation., Sankhya: The Indian Journal of Statistics, Series A., 28(2/3), 261-270. · Zbl 0192.25703
[17] Wang, L. (2011). GEE analysis of clustered binary data with diverging number of covariates., Ann. Statist.39(1), 389-417. · Zbl 1209.62138 · doi:10.1214/10-AOS846
This reference list is based on information provided by the publisher or from digital mathematics libraries. Its items are heuristically matched to zbMATH identifiers and may contain data conversion errors. In some cases that data have been complemented/enhanced by data from zbMATH Open. This attempts to reflect the references listed in the original paper as accurately as possible without claiming completeness or a perfect matching.