×

zbMATH — the first resource for mathematics

Pairwise local Fisher and naive Bayes: improving two standard discriminants. (English) Zbl 1456.62062
Summary: The Fisher discriminant is probably the best known likelihood discriminant for continuous data. Another benchmark discriminant is the naive Bayes, which is based on marginals only. In this paper we extend both discriminants by modeling dependence between pairs of variables. In the continuous case this is done by local Gaussian versions of the Fisher discriminant. In the discrete case the naive Bayes is extended by taking geometric averages of pairwise joint probabilities. We also indicate how the two approaches can be combined for mixed continuous and discrete data. The new discriminants show promising results in a number of simulation experiments and real data illustrations.
MSC:
62G07 Density estimation
62G20 Asymptotic properties of nonparametric inference
62H30 Classification and discrimination; cluster analysis (statistical aspects)
62P20 Applications of statistics to economics
Software:
gamair; lg; localgauss; np; R
PDF BibTeX XML Cite
Full Text: DOI
References:
[1] Aas, K.; Czado, C.; Frigessi, A.; Bakken, H., Pair-copula constructions of multiple dependence, Insurance Math. Econom., 44, 2, 182-198 (2009) · Zbl 1165.60009
[2] Aggarwal, C. C.; Zhai, C., A survey of text classification algorithms, (Mining Text Data (2012), Springer), 163-222
[3] Aitchison, J.; Aitken, C. G., Multivariate binary discrimination by the kernel method, Biometrika, 63, 3, 413-420 (1976) · Zbl 0344.62035
[4] Azzalini, A., A note on the estimation of a distribution function and quantiles by a kernel method, Biometrika, 68, 1, 326-328 (1981)
[5] Berentsen, G. D.; Kleppe, T. S.; Tjøstheim, D., Introducing localgauss, an R-package for estimating and visualising local Gaussian correlation, J. Stat. Softw., 56, 1, 1-18 (2014)
[6] Berentsen, G. D.; Støve, B.; Tjøstheim, D.; Nordbø, T., Recognizing and visualizing copulas: an approach using local Gaussian approximation, Insurance Math. Econom., 57, 90-103 (2014) · Zbl 1304.62085
[7] Berentsen, G. D.; Tjøstheim, D., Recognizing and visualizing departures from independence in bivariate data using local Gaussian correlation, Stat. Comput., 24, 5, 785-801 (2014) · Zbl 1322.62140
[8] Blanzieri, E.; Bryl, A., A survey of learning-based techniques of email spam filtering, Artif. Intell. Rev., 29, 1, 63-92 (2008)
[9] Box, G. E.P.; Tiao, G. C., Bayesian Inference in Statistical Analysis (1973), John Wiley & Sons · Zbl 0271.62044
[10] Brier, G. W., Verification of forecasts expressed in terms of probability, Mon. Weather Rev., 78, 1, 1-3 (1950)
[11] Burman, P., A comparative study of ordinary cross-validation, v-fold cross-validation and the repeated learning-testing methods, Biometrika, 76, 3, 503-514 (1989) · Zbl 0677.62065
[12] Chaudhuri, P.; Ghosh, A. K.; Oja, H., Classification based on hybridization of parametric and nonparametric classifiers, IEEE Trans. Pattern Anal. Mach. Intell., 31, 7, 1153-1164 (2009)
[13] Fawcett, T., An introduction to roc analysis, Pattern Recognit. Lett., 27, 8, 861-874 (2006)
[14] Fisher, R. A., The use of multiple measurements in taxonomic problems, Ann. Eugen., 7, 2, 179-188 (1936)
[15] Ghosh, A. K.; Chaudhuri, P., Optimal smoothing in kernel discriminant analysis, Statist. Sinica, 14, 457-483 (2004) · Zbl 1045.62024
[16] Ghosh, A.; Hall, P., On error rate estimation in nonparametric classification, Statist. Sinica, 18, 1081-1100 (2008) · Zbl 1149.62026
[17] Hall, P.; Racine, J.; Li, Q., Cross-validation and the estimation of probability densities, J. Amer. Statist. Assoc., 99, 99, 1015-1026 (2004) · Zbl 1055.62035
[18] Hart, J. D.; Vieu, P., Data-driven bandwidth choice for density estimation based on dependent data, Ann. Statist., 18, 873-890 (1990) · Zbl 0703.62045
[19] Hastie, T. J.; Tibshirani, R. J., Generalized Additive Models (1990), Chapman and Hall: Chapman and Hall London · Zbl 0747.62061
[20] Hastie, T.; Tibshirani, R.; Friedman, J., The Elements of Statistical Learning (2009), Springer: Springer New York
[21] Hayfield, T.; Racine, J. S., Nonparametric econometrics: The np package, J. Stat. Softw., 27, 5, 1-32 (2008)
[22] Hjort, N. L.; Glad, I. K., Nonparametric density estimation with a parametric start, Ann. Statist., 23, 882-904 (1995) · Zbl 0838.62027
[23] Hjort, N.; Jones, M., Locally parametric nonparametric density estimation, Ann. Stat., 24, 1619-1647 (1996) · Zbl 0867.62030
[24] Johnson, R. A.; Wichern, D. W., Applied Multivariate Statistical Analysis, Sixth Edition (2007), Pearson Education Iternational
[25] Jones, M. C.; Signorini, D., A comparison of higher-order bias kernel density estimators, J. Amer. Statist. Assoc., 92, 439, 1063-1073 (1997) · Zbl 0888.62035
[26] Jordanger, L. A.; Tjøstheim, D., Nonlinear spectral analysis: A local Gaussian approach (2019), Preprint arXiv:1708.02166
[27] Jullum, M.; Løland, A.; Huseby, R. B.; Ånonsen, G.; Lorentzen, J. P., Detecting money laundering transactions with machine learning, J. Money Laund. Control, 23, 1, 173-186 (2020)
[28] Kohavi, R., A study of cross-validation and bootstrap for accuracy estimation and model selection, (International Joint Conference on Artificial Intelligence (IJCAI), Vol. 14 (1995), Montreal: Montreal Canada), 1137-1145
[29] Lacal, V.; Tjøstheim, D., Local Gaussian autocorrelation and tests of serial independence, J. Time Series Anal., 38, 1, 51-71 (2017) · Zbl 1356.62145
[30] Lacal, V.; Tjøstheim, D., Estimating and testing nonlinear local dependence between two time series, J. Bus. Econom. Statist., 37, 4, 648-660 (2019)
[31] Li, J.; Cuesta-Albertos, J. A.; Liu, R. Y., Dd-classifier: Nonparametric classification procedure based dd-plot, J. Amer. Statist. Assoc., 107, 498, 737-753 (2012) · Zbl 1261.62058
[32] Li, Q.; Racine, J. S., Nonparametric Econometrics: Theory and Practice (2007), Princeton University Press: Princeton University Press Princeton · Zbl 1183.62200
[33] Li, Q.; Racine, J. S., Nonparametric estimation of conditional cdf and quantile functions with mixed categorical and continuous data, J. Bus. Econom. Statist., 26, 4, 423-434 (2008)
[34] Loader, C. R., Local likelihood density estimation, Ann. Statist., 34, 1602-1618 (1996) · Zbl 0867.62034
[35] Marron, J. S., Optimal rates of convergence to Bayes risk in nonparametric discrimination, Ann. Statist., 11, 4, 1142-1155 (1983) · Zbl 0554.62053
[36] Min, J. H.; Jeong, C., A binary classification method for bankruptcy prediction, Expert Syst. Appl., 36, 3, 5256-5263 (2009)
[37] Nadaraya, E. A., Some new estimates for distribution functions, Theory Probab. Appl., 9, 3, 497-500 (1964) · Zbl 0152.17605
[38] Nelsen, R. B., An Introduction to Copulas (2007), Springer Science & Business Media
[39] Otneim, H., lg: Locally Gaussian distributions: Estimation and methods (2019), R package version 0.4.1
[40] Otneim, H.; Tjøstheim, D., The locally Gaussian density estimator for multivariate data, Stat. Comput., 27, 6, 1595-1616 (2017) · Zbl 1384.62128
[41] Otneim, H.; Tjøstheim, D., Conditional density estimation using the local Gaussian correlation, Stat. Comput., 28, 2, 303-321 (2018) · Zbl 1384.62127
[42] Phua, C.; Lee, V.; Smith, K.; Gayler, R., A comprehensive survey of data mining-based fraud detection research (2010), arXiv preprint arXiv:1009.6119
[43] R: A Language and Environment for Statistical Computing (2018), R Foundation for Statistical Computing: R Foundation for Statistical Computing Vienna, Austria
[44] Ranjan, R.; Gneiting, T., Combining probability forecasts, J. R. Stat. Soc. Ser. B Stat. Methodol., 72, 1, 71-91 (2010) · Zbl 1411.62270
[45] Samworth, R., Optimal weighted nearest neighbour classifiers, Ann. Stat., 40, 2733-2763 (2012) · Zbl 1373.62317
[46] Satabdi, P., A SVM Approach for Classification and Prediction of Credit Rating in the Indian MarketWorking Paper (2018)
[47] Schott, P. A., Reference Guide to Anti-Money Laundering and Combating the Financing of Terrorism (2006), The World Bank
[48] Silverman, B. W., Density Estimation for Statistics and Data Analysis (1986), Chapman and Hall: Chapman and Hall London · Zbl 0617.62042
[49] Stone, C. J., Large-sample inference for log-spline models, Ann. Statist., 717-741 (1990) · Zbl 0712.62036
[50] Stone, C. J.; Hansen, M. H.; Kooperberg, C.; Truong, Y. K., Polynomial splines and their tensor products in extended linear modeling: 1994 wald memorial lecture, Ann. Statist., 25, 4, 1371-1470 (1997) · Zbl 0924.62036
[51] Tjøstheim, D., Improved seismic discrimination using pattern recognition, Phys. Earth Planet. Inter., 16, 85-108 (1978)
[52] Tjøstheim, D.; Hufthammer, K. O., Local Gaussian correlation: A new measure of dependence, J. Econometrics, 172, 33-48 (2013) · Zbl 1443.62288
[53] Wood, S., Generalized Additive Models: An Introduction with R (2017), Chapman and Hall/CRC · Zbl 1368.62004
[54] Zheng, R.; Li, J.; Chen, H.; Huang, Z., A framework for authorship identification of online messages: Writing style features and classification techniques, J. Am. Soc. Inf. Sci. Technol., 57, 3, 378-393 (2006)
This reference list is based on information provided by the publisher or from digital mathematics libraries. Its items are heuristically matched to zbMATH identifiers and may contain data conversion errors. It attempts to reflect the references listed in the original paper as accurately as possible without claiming the completeness or perfect precision of the matching.