×

Feature representation and discrimination based on Gaussian mixture model probability densities – practices and algorithms. (English) Zbl 1095.68672

Summary: We investigate the use of statistical methods in specific classification tasks since statistical methods have certain advantages which advocate their use in pattern recognition. One central problem in statistical methods is estimation of class conditional probability density functions based on examples in a training set. In this study maximum likelihood estimation methods for Gaussian mixture models are reviewed and discussed from a practical point of view. In addition, good practices for utilizing probability densities in feature classification and selection are discussed for Bayesian and, more importantly, for non-Bayesian tasks. As a result, the use of confidence information in the classification is proposed and a method for confidence estimation is presented. The propositions are tested experimentally.

MSC:

68T10 Pattern recognition, speech recognition

Software:

GMMBAYES; MVELPS
PDFBibTeX XMLCite
Full Text: DOI

References:

[1] Bishop, C., Neural Networks for Pattern Recognition (1995), Oxford University Press: Oxford University Press Oxford
[2] Cristiani, N.; Shawe-Taylor, J., An Introduction to Support Vector Machines and Other Kernel-Based Learning Methods (2000), Cambridge University Press: Cambridge University Press Cambridge
[3] Freund, Y.; Schapire, R. E., A decision theoretic generalization of on-line learning and an application to boosting, J. Comput. Syst. Sci., 55, 1, 119-139 (1997) · Zbl 0880.68103
[4] Hamouz, M.; Kittler, J.; Kamarainen, J.-K.; Paalanen, P.; Kälviäinen, H., Affine-invariant face detection and localization using GMM-based feature detector and enhanced appearance model, (Proceedings of the 6th International Conference on Automatic Face and Gesture Recognition. Proceedings of the 6th International Conference on Automatic Face and Gesture Recognition, Seoul, Korea (2004)), 67-72
[5] Hamouz, M.; Kittler, J.; Kamarainen, J.-K.; Paalanen, P.; Kälviäinen, H., J. Matas, Feature-based affine-invariant detection and localization of faces, IEEE Trans. Pattern Anal. Mach. Intell., 27, 9, 1490-1495 (2005)
[6] Schlesinger, M.; Hlavac, V., Ten Lectures on Statistical and Structural Pattern Recognition, Computational Imaging and Vision (2002), Kluwer Academic Publishers: Kluwer Academic Publishers Dordrecht · Zbl 1087.62505
[7] Kyrki, V.; Kamarainen, J.-K.; Kälviäinen, H., Simple Gabor feature space for invariant object recognition, Pattern Recognition Lett., 25, 3, 311-318 (2003)
[8] GMMBAYES Matlab Toolbox, \( \langle;\) http://www.it.lut.fi/project/gmmbayes \(\rangle;\); GMMBAYES Matlab Toolbox, \( \langle;\) http://www.it.lut.fi/project/gmmbayes \(\rangle;\)
[9] Tong, Y., The multivariate normal distribution, Springer Series in Statistics (1990), Springer: Springer Berlin
[10] Goodman, N., Statistical analysis based on a certain multivariate complex Gaussian distribution (an introduction), Ann. Math. Stat., 34, 1, 152-177 (1963) · Zbl 0122.36903
[11] Ilonen, J.; Kamarainen, J.-K.; Kälviäinen, H.; Anttalainen, O., Automatic detection and recognition of hazardous chemical agents, (Proceedings of the 14th International Conference on Digital Signal Processing, vol. 2 (2002)), 1345-1348
[12] Lindh, T.; Ahola, J.; Kamarainen, J.-K.; Kyrki, V.; Partanen, J., Bearing damage detection based on statistical discrimination of stator current, (Proceedings of the 4th IEEE International Symposium on Diagnostics for Electric Machines, Power Electronics and Drives. Proceedings of the 4th IEEE International Symposium on Diagnostics for Electric Machines, Power Electronics and Drives, Atlanta, Georgia, USA (2003)), 177-181
[13] Everitt, B.; Hand, D., Finite Mixture Distributions Monographs on Applied Probability and Statistics (1981), Chapman & Hall: Chapman & Hall London
[14] J.-K. Kämäräinen, Feature extraction using Gabor filters, Ph.D. Thesis, Lappeenranta University of Technology, 2003.; J.-K. Kämäräinen, Feature extraction using Gabor filters, Ph.D. Thesis, Lappeenranta University of Technology, 2003.
[15] J. Bilmes, A gentle tutorial on the EM algorithm and its application to parameter estimation for Gaussian mixture and hidden Markov models, 1997, URL: \( \langle;\) citeseer.nj.nec.com/bilmes98gentle.html \(\rangle;\); J. Bilmes, A gentle tutorial on the EM algorithm and its application to parameter estimation for Gaussian mixture and hidden Markov models, 1997, URL: \( \langle;\) citeseer.nj.nec.com/bilmes98gentle.html \(\rangle;\)
[16] Duda, R.; Hart, P.; Stork, D., Pattern Classification (2001), Wiley: Wiley New York · Zbl 0968.68140
[17] Figueiredo, M.; Jain, A., Unsupervised learning of finite mixture models, IEEE Trans. Pattern Anal. Mach. Intell., 24, 3, 381-396 (2002)
[18] Theodoridis, S.; Koutroumbas, K., Pattern Recognition (1999), Academic Press: Academic Press New York
[19] Verbeek, J. J.; Vlassis, N.; Kröse, B., Efficient greedy learning of Gaussian mixture models, Neural Comput., 5, 2, 469-485 (2003) · Zbl 1047.68114
[20] Nagy, I.; Nedoma, P.; Karny, M., Factorized EM algorithm for mixture estimation, (Proceedings of the International Conference on Artificial Neural Networks and Genetic Algorithms. Proceedings of the International Conference on Artificial Neural Networks and Genetic Algorithms, Wien (2001)) · Zbl 1011.68107
[21] Jaaskelainen, T.; Silvennoinen, R.; Hiltunen, J.; Parkkinen, J. P.S., Classification of the reflectance spectra of pine, spruce, and birch, Appl. Opt., 33, 12, 2356-2362 (1994)
[22] Kamarainen, J.-K.; Kyrki, V.; Ilonen, J.; Kälviäinen, H., Improving similarity measures of histograms using smoothing projections, Pattern Recognition Lett., 24, 12, 2009-2019 (2003)
[23] P. Paalanen, J.-K. Kamarainen, J. Ilonen, H. Kälviäinen, Feature representation and discrimination based on Gaussian mixture model probability densities—practices and algorithms, Research Report 95, Department of Information Technology, Lappeenranta University of Technology, 2005.; P. Paalanen, J.-K. Kamarainen, J. Ilonen, H. Kälviäinen, Feature representation and discrimination based on Gaussian mixture model probability densities—practices and algorithms, Research Report 95, Department of Information Technology, Lappeenranta University of Technology, 2005. · Zbl 1095.68672
[24] L. Breiman, J. Friedman, R. Olshen, C. Stone, Classification and Regression Trees, Wadsworth International Group, Belmont, California, 1984, pp. 43-49.; L. Breiman, J. Friedman, R. Olshen, C. Stone, Classification and Regression Trees, Wadsworth International Group, Belmont, California, 1984, pp. 43-49. · Zbl 0541.62042
[25] Hyndman, R. J., Computing and graphing highest density regions, Am. Stat., 50, 2, 120-126 (1996)
[26] Somerville, P. N., Numerical computation of multivariate normal and multivariate-\(t\) probabilities over convex regions, J. Comput. Graphical Stat., 7, 4, 529-544 (1998)
[27] P.N. Somerville, Numerical computation of multivariate normal and multivariate \(t\); P.N. Somerville, Numerical computation of multivariate normal and multivariate \(t\)
[28] Tee, G. J., Surface area and capacity of ellipsoids in \(n\) dimensions, New Zeal. J. Math., 34, 165-198 (2005) · Zbl 1115.41031
This reference list is based on information provided by the publisher or from digital mathematics libraries. Its items are heuristically matched to zbMATH identifiers and may contain data conversion errors. In some cases that data have been complemented/enhanced by data from zbMATH Open. This attempts to reflect the references listed in the original paper as accurately as possible without claiming completeness or a perfect matching.