Information theoretic novelty detection. (English) Zbl 1187.68450

Filippone, Maurizio; Sanguinetti, Guido

Pattern Recognition 43, No. 3, 805-814 (2010).

Summary: We present a novel approach to online change detection problems when the training sample size is small. The proposed approach is based on estimating the expected information content of a new data point and allows an accurate control of the false positive rate even for small data sets. In the case of the Gaussian distribution, our approach is analytically tractable and closely related to classical statistical tests. We then propose an approximation scheme to extend our approach to the case of the mixture of Gaussians. We evaluate extensively our approach on synthetic data and on three real benchmark data sets. The experimental validation shows that our method maintains a good overall accuracy, but significantly improves the control over the false positive rate.

MSC:

68T10

Pattern recognition, speech recognition

Keywords:

novelty detection; information theory; mixture of gaussians; density estimation

Software:

PRMLT; UCI-ml

References:

[1]	Anderson, T. W.; Anderson, T. W., An Introduction to Multivariate Statistical Analysis (1984), Wiley-Interscience · Zbl 0651.62041
[4]	Barnett, V.; Lewis, T., Outliers in Statistical Data, (Wiley Series in Probability and Statistics (1994), Wiley: Wiley New York) · Zbl 0801.62001
[5]	Bishop, C. M., Novelty detection and neural network validation, IEE Proceedings on Vision, Image and Signal processing, 141, 4, 217-222 (1994)
[10]	Eguchi, S.; Copas, J., Interpreting Kullback-Leibler divergence with the Neyman-Pearson lemma, Journal of Multivariate Analysis, 97, 9, 2034-2040 (2006) · Zbl 1101.62004
[12]	Fisher, R. A., The use of multiple measurements in taxonomic problems, Annals Eugenics, 7, 179-188 (1936)
[13]	Hayton, P.; Schölkopf, B.; Tarassenko, L.; Anuzis, P., Support vector novelty detection applied to jet engine vibration spectra, (Leen, T. K.; Dietterich, T. G.; Tresp, V., Advances in Neural Information Processing Systems, NIPS 2000, vol. 13 (2000), MIT Press: MIT Press Cambridge), 946-952
[14]	He, C.; Girolami, M.; Ross, G., Employing optimized combinations of one-class classifiers for automated currency validation, Pattern Recognition, 37, 6, 1085-1096 (2004)
[16]	Horton, P.; Nakai, K., A probabilistic classification system for predicting the cellular localization sites of proteins, (Proceedings of the Fourth International Conference on Intelligent Systems for Molecular Biology (1996), AAAI Press), 109-115
[18]	Markou, M.; Singh, S., Novelty detection: a review—part 1: statistical approaches, Signal Processing, 83, 12, 2481-2497 (2003) · Zbl 1145.94402
[19]	Martinez, D., Neural tree density estimation for novelty detection, IEEE Transactions on Neural Networks, 9, 2, 330-338 (1998)
[20]	Quinn, J. A.; Williams, C. K.I., Known unknowns: novelty detection in condition monitoring, (Martí, J.; Benedí, J. M.; Mendonça, A. M.; Serrat, J., Third Iberian Conference on Pattern Recognition and Image Analysis, IbPRIA 2007, Lecture Notes in Computer Science, vol. 4477 (June 2007), Springer: Springer Berlin), 1-6
[21]	Roberts, S. J., Novelty detection using extreme value statistics, IEE Proceedings on Vision, Image and Signal Processing, 146, 3, 124-129 (1999)
[22]	Schölkopf, B.; Williamson, R. C.; Smola, A. J.; Taylor, J. S.; Platt, J. C., Support vector method for novelty detection, (Solla, S. A.; Leen, T. K.; Müller, K. R.; Solla, S. A.; Leen, T. K.; Müller, K. R., Advances in Neural Information Processing Systems, NIPS 1999, vol. 12 (1999), The MIT Press: The MIT Press Cambridge), 582-588
[23]	Singer, Y.; Warmuth, M. K., Batch and on-line parameter estimation of Gaussian mixtures based on the joint entropy, (Kearns, M. J.; Solla, S. A.; Cohn, D. A., Advances in Neural Information Processing Systems, NIPS 1998, vol. 11 (1998), The MIT Press: The MIT Press Cambridge), 578-584