Information theoretic novelty detection. (English) Zbl 1187.68450

Summary: We present a novel approach to online change detection problems when the training sample size is small. The proposed approach is based on estimating the expected information content of a new data point and allows an accurate control of the false positive rate even for small data sets. In the case of the Gaussian distribution, our approach is analytically tractable and closely related to classical statistical tests. We then propose an approximation scheme to extend our approach to the case of the mixture of Gaussians. We evaluate extensively our approach on synthetic data and on three real benchmark data sets. The experimental validation shows that our method maintains a good overall accuracy, but significantly improves the control over the false positive rate.


68T10 Pattern recognition, speech recognition


Full Text: DOI Link


[1] Anderson, T. W.; Anderson, T. W., An Introduction to Multivariate Statistical Analysis (1984), Wiley-Interscience · Zbl 0651.62041
[4] Barnett, V.; Lewis, T., Outliers in Statistical Data, (Wiley Series in Probability and Statistics (1994), Wiley: Wiley New York) · Zbl 0801.62001
[5] Bishop, C. M., Novelty detection and neural network validation, IEE Proceedings on Vision, Image and Signal processing, 141, 4, 217-222 (1994)
[10] Eguchi, S.; Copas, J., Interpreting Kullback-Leibler divergence with the Neyman-Pearson lemma, Journal of Multivariate Analysis, 97, 9, 2034-2040 (2006) · Zbl 1101.62004
[12] Fisher, R. A., The use of multiple measurements in taxonomic problems, Annals Eugenics, 7, 179-188 (1936)
[13] Hayton, P.; Schölkopf, B.; Tarassenko, L.; Anuzis, P., Support vector novelty detection applied to jet engine vibration spectra, (Leen, T. K.; Dietterich, T. G.; Tresp, V., Advances in Neural Information Processing Systems, NIPS 2000, vol. 13 (2000), MIT Press: MIT Press Cambridge), 946-952
[14] He, C.; Girolami, M.; Ross, G., Employing optimized combinations of one-class classifiers for automated currency validation, Pattern Recognition, 37, 6, 1085-1096 (2004)
[16] Horton, P.; Nakai, K., A probabilistic classification system for predicting the cellular localization sites of proteins, (Proceedings of the Fourth International Conference on Intelligent Systems for Molecular Biology (1996), AAAI Press), 109-115
[18] Markou, M.; Singh, S., Novelty detection: a review—part 1: statistical approaches, Signal Processing, 83, 12, 2481-2497 (2003) · Zbl 1145.94402
[19] Martinez, D., Neural tree density estimation for novelty detection, IEEE Transactions on Neural Networks, 9, 2, 330-338 (1998)
[20] Quinn, J. A.; Williams, C. K.I., Known unknowns: novelty detection in condition monitoring, (Martí, J.; Benedí, J. M.; Mendonça, A. M.; Serrat, J., Third Iberian Conference on Pattern Recognition and Image Analysis, IbPRIA 2007, Lecture Notes in Computer Science, vol. 4477 (June 2007), Springer: Springer Berlin), 1-6
[21] Roberts, S. J., Novelty detection using extreme value statistics, IEE Proceedings on Vision, Image and Signal Processing, 146, 3, 124-129 (1999)
[22] Schölkopf, B.; Williamson, R. C.; Smola, A. J.; Taylor, J. S.; Platt, J. C., Support vector method for novelty detection, (Solla, S. A.; Leen, T. K.; Müller, K. R.; Solla, S. A.; Leen, T. K.; Müller, K. R., Advances in Neural Information Processing Systems, NIPS 1999, vol. 12 (1999), The MIT Press: The MIT Press Cambridge), 582-588
[23] Singer, Y.; Warmuth, M. K., Batch and on-line parameter estimation of Gaussian mixtures based on the joint entropy, (Kearns, M. J.; Solla, S. A.; Cohn, D. A., Advances in Neural Information Processing Systems, NIPS 1998, vol. 11 (1998), The MIT Press: The MIT Press Cambridge), 578-584
This reference list is based on information provided by the publisher or from digital mathematics libraries. Its items are heuristically matched to zbMATH identifiers and may contain data conversion errors. It attempts to reflect the references listed in the original paper as accurately as possible without claiming the completeness or perfect precision of the matching.