Fusing multimodal biometrics with quality estimates via a Bayesian belief network. (English) Zbl 1132.68649

Summary: Biometric systems for today’s high security applications must meet stringent performance requirements; fusing multiple biometrics can help lower system error rates. Fusion methods include processing biometric modalities sequentially until an acceptable match is obtained, using logical (AND/OR) operations, or summing similarity scores. More sophisticated methods combine scores from separate classifiers for each modality. This paper develops a novel fusion architecture based on Bayesian belief networks. Although Bayesian update methods have been used before, our approach more fully exploits the graphical structure of Bayes nets to define and explicitly model statistical dependencies between relevant variables: per sample measurements, such as match scores and corresponding quality estimates, and global decision variables. These statistical dependencies are in the form of conditional distributions which we model as Gaussian, gamma, log-normal or beta, each of which is determined by its mean and variance, thus significantly reducing training data requirements. Moreover, by conditioning decision variables on quality as well as match score, we can extract information from lower quality measurements rather than rejecting them out of hand. Another feature of our method is a global quality measure designed to be used as a confidence estimate supporting decision making. Preliminary studies using the architecture to fuse fingerprints and voice are reported.


68T10 Pattern recognition, speech recognition


Full Text: DOI


[1] Jain, A.K.; Prabhakar, S.; Chen, S., Combining multiple matchers for a high security fingerprint verification system, Pattern recognition lett., 20, 1371-1379, (1999)
[2] Ross, A.; Jain, A.K., Information fusion in biometrics, Pattern recognition lett., 24, 2115-2125, (2003)
[3] Kittler, J., On combining classifiers, IEEE trans. pattern anal. Mach. intell., 20, 3, 226-239, (1998)
[4] Bigun, J., Multimodal biometric authentication using quality signals in mobile communications, (), 2-13
[5] Rukhin, A.L.; Malioutov, I., Fusion of biometric algorithms in the recognition problem, Pattern recognition lett., 26, 679-684, (2005)
[6] Verlinde, P.; Chollet, G.; Acheroy, M., Multimodal identity verification using expert fusion, Inf. fusion, 1, 1, 17-33, (2000)
[7] J. Fierrez-Aguilar, J. Ortega-Garcia, J. Gonzalez-Rodriguez, Fusion strategies in multimodal biometric verification, in: International Conference, Proceedings on Multimedia and Expo (ICME ’03), vol. 3(6-9), 2003, pp. 5-8. · Zbl 1050.68722
[8] Fierrez-Aguilar, J., Kernel-based multimodal biometric verification using quality signals, (), 544-554
[9] B. Gutschoven, P. Verlinde, Multimodal identity verification using support vector machines (SVM), in: Proceedings of the Third International Conference on Information Fusion, vol. 2, 2000, pp. 3-8.
[10] E. Tabassi, C. Wilson, C. Watson, Fingerprint image quality, Technical Report 7151, 2004, (Appendices for NISTIR 7151 can be found at \(\langle\)http://fingerprint.nist.gov/NFIS⟩).
[11] Y. Chen, S. Dass, A.J. Jain, Fingerprint quality indices for predicting authentication performance, in: T. Kanade, A.K. Jain, N.K. Ratha (Eds.), Fifth International Conference AVBPA Proceedings, Springer Lecture Notes in Computer Science, vol. 3546, 2005, pp. 160-170.
[12] Wein, L.M.; Baveja, M., Using fingerprint image quality to improve the identification performance of the U.S. visitor and immigrant status indicator technology program, Proc. natl. acad. sci., 102, 21, 7772-7775, (2005)
[13] K. Nandakumar, Y. Chen, A.K. Jain, S.C. Dass, Quality-based score level fusion in multibiometric systems, in: Proceedings of the 18th International Conference on Pattern Recognition (ICPR06), 2006, pp. 473-476.
[14] Fierrez-Aguilar, J.; Ortega-Garcia, J.; Gonzales-Rodriguez, J., Discriminative multimodal biometric authentication based on quality measures, Pattern recognition, 38, 777-779, (2005)
[15] Baker, J.P.; Maurer, D.E., Fusion of biometric data with quality estimates via a Bayesian belief network, (), 21-22
[16] J. Richiardi, P. Prodanov, A. Drygajlo, A probabilistic measure of modality reliability in speaker verification, in: Proceedings of the IEEE International Conference on Acoustics, Speech, and Signal Processing, ICASSP ’05, vol. 1, 2005, pp. 709-712.
[17] Teoh, A.B.J.; Samad, S.A.; Hussain, A., A face and speech biometric verification system using a simple Bayesian structure, J. inf. sci. eng., 21, 1121-1137, (2005)
[18] E.S. Bigun, J. Bigun, B. Duc, S. Fischer, Expert conciliation for multi modal person authentication systems by Bayesian statistics, in: J. Bigun, G. Chollet, G. Borgefors (Eds.), First International Conference AVBPA Proceedings, Springer Lecture Notes in Computer Science, vol. 1206, 1997, pp. 291-300.
[19] Jensen, F.V., Bayesian networks and decision graphs, (2001), Springer New York · Zbl 0973.62005
[20] Prabhakar, S.; Jain, A.K., Decision-level fusion in fingerprint verification, Pattern recognition, 35, 861-874, (2002) · Zbl 0999.68579
[21] P. Domingos, M. Pazzani, Beyond independence conditions for the optimality of the simple Bayesian classifier, in: Proceedings of the 13th International Conference on Machine Learning (ICML 96), 1996, pp. 105-112.
[22] S.C. Dass, K. Nandakumar, A.K. Jain, A principled approach to score level fusion in multimodal biometric systems, in: T. Kanade, A.K. Jain, N.K. Ratha (Eds.), Fifth International Conference AVBPA Proceedings, Springer Lecture Notes in Computer Science, vol. 3546, 2005, pp. 1049-1058
[23] \(\langle\)http://www.int-evry.fr/biometrics/BMEC2007/⟩.
[24] C. Watson, NIST Special Database 14: 8-bit Gray Scale Images of Mated Fingerprint Card Pairs 2, CD-ROM & Documentation, 1993.
[25] C. Watson, NIST Fingerprint Image Software 2 (NFIS2), Rel. 28-2.2, CDROM and Documentation, 2004.
[26] K. Messer, J. Matas, J. Kittler, J. Luettin, and G. Maitre, XM2VTSdb: The Extended M2VTS Database, in: Second International Conference AVBPA Proceedings, Washington, DC, 1999, pp. 166-171 (Can be found at \(\langle\)http://www.ee.surrey.ac.uk/CVSSP/Publications⟩).
[27] N. Poh, S. Bengio, Database, protocol and tools for evaluating score-level fusion algorithms in biometric authentication, in: T. Kanade, A.K. Jain, N.K. Ratha (Eds.), Fifth International Conference AVBPA Proceedings, Springer Lecture Notes in Computer Science, vol. 3546, 2005, pp. 1059-1070.
[28] National Institute of Standards and Technology (NIST) Speech Quality Assurance (SPQA) Package 2.3 Documentation, URL: \(\langle\)http://www.nist.gov/speech/tools/spqa_23sphere25tarZ.htm⟩.
[29] T. Scheffer, R. Herbrich, Unbiased assessment of learning algorithms, in: Proceedings of the 15th International Joint Conference on Artificial Intelligence, 1997, pp. 798-903.
[30] C.L. Wilson, M.D. Garris, C.I. Watson, Matching performance for the US_VISIT IDENT system using flat fingerprints, National Institute of Standards and Technology, NIST Internal Report 7110, 2004.
[31] C.L. Wilson, C.I. Watson, M.D. Garris, A. Hicklin, Using the NIST Verification Test Bed (VTB), National Institute of Standards and Technology, NIST Internal Report 7020, 2003.
This reference list is based on information provided by the publisher or from digital mathematics libraries. Its items are heuristically matched to zbMATH identifiers and may contain data conversion errors. It attempts to reflect the references listed in the original paper as accurately as possible without claiming the completeness or perfect precision of the matching.