×

zbMATH — the first resource for mathematics

Facial expression recognition under difficult conditions: a comprehensive study on edge directional texture patterns. (English) Zbl 06943871
Summary: In recent years, research in automated facial expression recognition has attained significant attention for its potential applicability in human-computer interaction, surveillance systems, animation, and consumer electronics. However, recognition in uncontrolled environments under the presence of illumination and pose variations, low-resolution video, occlusion, and random noise is still a challenging research problem. In this paper, we investigate recognition of facial expression in difficult conditions by means of an effective facial feature descriptor, namely the directional ternary pattern (DTP). Given a face image, the DTP operator describes the facial feature by quantizing the eight-directional edge response values, capturing essential texture properties, such as presence of edges, corners, points, lines, etc. We also present an enhancement of the basic DTP encoding method, namely the compressed DTP (cDTP) that can describe the local texture more effectively with fewer features. The recognition performances of the proposed DTP and cDTP descriptors are evaluated using the Cohn-Kanade (CK) and the Japanese female facial expression (JAFFE) database. In our experiments, we simulate difficult conditions using original database images with lighting variations, low-resolution images obtained by down-sampling the original, and images corrupted with Gaussian noise. In all cases, the proposed method outperforms some of the well-known face feature descriptors.
MSC:
68 Computer science
94 Information and communication theory, circuits
Software:
Cohn-Kanade; JAFFE
PDF BibTeX XML Cite
Full Text: DOI
References:
[1] Ahmed, F. (2012). Gradient directional pattern: A robust feature descriptor for facial expression recognition, IET Electronics Letters 48(19): 1203-1204.
[2] Ahmed, F. and Kabir, M.H. (2012a). Directional ternary pattern (DTP) for facial expression recognition, IEEE International Conference on Consumer Electronics, Berlin, Germany, pp. 265-266.
[3] Ahmed, F. and Kabir, M.H. (2012b). Facial feature representation with directional ternary pattern (DTP): Application to gender classification, Proceedings of the IEEE International Conference on Information Reuse and Integration, Las Vegas, NV, USA, pp. 159-164.
[4] Ahonen, T., Hadid, A. and Pietikainen, M. (2006). Face description with local binary patterns: Application to face recognition, IEEE Transactions on Pattern Analysis and Machine Intelligence 28(12): 2037-2041. · Zbl 1098.68717
[5] Chen, H., Belhumeur, P. and Jacobs, D. (2000). In search of illumination invariants, Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Hilton Head Island, SC, USA, Vol. 1, pp. 254-261.
[6] Donato, G., Bartlett, M.S., Hagar, J.C., Ekman, P. and Sejnowski, T.J. (1999). Classifying facial actions, IEEE Transactions on Pattern Analysis and Machince Intelligence 21(10): 974-989.
[7] Ekman, P. and Friesen, W. (1978). Facial Action Coding System: A Technique for Measurement of Facial Movement, Consulting Psychologists Press, Palo Alto, CA.
[8] Fa, C.C. and Shin, F.Y. (2006). Recognizing facial action units using independent component analysis and support vector machine, Pattern Recognition 39(9): 1795-1798. · Zbl 1096.68694
[9] Gundimada, S. and Asari, V.K. (2009). Facial recognition using multisensor images based on localized kernel eigen spaces, IEEE Transactions on Image Processing 18(6): 1314-1325. · Zbl 1371.94148
[10] Guo, G.D. and Dyer, C.R. (2003). Simultaneous feature selection and classifier training via linear programming: A case study for face expression recognition, Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Madison, WI, USA, pp. 346-352.
[11] Guo, Z., Zhang, L. and Zhang, D. (2010). Rotation invariant texture classification using LBP variance (LBPV) with global matching, Pattern Recognition 43(3): 706-719. · Zbl 1187.68457
[12] Hsu, C.W. and Lin, C.J. (2002). A comparison on methods for multiclass support vector machines, IEEE Transactions on Neural Networks 13(2): 415-425.
[13] Jabid, T., Kabir, M.H. and Chae, O. (2010). Robust facial expression recognition based on local directional pattern, ETRI Journal 32(5): 784-794.
[14] Jabid, T., Kabir, M.H. and Chae, O. (2012). Local directional pattern (LDP) for face recognition, International Journal of Innovative Computing, Information and Control 8(4): 2423-2437.
[15] Kabir, H., Jabid, T. and Chae, O. (2012). Local directional pattern variance (LDPV): A robust feature descriptor for facial expression recognition, International Arab Journal of Information Technology 9(4): 382-391.
[16] Kanade, T., Cohn, J. and Tian, Y. (2000). Comprehensive database for facial expression analysis, Proceedings of the IEEE International Conference on Automated Face and Gesture Recognition, Grenoble, France, pp. 46-53.
[17] Ling, H., Soatto, S., Ramanathan, N. and Jacobs, D. (2007). A study of face recognition as people age, Proceedings of the IEEE International Conference on Computer Vision, Rio de Janeiro, Brazil, pp. 1-8.
[18] Lyons, M.J., Budynek, J. and Akamatsu, S. (1999). Automatic classification of single facial images, IEEE Transactions on Pattern Analysis and Machine Intelligence 21(12): 1357-1362.
[19] Ojala, T., Pietikainen, M. and Maenpaa, T. (2002). Multiresolution gray-scale and rotation invariant texture classification with local binary patterns, IEEE Transactions on Pattern Analysis and Machine Intelligence 24(7): 971-987. · Zbl 0977.68853
[20] Padgett, C. and Cottrell, G. (1997). Representing face images for emotion classification, in M. Mozer et al. (Eds.), Advances in Neural Information Processing Systems, Vol. 9, MIT Press, Cambridge, MA, pp. 894-900.
[21] Rivera, A.R., Castillo, J.R. and Chae, O. (2013). Local directional number pattern for face analysis: Face and expression recognition, IEEE Transactions on Image Processing 22(5): 1740-1752. · Zbl 1373.94341
[22] Shan, C., Gong, S. and McOwan, P.W. (2009). Facial expression recognition based on local binary patterns: A comprehensive study, Image and Vision Computing 27(6): 803-816.
[23] Sniezynski, B. (2015). A strategy learning model for autonomous agents based on classification, International Journal of Applied Mathematics and Computer Science 25(3): 471-482, DOI: 10.1515/amcs-2015-0035. · Zbl 1322.68203
[24] Tan, X. and Triggs, B. (2007). Enhanced local texture feature sets for face recognition under difficult lighting conditions, Proceedings of the IEEE International Workshop on Analysis and Modeling of Faces and Gestures, Rio de Janeiro, Brazil, pp. 168-182. · Zbl 1371.68311
[25] Tenne, Y. (2017). Machine-learning in optimization of expensive black-box functions, International Journal of Applied Mathematics and Computer Science 27(1): 105-118, DOI: 10.1515/amcs-2017-0008. · Zbl 1367.90070
[26] Uddin, M.Z., Lee, J.J. and Kim, T.S. (2009). An enhanced independent component-based human expression recognition from video, IEEE Transactions on Consumer Electronics 55(4): 2216-2224.
[27] Umbaugh, S.E. (2011). Digital Image Processing and Analysis, CRC Press, Boca Raton, FL. Valstar, M., Patras, I. and Pantic, M. (2005). Facial action unit detection using probabilistic actively learned support vector machines on tracked facial point data, Proceedings of the IEEE CVPR Workshop, San Diego, CA, USA, Vol. 3, pp. 76-84.
[28] Viola, P. and Jones, M. (2004). Robust real-time face detection, International Journal of Computer Vision 57(2): 137-154.
[29] Yang, J., Zhang, D., Frangi, A. and Yang, J. (2004). Two-dimensional PCA: A new approach to appearance-based face representation and recognition, IEEE Transactions on Pattern Analysis and Machine Intelligence 26(1): 131-137.
[30] Yao, B., Hu, P., Zhang, M. and Jin, M. (2014). A support vector machine with the tabu search algorithm for freeway incident detection, International Journal of Applied Mathematics and Computer Science 24(2): 397-404, DOI: 10.2478/amcs-2014-0030. · Zbl 1293.93529
[31] Zhang, Z. (1999). Feature-based facial expression recognition: Sensitivity analysis and experiment with a multi-layer perceptron, International Journal of Pattern Recognition and Artificial Intelligence 13(6): 893-911.
[32] Zhao, G. and Pietikainen, M. (2009). Boosted multi-resolution spatiotemporal descriptors for facial expression recognition, Pattern Recognition Letters 30(12): 1117-1127.
[33] Zhao, S., Gao, Y. and Zhang, B. (2008). Sobel-LBP, Proceedings of the IEEE International Conference on Image Processing, San Diego, CA, USA, pp. 2144-2147.
This reference list is based on information provided by the publisher or from digital mathematics libraries. Its items are heuristically matched to zbMATH identifiers and may contain data conversion errors. It attempts to reflect the references listed in the original paper as accurately as possible without claiming the completeness or perfect precision of the matching.