Iconic and multi-stroke gesture recognition. (English) Zbl 1182.68210

Summary: Many handwritten gestures, characters, and symbols comprise multiple pendown strokes separated by penup strokes. In this paper, a large number of features known from the literature are explored for the recognition of such multi-stroke gestures. Features are computed from a global gesture shape. From its constituent strokes, the mean and standard deviation of each feature are computed. We show that using these new stroke-based features, significant improvements in classification accuracy can be obtained between 10% and 50% compared to global feature representations. These results are consistent over four different databases, containing iconic pen gestures, handwritten symbols, and upper-case characters. Compared to two other multi-stroke recognition techniques, improvements between 25% and 39% are achieved, averaged over all four databases.


68T10 Pattern recognition, speech recognition


Full Text: DOI


[1] F. Tian, T. Cheng, H. Wang, G. Dai, Advances in Computer Graphics, in: Lecture Notes in Computer Science, vol. 4035/2006, Springer, Berlin/Heidelberg, 2006, Ch. Research on User-Centered Design and Recognition of Pen Gestures, pp. 312-323.
[2] Fitrianie, S.; Rothkrantz, L., A visual communication language for crisis management, International journal of intelligent control and systems (special issue of distributed intelligent systems), 12, 2, 208-216, (2007)
[3] D. Willems, L. Vuurpijl, Designing interactive maps for crisis management, in: Proceedings of the Fourth International Conference on Information Systems for Crisis Response and Management (ISCRAM2007), 2007, pp. 159-166.
[4] D. Willems, L. Vuurpijl, Pen gestures in online map and photograph annotation tasks, in: Proceedings of the Tenth International Workshop on Frontiers in Handwriting Recognition (IWFHR06), La Baule, France, 2006, pp. 297-402.
[5] R. Niels, D. Willems, L. Vuurpijl, The NicIcon collection of handwritten icons, in: ICFHR8, the Eleventh International Conference on Frontiers of Handwriting Recognition, Montreal, Canada, 2008, pp. 296-301.
[6] Rubine, D., Specifying gestures by example, Computer graphics, 25, 4, 329-337, (1991)
[7] Cohen, P.R.; Johnston, M.; McGee, D.; Oviatt, S.; Pittman, J.; Smith, I.; Chen, L.; Clow, J., Quickset: multimodal interaction for distributed applications, (), 31-40
[8] J.A. Landay, R.B. Dannenberg, Interactive sketching for the early stages of user interface design, in: CHI95 Computer Human Interaction, 1995, pp. 43-50.
[9] B. Signer, U. Kurmann, M. Norrie, Igesture: a general gesture recognition framework, in: Proceedings of the Ninth International Conference on Document Analysis and Recognition (ICDAR2007), Curitiba, Brazil, 2007, pp. 954-958.
[10] Lipscomb, J., A trainable gesture recognizer, Pattern recognition, 24, 9, 895-907, (1991)
[11] Fonseca, M.; Jorge, J., Experimental evaluation of an on-line scribble recognizer, Pattern recognition letters, 22, 12, 1311-1319, (2001) · Zbl 0990.68619
[12] M. Egger, Find new meaning in your ink with tablet PC APIs in Windows Vista, Technical Report, Microsoft Corporation, May 2006.
[13] J. Hong, J. Landay, Satin: a toolkit for informal ink-based applications, in: UIST00, Thirteenth Annual ACM Symposium on User Interface Software and Technology, San Diego, USA, 2000, pp. 63-72.
[14] C. Blickenstorfer, Graffiti: Wow! Pen Computing Magazine, 1995, pp. 30-31.
[15] D. Goldberg, C. Richardson, Touch-typing with a stylus, in: CHI ’93: Proceedings of the INTERACT ’93 and CHI ’93 Conference on Human Factors in Computing Systems, 1993, pp. 80-87.
[16] Zhang, L.; Sun, Z., An experimental comparison of machine learning for adaptive sketch recognition, Applied mathematics and computation, 185, 2, 1138-1148, (2007) · Zbl 1113.68460
[17] I. Guyon, L. Schomaker, R. Plamondon, M. Liberman, S. Janet, UNIPEN project of on-line data exchange and recognizer benchmarks, in: Proceedings ICPR’94, 1994, pp. 29-33.
[18] C. Viard-Gaudin, P.M. Lallican, P. Binter, S. Knerr, The IRESTE on/off (IRONOFF) dual handwriting database, in: Proceedings of the International Conference on Document Analysis and Recognition, ICDAR‘99, Bangalore, India, 1999, pp. 455-458.
[19] W. Jiang, Z.-X. Sun, HMM-based on-line multi-stroke sketch recognition, in: Machine Learning and Cybernetics, 2005, Lecture Notes in Computer Science, vol. 7, Springer, Berlin, 2005, pp. 4564-4570.
[20] Sezgin, T.; Davis, R., Sketch recognition in interspersed drawings using time-based graphical models, Computers & graphics, 32, 500-510, (2008)
[21] Guyon, I.; Albrecht, P.; Le Cun, Y.; Denker, J.; Hubbard, W., Design of a neural network character recognizer for a touch terminal, Pattern recognition, 24, 2, 105-119, (1991)
[22] Vuori, V.; Oja, E.; Kangas, J., Experiments with adaptation strategies for a prototype-based recognition system for isolated handwritten characters, International journal on document analysis and recognition, 3, 150-159, (2001)
[23] Webb, A., Statistical pattern recognition, (2002), Wiley, (Chapter 9. Feature selection and extraction, pp. 305-359)
[24] Vuurpijl, L.; Schomaker, L., Finding structure in diversity: a hierarchical clustering method for the categorization of allographs in handwriting, (), 387-393
[25] Niels, R.; Vuurpijl, L.; Schomaker, L., Automatic allograph matching in forensic writer identification, International journal of pattern recognition and artificial intelligence, 21, 1, 61-81, (2007)
[26] Homeland Security Working Group, Symbology Reference, Version 2.20, Released September 14, 2005 \(\langle\)http://www.fgdc.gov/HSWG⟩.
[27] U. Marti, H. Bunke, A full English sentence database for off-line handwriting recognition, in: Proceedings of the Fifth International Conference on Document Analysis and Recognition (ICDAR’99), Bangalore, India, 1999, pp. 705-708.
[28] Jaeger, S.; Nakagawa, M., Two on-line Japanese character databases in unipen format, (), 566-570
[29] Nakagawa, M.; Matsumoto, K., Collection of on-line handwritten Japanese character pattern databases and their analysis, International journal on document analysis and recognition, 7, 1, 69-81, (2004)
[30] A.S. Bhaskarabhatla, S. Madhvanath, Experiences in collection of handwriting data for online handwriting recognition in Indic scripts, in: Proceedings of the Fourth International Conference on Linguistic Resources and Evaluation (LREC), 2004, CDROM.
[31] U. Bhattacharya, Handwritten character databases of Indic scripts, 2004 \(\langle\)http://www.isical.ac.in/∼ujjwal/download/database.html⟩.
[32] M. Pechwitz, S. Maddouri, V. Märgner, N. Ellouze, H. Amiri, Ifn/enit-database of handwritten Arabic words, in: Seventh Colloque International Francophone sur l’Ecrit et le Document (CIFED02), Hammamet, Tunis, 2002, pp. 1-8.
[33] D. Willems, R. Niels, Definitions for features used in online pen gesture recognition, Technical Report, NICI, Radboud University Nijmegen, 2008 \(\langle\)http://unipen.nici.ru.nl/NicIcon/⟩.
[34] J. Iivarinen, M. Peura, J. Särelä, A. Visa, Comparison of combined shape descriptors for irregular objects, in: A. Clark (Ed.), Eighth British Machine Vision Conference, BMVC’97, Essex, UK, 1997, pp. 430-439.
[35] Peura, M.; Iivarinen, J., Efficiency of simple shape descriptors, (), 443-451
[36] LaViola, J.; Zeleznik, J.J.R., A practical approach for writer-dependent symbol recognition using a writer-independent symbol recognizer, IEEE transactions on pattern analysis and machine intelligence, 29, 11, 1917-1926, (2007)
[37] Fukunaga, K.; Hostetler, L., The estimation of the gradient of a density function, with applications in pattern recognition, IEEE transactions on information theory, 21, 32-40, (1975) · Zbl 0297.62025
[38] Jain, A.; Duin, R.; Mao, J., Statistical pattern recognition: a review, IEEE transactions on pattern analysis and machine intelligence, 22, 1, 4-37, (2000)
[39] Piramuthu, S., Evaluating feature selection methods for learning in data mining applications, European journal of operational research, 156, 483-494, (2004) · Zbl 1056.90091
[40] Vapnik, V., The nature of statistical learning theory, (1995), Springer Berlin · Zbl 0833.62008
[41] C.-C. Chang, C.-J. Lin, LIBSVM: a library for support vector machines, 2001 \(\langle\)www.csie.ntu.edu.tw/∼cjlin/libsvm⟩.
This reference list is based on information provided by the publisher or from digital mathematics libraries. Its items are heuristically matched to zbMATH identifiers and may contain data conversion errors. It attempts to reflect the references listed in the original paper as accurately as possible without claiming the completeness or perfect precision of the matching.