×

A Bayesian model for recognizing handwritten mathematical expressions. (English) Zbl 1394.68341

Summary: Recognizing handwritten mathematics is a challenging classification problem, requiring simultaneous identification of all the symbols comprising an input as well as the complex two-dimensional relationships between symbols and subexpressions. Because of the ambiguity present in handwritten input, it is often unrealistic to hope for consistently perfect recognition accuracy. We present a system which captures all recognizable interpretations of the input and organizes them in a parse forest from which individual parse trees may be extracted and reported. If the top-ranked interpretation is incorrect, the user may request alternates and select the recognition result they desire. The tree extraction step uses a novel probabilistic tree scoring strategy in which a Bayesian network is constructed based on the structure of the input, and each joint variable assignment corresponds to a different parse tree. Parse trees are then reported in order of decreasing probability. Two accuracy evaluations demonstrate that the resulting recognition system is more accurate than previous versions (which used non-probabilistic methods) and other academic math recognizers.

MSC:

68T10 Pattern recognition, speech recognition
PDFBibTeX XMLCite
Full Text: DOI arXiv

References:

[1] Francisco Álvaro, Joan-Andreu Sánchez, José-Miguel Benedí, Recognition of printed mathematical expressions using two-dimensional stochastic context-free grammars, in: Proceedings of the International Conference on Document Analysis and Recognition, 2011, pp. 1225-1229.
[2] Robert H. Anderson, Syntax-directed recognition of hand-printed two-dimensional mathematics (PhD thesis), Harvard University, 1968.
[3] A.-M. Awal, H. Mouchère, C. Viard-Gaudin, Improving online handwritten mathematical expressions recognition with contextual modeling, in: 2010 International Conference on Frontiers in Handwriting Recognition (ICFHR), 2010, pp. 427-432.
[4] Oleg Golubitsky, Stephen M. Watt, Online computation of similarity between handwritten characters, in: Proceedings of Document Recognition and Retrieval XVI, 2009. · Zbl 1247.68228
[5] G. Labahn, E. Lank, S. MacLean, M. Marzouk, D. Tausky, Mathbrush: a system for doing math on pen-based devices, in: Proceedings of the Eighth IAPR Workshop on Document Analysis Systems, 2008. · Zbl 1114.68627
[6] S. MacLean, D. Tausky, G. Labahn, E. Lank, M. Marzouk, Is the iPad useful for sketch input? A comparison with the tablet PC, in: Proceedings of the Eighth Eurographics Symposium on Sketch-Based Interfaces and Modeling, SBIM ׳11, 2011, pp. 7-14.
[7] Scott MacLean, Automated recognition of handwritten mathematics (PhD thesis), University of Waterloo, 2014.
[8] Scott MacLean, George Labahn, Elastic matching in linear time and constant space, in: Proceedings of Ninth IAPR International Workshop on Document Analysis Systems, 2010, pp. 551-554 (short paper).
[9] MacLean, Scott; Labahn, George, A new approach for recognizing handwritten mathematics using relational grammars and fuzzy sets, Int. J. Doc. Anal. Recognit. (IJDAR), 16, 2, 139-163, (2013)
[10] Manning, Christopher D.; Schuetze, Hinrich, Foundations of statistical natural language processing, (1999), The MIT Press · Zbl 0951.68158
[11] H. Mouchère, C. Viard-Gaudin, U. Garain, D.H. Kim, J.H. Kim, CROHME2011: competition on recognition of online handwritten mathematical expressions, in: Proceedings of the 11th Int׳l. Conference on Document Analysis and Recognition, Beijing, China, 2011.
[12] H. Mouchère, C. Viard-Gaudin, U. Garain, D.H. Kim, J.H. Kim, Competition on recognition of online mathematical expressions (CROHME 2012), in: Proceedings of the 13th International Conference on Frontiers in Handwriting Recognition, 2012.
[13] H. Mouchère, C. Viard-Gaudin, R. Zanibbi, U. Garain, D.H. Kim, J.H. Kim, CROHME - third international competition on recognition of online mathematical expressions, in: Proceedings of the 12th International Conference on Document Analysis and Recognition, 2013.
[14] Yu Shi, HaiYang Li, F.K. Soong, A unified framework for symbol segmentation and recognition of handwritten mathematical expressions, in: Proceedings of the 9th International Conference on Document Analysis and Recognition, 2007, pp. 854-858.
[15] Masakazu Suzuki, Seiichi Uchida, Akihiro Nomura, A ground-truthed mathematical character and symbol image database, in: International Conference on Document Analysis and Recognition, 2005, pp. 675-679.
[16] Arit Thammano, Sukhumal Rugkunchon, A neural network model for online handwritten mathematical symbol recognition, in: Lecture Notes in Computer Science, vol. 4113, 2006.
[17] Unger, Stephen H., A global parser for context-free phrase structure grammars, Commun. ACM, 11, April, 240-247, (1968) · Zbl 0159.45509
[18] R. Yamamoto, S. Sako, T. Nishimoto, S. Sagayama, On-line recognition of handwritten mathematical expression based on stroke-based stochastic context-free grammar, in: The Tenth International Workshop on Frontiers in Handwriting Recognition, 2006.
This reference list is based on information provided by the publisher or from digital mathematics libraries. Its items are heuristically matched to zbMATH identifiers and may contain data conversion errors. In some cases that data have been complemented/enhanced by data from zbMATH Open. This attempts to reflect the references listed in the original paper as accurately as possible without claiming completeness or a perfect matching.