Real-time hand tracking using a mean shift embedded particle filter. (English) Zbl 1111.68648

Shan, Caifeng; Tan, Tieniu; Wei, Yucheng

Pattern Recognition 40, No. 7, 1958-1970 (2007).

Summary: Particle filtering and Mean Shift (MS) are two successful approaches to visual tracking. Both have their respective strengths and weaknesses. In this paper, we propose to integrate advantages of the two approaches for improved tracking. By incorporating the MS optimization into particle filtering to move particles to local peaks in the likelihood, the proposed Mean Shift Embedded Particle Filter (MSEPF) improves the sampling efficiency considerably. Our work is conducted in the context of developing a hand control interface for a robotic wheelchair. We realize real-time hand tracking in dynamic environments of the wheelchair using MSEPF. Extensive experimental results demonstrate that MSEPF outperforms the MS tracker and the conventional particle filter in hand tracking. Our approach produces reliable tracking while effectively handling rapid motion and distraction with roughly 85% fewer particles. We also present a simple method for dynamic gesture recognition. The hand control interface based on the proposed algorithms works well in dynamic environments of the wheelchair.

MSC:

68T10

Pattern recognition, speech recognition

Keywords:

hand tracking; particle filter; mean shift; hand gesture recognition; human; computer interaction

References:

[2]	Pavlovic, V.; Sharma, R.; Huang, T., Visual interpretation of hand gestures for human-computer interaction: a review, IEEE Trans. Pattern Anal. Mach. Intell., 19, 7, 677-695 (1997)
[3]	McAllister, G.; McKenna, S.; Ricketts, I., Hand tracking for behaviour understanding, Image Vision Comput., 20, 12, 827-840 (2002)
[9]	Tsap, L., Gesture-tracking in real time with dynamic regional range computation, Real Time Imaging, 8, 2, 115-126 (2002) · Zbl 1011.68778
[14]	Inaguma, T.; Saji, H.; Nakatani, H., Hand motion tracking based on a constraint of three-dimensional continuity, J. Electron. Imaging, 14, 1, 013021 (2005)
[16]	Wu, S.; Hong, L., Hand tracking in a natural conversational environment by the interacting multiple model and probabilistic data association (imm-pda) algorithm, Pattern Recognition, 38, 2143-2158 (2005)
[17]	Stenger, B.; Thayananthan, A.; Torr, P. H.S.; Cipolla, R., Model-based hand tracking using a hierarchical Bayesian filter, IEEE Trans. Pattern Anal. Mach. Intell., 28, 9, 1372-1384 (2006)
[18]	Isard, M.; Blake, A., Condensation—conditional density propagation for visual tracking, Int. J. Comput. Vision, 29, 1, 5-28 (1998)
[19]	Ng, C.; Ranganath, S., Real-time gesture recognition system and application, Image Vision Comput., 20, 13-14, 993-1007 (2002)
[21]	Yang, M.; Ahuja, N.; Tabb, M., Extraction of 2d motion trajectories and its application to hand gesture recognition, IEEE Trans. Pattern Anal. Mach. Intell., 24, 8, 1061-1074 (2002)
[24]	Arulampalam, M.; Maskell, S.; Gordon, N.; Clapp, T., A tutorial on particle filters for online nonlinear/non-Gaussian Bayesian tracking, IEEE Trans. Signal Process., 50, 2, 174-189 (2002)
[35]	Nummiaro, K.; Koller-Meier, E.; Van Gool, L., An adaptive color-based particle filter, Image Vision Comput., 21, 1, 99-110 (2003)
[39]	Blake, A.; Isard, M., Active Contours (1998), Springer: Springer Berlin
[42]	Starner, T.; Weaver, J.; Pentland, A., Real-time american sign language recognition using desk and wearable computer based video, IEEE Trans. Pattern Anal. Mach. Intell., 20, 12, 1371-1375 (1998)
[44]	Psarrou, A.; Gong, S.; Walter, M., Recognition of human gestures and behaviour based on motion trajectories, Image Vision Comput., 20, 5-6, 349-358 (2002)
[45]	Bobick, A.; Davis, J., The recognition of human movement using temporal templates, IEEE Trans. Pattern Anal. Mach. Intell., 23, 3, 257-267 (2001)