Geometric expansion for local feature analysis and matching. (English) Zbl 1352.68261

Farhan, Erez; Hagege, Rami

SIAM J. Imaging Sci. 8, No. 4, 2771-2813 (2015).

Summary: We present a novel method for affine transformation estimation of image regions. We illustrate the benefits of using the proposed method in three realms: (1) locating large amounts of point matches with highly accurate localization; (2) producing very accurate affine transformation estimations of for each matched region; (3) effectively rejecting false matches. We show that this is achievable with low computational demand. Point matching is one of the most fundamental tasks in computer vision, being used extensively in applications such as object detection, object tracking, and structure from motion. The major challenge in point matching is to preserve large numbers of accurate matches between corresponding scene locations under different geometric and radiometric conditions, while keeping the number of false matches small. Recent publications have shown that applying the affine transformation model on local regions is a particularly suitable approach for point matching. Yet affine invariant methods are not used extensively for two reasons. First, these methods are computationally demanding; second, the derived affine estimations have limited accuracy. In this work, we propose a novel method of region expansion that enhances region matches detected by any state-of-the-art method. The method is based on accurate estimation of affine transformations, which is used to predict matching locations beyond initially detected matches. The suggested method achieves improved localization of the detected matches. The accuracy of local transformation estimations plays a crucial part in tasks that are based on geometric verification. We prove that expansion beyond local region matches is crucial for obtaining much more accurate local transformation estimations than previously possible. We utilize the improved estimations of affine transformations in order to locally verify tentative matches in an efficient way. We systematically reject false matches, while improving the localization of correct matches that are usually rejected by state-of-the-art methods.

MSC:

68T45	Machine vision and scene understanding
68U05	Computer graphics; computational geometry (digital and algorithmic aspects)
68U10	Computing methodologies for image processing

Keywords:

point matching; feature matching; affine transformation; outlier rejection

Software:

ASIFT; SURF; SIFT; FREAK; Vlfeat

References:

[1]	A. Alahi, R Ortiz, and P. Vandergheynst, {\it FREAK: Fast retina keypoint}, in Proceedings of the 2012 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), IEEE, 2012, pp. 510-517.
[2]	H. Bay, T. Tuytelaars, and L. Van Gool, {\it SURF: Speeded up robust features}, in Proceedings of the 9th European Conference on Computer Vision (ECCV 2006), Springer, 2006, pp. 404-417.
[3]	M. Calonder, V. Lepetit, C. Strecha, and P. Fua, {\it BRIEF: Binary robust independent elementary features}, in Proceedings of the 11th European Conference on Computer Vision (ECCV 2010), Springer, 2010, pp. 778-792.
[4]	F. Cao, J.-L. Lisani, J.-M. Morel, P. Musé, and F. Sur, {\it A Theory of Shape Identification}, Springer, 2008. · Zbl 1156.68002
[5]	J. Cech, J. Matas, and M. Perdoch, {\it Efficient sequential correspondence selection by cosegmentation}, IEEE Trans. Pattern Anal. Mach. Intell., 32 (2010), pp. 1568-1581.
[6]	L. Cheng, J. Gong, X. Yang, C. Fan, and P. Han, {\it Robust affine invariant feature extraction for image matching}, IEEE Geosci. Remote Sensing Lett., 5 (2008), pp. 246-250.
[7]	O. Chum and J. Matas, {\it Matching with PROSAC: Progressive sample consensus}, in Proceedings of the 2005 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR 2005), Vol. 1, IEEE, 2005, pp. 220-226.
[8]	V. Ferrari, T. Tuytelaars, and L. Van Gool, {\it Simultaneous object recognition and segmentation by image exploration}, in Proceedings of the 8th European Conference on Computer Vision (ECCV 2004), Springer, 2004, pp. 40-54. · Zbl 1098.68761
[9]	M. A. Fischler and R. C. Bolles, {\it Random sample consensus: A paradigm for model fitting with applications to image analysis and automated cartography}, Comm. ACM, 24 (1981), pp. 381-395.
[10]	C. Harris and M. Stephens, {\it A combined corner and edge detector}, in Alvey Vision Conference, Vol. 15, Manchester, UK, 1988, p. 50.
[11]	J.-G. Leu, {\it Shape normalization through compacting}, Pattern Recognition Lett., 10 (1989), pp. 243-250. · Zbl 0800.68795
[12]	D. G. Lowe, {\it Distinctive image features from scale-invariant keypoints}, Internat. J. Comput. Vision, 60 (2004), pp. 91-110.
[13]	J. Matas, O. Chum, M. Urban, and T. Pajdla, {\it Robust wide-baseline stereo from maximally stable extremal regions}, Image and Vision Comput., 22 (2004), pp. 761-767.
[14]	K. Mikolajczyk, {\it Affine Covariant Features}, \burlhttp://www.robots.ox.ac.uk/ vgg/research/affine/index.html. Accessed 2014-01-29.
[15]	K. Mikolajczyk and C. Schmid, {\it Scale & affine invariant interest point detectors}, Internat. J. Comput. Vision, 60 (2004), pp. 63-86.
[16]	J.-M. Morel and G. Yu, {\it ASIFT: A new framework for fully affine invariant image comparison}, SIAM J. Imaging Sci., 2 (2009), pp. 438-469. · Zbl 1181.68252
[17]	P. Pritchett and A. Zisserman, {\it Wide baseline stereo matching}, in Proceedings of the 6th International Conference on Computer Vision, IEEE, 1998, pp. 754-760.
[18]	E. Rublee, V. Rabaud, K. Konolige, and G. Bradski, {\it ORB: An efficient alternative to SIFT or SURF}, in Proceedings of the 2011 IEEE International Conference on Computer Vision (ICCV 2011), IEEE, 2011, pp. 2564-2571.
[19]	C. Tomasi and J. Shi, {\it Good features to track}, in Proceedings of the 1994 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR’94), IEEE, 1994, pp. 593-600.
[20]	A. Vedaldi and B. Fulkerson, {\it VLFeat: An open and portable library of computer vision algorithms}, in Proceedings of the International Conference on Multimedia (MM ’10), ACM, New York, 2010, pp. 1469-1472.
[21]	A. Vedaldi and S. Soatto, {\it Local features, all grown up}, in Proceedings of the 2006 IEEE Computer Society Conference on Computer Vision and Pattern Recognition, Vol. 2, IEEE, 2006, pp. 1753-1760.
[22]	G. Yang, C. V. Stewart, M. Sofka, and C.-L. Tsai, {\it Registration of challenging image pairs: Initialization, estimation, and decision}, IEEE Trans. Pattern Anal. Mach. Intell., 29 (2007), pp. 1973-1989.
[23]	G. Yu and J.-M. Morel, {\it ASIFT: An algorithm for fully affine invariant comparison}, Image Processing On Line, 1 (2011).
[24]	Q. Zhang, Y. Wang, and L. Wang, {\it Registration of images with affine geometric distortion based on maximally stable extremal regions and phase congruency}, Image and Vision Comput., 36 (2015), pp. 23-39.