×

Morphologically invariant matching of structures with the complete rank transform. (English) Zbl 1398.68574

Summary: Invariances are one of the key concepts to render computer vision algorithms robust against severe illumination changes. However, there is no free lunch: with any invariance comes an unavoidable loss of information. The goal of our paper is to introduce two novel descriptors which minimise this loss: the complete rank transform and the complete census transform. They are invariant under monotonically increasing intensity rescalings, while containing a maximum possible amount of information. To analyse our descriptors, we embed them as constancy assumptions into a variational framework for optic flow computation. As a suitable regularisation term, we choose total generalised variation that favours piecewise affine solutions. Our experiments focus on the KITTI benchmark where robustness with reference to illumination changes is one of the main issues. The results demonstrate that our descriptors yield state-of-the-art accuracy.

MSC:

68T45 Machine vision and scene understanding

Software:

BRIEF; OEIS; SIFT; KITTI; DeepFlow
PDF BibTeX XML Cite
Full Text: DOI

References:

[1] Alvarez, L; Guichard, F; Lions, PL; Morel, JM, Axioms and fundamental equations in image processing, Archive for Rational Mechanics and Analysis, 123, 199-257, (1993) · Zbl 0788.68153
[2] Baker, S; Scharstein, D; Lewis, JP; Roth, S; Black, MJ; Szeliski, R, A database and evaluation methodology for optical flow, International Journal of Computer Vision, 92, 1-31, (2011)
[3] Bhat, DN; Nayar, SK, Ordinal measures for image correspondence, IEEE Transactions on Pattern Analysis and Machine Intelligence, 20, 415-423, (1998)
[4] Braux-Zin, J., Dupont, R., & Bartoli, A.. (2013). A general dense image matching framework combining direct and feature-based costs. In: Proceedings of the IEEE international conference on computer vision (ICCV), Sydney (pp 185-192).
[5] Bredies, K; Kunisch, K; Pock, T, Total generalized variation, SIAM Journal on Imaging Sciences, 3, 492-526, (2010) · Zbl 1195.49025
[6] Brox, T; Bruhn, A; Papenberg, N; Weickert, J; Pajdla, T (ed.); Matas, J (ed.), High accuracy optical flow estimation based on a theory for warping, No. 3024, 25-36, (2004), Berlin · Zbl 1098.68736
[7] Bruhn, A., & Weickert, J. (2005). Towards ultimate motion estimation: Combining highest accuracy with real-time performance. In: Proceedings of the IEEE international conference on computer vision (ICCV), Beijing (vol. 1, pp. 749-755).
[8] Calonder, M; Lepetit, V; Ozuysal, M; Trzcinski, T; Strecha, C; Fua, P, BRIEF: computing a local binary descriptor very fast, IEEE Transactions on Pattern Analysis and Machine Intelligence, 34, 1281-1298, (2012)
[9] Chan, CH; Goswami, B; Kittler, J; Christmas, W, Local ordinal contrast pattern histograms for spatiotemporal, lip-based speaker authentication, IEEE Transactions on Information Forensics and Security, 7, 602-612, (2012)
[10] Charbonnier, P., Blanc-Féraud, L., Aubert, G., & Barlaud, M. (1994). Two deterministic half-quadratic regularization algorithms for computed imaging. In: Proceedings of the IEEE international conference on image processing, IEEE Computer Society Press, Austin, TX (vol. 2, pp. 168-172).
[11] Chen, J; Kellokumpu, VP; Zhao, G; Pietikinen, M; Burghardt, T (ed.); Damen, D (ed.); Mayol-Cuevas, W (ed.); Mirmehdi, M (ed.), RLBP: robust local binary pattern, (2013), Bristol
[12] Demetz, O; Hafner, D; Weickert, J; Burghardt, T (ed.); Damen, D (ed.); Mayol-Cuevas, W (ed.); Mirmehdi, M (ed.), The complete rank transform: A tool for accurate and morphologically invariant matching of structures, (2013), Bristol
[13] Demetz, O; Stoll, M; Voltz, S; Weickert, J; Bruhn, A; Fleet, D (ed.); Pajdla, T (ed.); Schiele, B (ed.); Tuytelaars, T (ed.), Learning brightness transfer functions for the joint recovery of illumination changes and optical flow, No. 8689, 455-471, (2014), Switzerland
[14] Fröba, B., & Ernst, A. (2004). Face detection with the modified census transform. In: Proceedings of the IEEE international conference on automatic face and gesture recognition (FGR), Seoul, pp. 91-96.
[15] Geiger, A., Lenz, P., & Urtasun, R. (2012). Are we ready for autonomous driving? The KITTI vision benchmark suite. In: Proceedings of the IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR), Providence, RI, pp. 3354-3361.
[16] Gelfand, I. M., & Fomin, S. V. (2000). Calculus of variations. New York: Dover. · Zbl 0964.49001
[17] Gennert, M. A., & Negahdaripour, S. (1987). Relaxing the brightness constancy assumption in computing optical flow. Technical Report 975, Artificial Intelligence Laboratory, Massachusetts Institiute of Technology.
[18] Grewenig, S., Weickert, J., Schroers, C., & Bruhn, A. (2013). Cyclic schemes for PDE-based image analysis. Technical Report 327, Department of Mathematics, Saarland University, Saarbrücken. · Zbl 1398.68600
[19] Hafner, D; Demetz, O; Weickert, J; Kuijper, A (ed.); Pock, T (ed.); Bredies, K (ed.); Bischof, H (ed.), Why is the census transform good for robust optic flow computation?, No. 7893, 210-221, (2013), Berlin
[20] Hermann, S; Klette, R; Park, JI (ed.); Kim, J (ed.), Hierarchical scan-line dynamic programming for optical flow using semi-global matching, No. 7729, 556-567, (2013), Berlin
[21] Hewer, A; Weickert, J; Scheffer, T; Seibert, H; Diebels, S; Burghardt, T (ed.); Damen, D (ed.); Mayol-Cuevas, W (ed.); Mirmehdi, M (ed.), Lagrangian strain tensor computation with higher order variational models, (2013), Bristol
[22] Horn, B; Schunck, B, Determining optical flow, Artificial Intelligence, 17, 185-203, (1981)
[23] Kim, T. H., Lee, H. S., & Lee, K. M. (2013). Optical flow via locally adaptive fusion of complementary data costs. In: Proceedings of the IEEE international conference on computer vision (ICCV), Sydney, pp. 3344-3351.
[24] Liu, C; Yuen, J; Torralba, A, SIFT flow: dense correspondence across scenes and its applications, IEEE Transactions on Pattern Analysis and Machine Intelligence, 33, 978-994, (2011)
[25] Lowe, DL, Distinctive image features from scale-invariant keypoints, International Journal of Computer Vision, 60, 91-110, (2004)
[26] Mei, X., Sun, X., Zhou, M., Jiao, S., Wang, H., & Zhang, X. (2011). On building an accurate stereo matching system on graphics hardware. In: Proceedings of the IEEE international conference on computer vision workshops (ICCV workshops), Barcelona, pp. 467-474.
[27] Mileva, Y; Bruhn, A; Weickert, J; Hamprecht, FA (ed.); Schnör, C (ed.); Jähne, B (ed.), Illumination-robust variational optical flow with photometric invariants, (2007), Berlin
[28] Mittal, A., & Ramesh, V. (2006). An intensity-augmented ordinal measure for visual correspondence. In: Proceedings of the IEEE computer society conference on computer vision and pattern recognition (CVPR), New York, NY (vol 1, pp. 849-856).
[29] Mohamed, M; Rashwan, H; Mertsching, B; Garcia, M; Puig, D, Illumination-robust optical flow using a local directional pattern, IEEE Transactions on Circuits and Systems for Video Technology, 24, 1499-1508, (2014)
[30] Müller, T; Rabe, C; Rannacher, J; Franke, U; Mester, R; Mester, R (ed.); Felsberg, M (ed.), Illumination robust dense optical flow using census signatures, (2011), Berlin
[31] Otte, M; Nagel, HH; Eklundh, JO (ed.), Optical flow estimation: advances and comparisons, No. 800, 49-60, (1994), Berlin
[32] Papenberg, N; Bruhn, A; Brox, T; Didas, S; Weickert, J, Highly accurate optic flow computation with theoretically justified warping, International Journal of Computer Vision, 67, 141-158, (2006)
[33] Pietikäinen, M., Hadid, A., Zhao, G., & Ahonen, T. (2011). Computer vision using local binary patterns. London: Springer.
[34] Press, W. H., Teukolsky, S. A., Vetterling, W. T., & Flannery, B. P. (2007). Numerical recipes: The art of scientific computing (3rd ed.). New York, NY: Cambridge University Press.
[35] Puxbaum, P; Ambrosch, K; Bebis, G (ed.); Boyle, RD (ed.); Parvin, B (ed.); Koracin, D (ed.); Chung, R (ed.); Hammoud, RI (ed.); Hussain, M (ed.); Tan, KH (ed.); Crawfis, R (ed.); Thalmann, D (ed.); Kao, D (ed.); Avila, L (ed.), Gradient-based modified census transform for optical flow, No. 6453, (2010), Berlin
[36] Ranftl, R., Gehrig, S., Pock, T., & Bischof, H. (2012). Pushing the limits of stereo using variational stereo estimation. In: Proc. IEEE Intelligent Vehicles Symposium, Alcala de Henares, Spain, pp. 401-407.
[37] Ranftl, R; Bredies, K; Pock, T; Fleet, D (ed.); Pajdla, T (ed.); Schiele, B (ed.); Tuytelaars, T (ed.), Non-local total generalized variation for optical flow estimation, No. 8689, 439-454, (2014), Berlin
[38] Rashwan, H; Mohamed, M; Garcia, M; Mertsching, B; Puig, D; Weickert, J (ed.); Hein, M (ed.); Schiele, B (ed.), Illumination robust optical flow model based on histogram of oriented gradients, No. 8142, 354-363, (2013), Berlin
[39] Sloane, N. J. A., & Plouffe, S. (1995). The encyclopedia of integer sequences. San Diego, CA: Academic Press. · Zbl 0845.11001
[40] Soatto, S. (2009). Actionable information in vision. In: Proceedings of the IEEE international conference on computer vision, IEEE Computer Society Press (pp. 2138-2145).
[41] Stein, F; Rasmussen, CE (ed.); Bülthoff, HH (ed.); Schölkopf, B (ed.); Giese, MA (ed.), Efficient computation of optical flow using the census transform, No. 3175, 79-86, (2004), Berlin
[42] Steinbrücker, F., Pock, T., & Cremers, D. (2009). Advanced data terms for variational optic flow estimation. In: M. A. Magnor, B. Rosenhahn, H. Theisel (Eds.) Proceedings of the vision, modeling, and visualization workshop (VMV), DNB (pp. 155-164).
[43] Sun, D; Roth, S; Black, M, A quantitative analysis of current practices in optical flow estimation and the principles behind them, International Journal of Computer Vision, 106, 115-137, (2014)
[44] Tang, F., Lim, S. H., Chang, N. L., & Tao, H. (2009). A novel feature descriptor invariant to complex brightness changes. In: Proceedings of the IEEE Computer Society conference on computer vision and pattern recognition (CVPR), Miami, FL (pp. 2631-2638).
[45] Tukey, J. W. (1971). Exploratory data analysis. Menlo Park, CA: Addison-Wesley.
[46] Uras, S; Girosi, F; Verri, A; Torre, V, A computational approach to motion perception, Biological Cybernetics, 60, 79-87, (1988)
[47] van de Weijer, J., & Gevers, T. (2004). Robust optical flow from photometric invariants. In: Proceedings of the IEEE international conference on image processing (ICIP), Singapore (pp. 1835-1838).
[48] Vogel, C; Roth, S; Schindler, K; Weickert, J (ed.); Hein, M (ed.); Schiele, B (ed.), An evaluation of data costs for optical flow, No. 8142, 343-353, (2013), Berlin
[49] Vogel, C. R., & Oman, M. E. (1996). Iterative methods for total variation denoising. SIAM Journal on Scientific Computing, 17(1), 227-238. · Zbl 0847.65083
[50] Wang, Z., Fan, B., & Wu, F. (2011). Local intensity order pattern for feature description. In: Proceedings of the IEEE International conference on computer vision (ICCV), Barcelona (pp. 603-610).
[51] Wedel, A; Pock, T; Zach, C; Cremers, D; Bischof, H; Cremers, D (ed.); Rosenhahn, B (ed.); Yuille, AL (ed.); Schmidt, FR (ed.), An improved algorithm for TV-L1 optical flow, No. 5604, (2008), Berlin
[52] Wei, D., Liu, C., & Freeman, W. T. (2014). A data-driven regularization model for stereo and flow. In: Proceedings of the IEEE international conference on 3D vision, Tokyo.
[53] Weinzaepfel, P., Revaud, J., Harchaoui, Z., & Schmid, C. (2013). Deepflow: Large displacement optical flow with deep matching. In: Proceedings of the IEEE international conference on computer vision (ICCV), Sydney (pp. 1385-1392).
[54] Werlberger, M., Pock, T., & Bischof, H. (2010). Motion estimation with non-local total variation regularization. In: Proceedings of the IEEE conference on computer vision and pattern recognition (CVPR), San Francisco, CA (pp. 2464-2471).
[55] Xu, L., Jia, J., & Matsushita, Y. (2010). Motion detail preserving optical flow estimation. In: Proceedings IEEE conference on computer vision and pattern recognition (CVPR), IEEE Computer Society Press (pp. 1293-1300).
[56] Zabih, R; Woodfill, J; Eklundh, JO (ed.), Non-parametric local transforms for computing visual correspondence, No. 801, 151-158, (1994), Berlin
[57] Zimmer, H; Bruhn, A; Weickert, J; Valgaerts, L; Salgado, A; Rosenhahn, B; Seidel, HP; Cremers, D (ed.); Boykov, Y (ed.); Blake, A (ed.); Schmidt, FR (ed.), Complementary optic flow, No. 5681, 207-220, (2009), Berlin
[58] Zimmer, H; Bruhn, A; Weickert, J, Optic flow in harmony, International Journal of Computer Vision, 3, 368-388, (2011) · Zbl 1235.94030
This reference list is based on information provided by the publisher or from digital mathematics libraries. Its items are heuristically matched to zbMATH identifiers and may contain data conversion errors. It attempts to reflect the references listed in the original paper as accurately as possible without claiming the completeness or perfect precision of the matching.