×

zbMATH — the first resource for mathematics

Sparse occlusion detection with optical flow. (English) Zbl 1235.68243
Summary: We tackle the problem of detecting occluded regions in a video stream. Under assumptions of Lambertian reflection and static illumination, the task can be posed as a variational optimization problem, and its solution approximated using convex minimization. We describe efficient numerical schemes that reach the global optimum of the relaxed cost functional, for any number of independently moving objects, and any number of occlusion layers. We test the proposed algorithm on benchmark datasets, expanded to enable evaluation of occlusion detection performance, in addition to optical flow.
Reviewer: Reviewer (Berlin)

MSC:
68T45 Machine vision and scene understanding
Software:
na28; NESTA
PDF BibTeX XML Cite
Full Text: DOI
References:
[1] Alvarez, L., Deriche, R., Papadopoulo, T., & Sánchez, J. (2007). Symmetrical dense optical flow estimation with occlusions detection. International Journal of Computer Vision, 75(3), 371–385. · Zbl 1034.68570 · doi:10.1007/s11263-007-0041-4
[2] Ayvaci, A., & Soatto, S. (2011). Efficient model selection for detachable object detection. In Proc. of energy minimization methods in computer vision and pattern recognition, July 2011.
[3] Ayvaci, A., Raptis, M., & Soatto, S. (2010). Occlusion detection and motion estimation with convex optimization. In Advances in neural information processing systems. · Zbl 1235.68243
[4] Baker, S., Scharstein, D., Lewis, J., Roth, S., Black, M., & Szeliski, R. (2007). A database and evaluation methodology for optical flow. In Proc. of the international conference on computer vision (pp. 1–8).
[5] Becker, S., Bobin, J., & Candes, E. (2009). Nesta: a fast and accurate first-order method for sparse recovery. Arxiv preprint arXiv, 904.
[6] Ben-Ari, R., & Sochen, N. (2007). Variational stereo vision with sharp discontinuities and occlusion handling. In Proc. of international conference on computer vision (pp. 1–7).
[7] Black, M., & Anandan, P. (1996). The robust estimation of multiple motions: Parametric and piecewise-smooth flow fields. Computer Vision and Image Understanding, 63(1), 75–104. · Zbl 05473582 · doi:10.1006/cviu.1996.0006
[8] Brox, T., Bruhn, A., Papenberg, N., & Weickert, J. (2004). High accuracy optical flow estimation based on a theory for warping. In Proc. of European conference on computer vision (pp. 25–36). · Zbl 1098.68736
[9] Bruhn, A., Weickert, J., & Schnörr, C. (2005). Lucas/Kanade meets Horn/Schunck: Combining local and global optic flow methods. International Journal of Computer Vision, 61(3), 211–231. · Zbl 02244082 · doi:10.1023/B:VISI.0000045324.43199.43
[10] Candes, E., Wakin, M., & Boyd, S. (2008). Enhancing sparsity by reweighted L1 minimization. The Journal of Fourier Analysis and Applications, 14(5), 877–905. · Zbl 1176.94014 · doi:10.1007/s00041-008-9045-x
[11] Caselles, V., Coll, B., & Morel, J.-M. (1999). Topographic maps and local contrast changes in natural images. International Journal of Computer Vision, 33(1), 5–27. · Zbl 05467140 · doi:10.1023/A:1008144113494
[12] Chan, T., Esedoglu, S., & Nikolova, M. (2006). Algorithms for finding global minimizers of denoising and segmentation models. SIAM Journal on Applied Mathematics, 66(1), 1632–1648 · Zbl 1117.94002 · doi:10.1137/040615286
[13] Dahl, J., Hansen, P., Jensen, S., & Jensen, T. (2009). Algorithms and software for total variation image reconstruction via first-order methods. Numerical Algorithms, 67–92. · Zbl 1181.94009
[14] Gibson, J. J. (1984). The ecological approach to visual perception. LEA.
[15] Goldstein, T., & Osher, S. (2009). The split Bregman method for L1 regularized problems. SIAM Journal on Imaging Sciences, 2(2), 323–343. · Zbl 1177.65088 · doi:10.1137/080725891
[16] He, X., & Yuille, A. (2010). Occlusion boundary detection using pseudo-depth. In Proc. of the European conference on computer vision.
[17] Horn, B., & Schunck, B. (1981). Determining optical flow. Computer Vision, 17, 185–203.
[18] Humayun, A., Mac Aodha, O., & Brostow, G. J. (2011). Learning to find occlusion regions. In Proc. of conference on computer vision and pattern recognition.
[19] Ince, S., & Konrad, J. (2008). Occlusion-aware optical flow estimation. IEEE Transactions on Image Processing, 17(8), 1443–1451. · Zbl 05516620 · doi:10.1109/TIP.2008.925381
[20] Jackson, J. D., Yezzi, A. J., & Soatto, S. (2005). Dynamic shape and appearance modeling via moving and deforming layers. In Proc. of workshop on energy minimization in computer vision and pattern recognition (EMMCVPR) (pp. 427–438).
[21] Jackson, J., Yezzi, A. J., & Soatto, S. (2008). Dynamic shape and appearance modeling via moving and deforming layers. International Journal of Computer Vision.
[22] Kim, Y., Martínez, A., & Kak, A. (2005). Robust motion estimation under varying illumination. Image and Vision Computing, 23(4), 365–375. · doi:10.1016/j.imavis.2004.05.010
[23] Kolmogorov, V., & Zabih, R. (2001). Computing visual correspondence with occlusions via graph cuts. In Proc. of international conference on computer vision (pp. 508–515).
[24] Lim, K., Das, A., & Chong, M. (2002). Estimation of occlusion and dense motion fields in a bidirectional Bayesian framework. IEEE Transactions on Pattern Analysis and Machine Intelligence, 712–718.
[25] Negahdaripour, S. (1998). Revised definition of optical flow: integration of radiometric and geometric cues for dynamic scene analysis. IEEE Transactions on Pattern Analysis and Machine Intelligence, 961–979.
[26] Nesterov, Y. (1983). A method for unconstrained convex minimization problem with the rate of convergence O (1/k 2). Doklady Akademii Nauk SSSR, 269, 543–547.
[27] Nesterov, Y. (2005). Smooth minimization of non-smooth functions. Mathematical Programming, 103(1), 127–152. · Zbl 1079.90102 · doi:10.1007/s10107-004-0552-5
[28] Proesmans, M., Van Gool, L., & Oosterlinck, A. (1994). Determination of optical flow and its discontinuities using a non-linear diffusion. In Proc. of European conference of computer vision.
[29] Robert, C. P. (2001). The Bayesian choice. New York: Springer. · Zbl 0980.62005
[30] Rudin, L., Osher, S., & Fatemi, E. (1992). Nonlinear total variation based noise removal algorithms. Physica. D, 60, 259–268. · Zbl 0780.49028 · doi:10.1016/0167-2789(92)90242-F
[31] Shulman, D., & Herve, J. (1989). Regularization of discontinuous flow fields. In Proc. of workshop on visual motion (pp. 81–86).
[32] Soatto, S. (2011). Actionable information in vision. In Machine learning for computer vision. Berlin: Springer. · Zbl 1251.68271
[33] Soatto, S., & Yezzi, A. (2002). Deformotion: deforming motion, shape average and the joint segmentation and registration of images. In Proc. of the European conference on computer vision (Vol. 3, pp. 32–47). · Zbl 1039.68727
[34] Soatto, S., Yezzi, A. J., & Jin, H. (2003). Tales of shape and radiance in multiview stereo. In Proc. of international conference on computer vision (pp. 974–981). October 2003.
[35] Stein, A., & Hebert, M. (2009). Occlusion boundaries from motion: low-level detection and mid-level reasoning. International Journal of Computer Vision, 82(3), 325–357. · Zbl 05671878 · doi:10.1007/s11263-008-0203-z
[36] Strecha, C., Fransens, R., & Van Gool, L. (2004). A probabilistic approach to large displacement optical flow and occlusion detection. In ECCV workshop SMVP (pp. 71–82). Berlin: Springer.
[37] Sun, J., Li, Y., Kang, S., & Shum, H. (2005). Symmetric stereo matching for occlusion handling. In Proc. of conference on computer vision and pattern recognition (Vol. 2, p. 399).
[38] Sun, D., Roth, S., & Black, M. (2010). Secrets of optical flow estimation and their principles. In Proc. of conference on computer vision and pattern recognition (pp. 2432–2439).
[39] Sundaramoorthi, G., Petersen, P., Varadarajan, V. S., & Soatto, S. (2009). On the set of images modulo viewpoint and contrast changes. In Proc. of conference on computer vision and pattern recognition.
[40] Teng, C., Lai, S., Chen, Y., & Hsu, W. (2005). Accurate optical flow computation under non-uniform brightness variations. Computer Vision and Image Understanding, 97(3), 315–346. · doi:10.1016/j.cviu.2004.08.002
[41] Tibshirani, R. (1996). Regression shrinkage and selection via the lasso. Journal of the Royal Statistical Society. Series B. Methodological, 58(1), 267–288. · Zbl 0850.62538
[42] Verri, A., & Poggio, T. (1989). Motion field and optical flow: Qualitative properties. IEEE Transactions on Pattern Analysis and Machine Intelligence, 11(5), 490–498. · Zbl 05111376 · doi:10.1109/34.24781
[43] Wang, J., & Adelson, E. (1994). Representing moving images with layers. IEEE Transactions on Image Processing, 3(5), 625–638. · doi:10.1109/83.334981
[44] Wang, Y., Yin, W., & Zhang, Y. (2007). A fast algorithm for image deblurring with total variation regularization (CAAM Technical Reports). Rice University.
[45] Wedel, A., Pock, T., Zach, C., Bischof, H., & Cremers, D. (2008). An improved algorithm for TV-L1 optical flow. In Proc. of statistical and geometrical approaches to visual motion analysis: International Dagstuhl seminar.
[46] Wedel, A., Cremers, D., Pock, T., & Bischof, H. (2009). Structure- and motion-adaptive regularization for high accuracy optic flow. In Proc. of international conference on computer vision.
[47] Weiss, P., Blanc-Féraud, L., & Aubert, G. (2009). Efficient schemes for total variation minimization under constraints in image processing. SIAM Journal on Scientific Computing, 31, 2047. · Zbl 1191.94029 · doi:10.1137/070696143
[48] Werlberger, M., Trobin, W., Pock, T., Wedel, A., Cremers, D., & Bischof, H. (2009). Anisotropic Huber-L1 optical flow. In Proc. of British machine vision conference.
[49] Xiao, J., Cheng, H., Sawhney, H., Rao, C., Isnardi, M., et al. (2006). Bilateral filtering-based optical flow estimation with occlusion detection. In Proc. of European conference of computer vision (Vol. 3951, p. 211).
This reference list is based on information provided by the publisher or from digital mathematics libraries. Its items are heuristically matched to zbMATH identifiers and may contain data conversion errors. It attempts to reflect the references listed in the original paper as accurately as possible without claiming the completeness or perfect precision of the matching.