A first-order primal-dual algorithm for convex problems with applications to imaging. (English) Zbl 1255.68217

Summary: In this paper we study a first-order primal-dual algorithm for non-smooth convex optimization problems with known saddle-point structure. We prove convergence to a saddle-point with rate \(O(1/N)\) in finite dimensions for the complete class of problems. We further show accelerations of the proposed algorithm to yield improved rates on problems with some degree of smoothness. In particular we show that we can achieve \(O(1/N^2)\) convergence on problems, where the primal or the dual objective is uniformly convex, and we can show linear convergence, i.e. \(O(\omega^N)\) for some \(\omega\in (0,1)\), on smooth problems. The wide applicability of the proposed algorithm is demonstrated on several imaging problems such as image denoising, image deconvolution, image inpainting, motion estimation and multi-label image segmentation.


68U10 Computing methodologies for image processing
90C25 Convex programming
94A08 Image processing (compression, reconstruction, etc.) in information and communication theory


Full Text: DOI


[1] Arrow, K.J., Hurwicz, L., Uzawa, H.: Studies in linear and non-linear programming. In: Chenery, H.B., Johnson, S.M., Karlin, S., Marschak, T., Solow, R.M. (eds.) Stanford Mathematical Studies in the Social Sciences, vol. II. Stanford University Press, Stanford (1958) · Zbl 0091.16002
[2] Beck, A., Teboulle, M.: A fast iterative shrinkage-thresholding algorithm for linear inverse problems. SIAM J. Imaging Sci. 2(1), 183–202 (2009) · Zbl 1175.94009
[3] Boyle, J.P., Dykstra, R.L.: A method for finding projections onto the intersection of convex sets in Hilbert spaces. In: Advances in Order Restricted Statistical Inference, Iowa City, Iowa, 1985. Lecture Notes in Statist., vol. 37, pp. 28–47. Springer, Berlin (1986)
[4] Brakke, K.A.: Soap films and covering spaces. J. Geom. Anal. 5(4), 445–514 (1995) · Zbl 0848.49025
[5] Candès, E., Demanet, L., Donoho, D., Ying, L.: Fast discrete curvelet transforms. Multiscale Model. Simul. 5(3), 861–899 (2006) (electronic) · Zbl 1122.65134
[6] Chambolle, A.: An algorithm for total variation minimization and applications. J. Math. Imaging Vis. 20(1–2), 89–97 (2004). Special issue on mathematics and image analysis · Zbl 1366.94048
[7] Chambolle, A.: Total variation minimization and a class of binary MRF models. In: Energy Minimization Methods in Computer Vision and Pattern Recognition, pp. 136–152 (2005)
[8] Chambolle, A., Cremers, D., Pock, T.: A convex approach for computing minimal partitions. Technical Report 649, CMAP, Ecole Polytechnique, France (2008) · Zbl 1256.49040
[9] Chambolle, A., Pock, T.: A first-order primal-dual algorithm for convex problems with applications to imaging. http://hal.archives-ouvertes.fr/hal-00490826 (2010) · Zbl 1255.68217
[10] Eckstein, J., Bertsekas, D.P.: On the Douglas-Rachford splitting method and the proximal point algorithm for maximal monotone operators. Math. Program., Ser. A 55(3), 293–318 (1992) · Zbl 0765.90073
[11] Esser, E.: Applications of Lagrangian-based alternating direction methods and connections to split Bregman. CAM Reports 09-31, UCLA, Center for Applied Math. (2009)
[12] Esser, E., Zhang, X., Chan, T.: A general framework for a class of first order primal-dual algorithms for tv minimization. CAM Reports 09-67, UCLA, Center for Applied Math. (2009)
[13] Fadili, J., Peyré, G.: Total variation projection with first order schemes. http://hal.archives-ouvertes.fr/hal-00380491/ (2009) · Zbl 1372.94077
[14] Fadili, J., Starck, J.-L., Elad, M., Donoho, D.: Mcalab: Reproducible research in signal and image decomposition and inpainting. Comput. Sci. Eng. 12(1), 44–63 (2010)
[15] Goldstein, T., Osher, S.: The split Bregman algorithm for l1 regularized problems. CAM Reports 08-29, UCLA, Center for Applied Math. (2008)
[16] He, B., Yuan, X.: Convergence analysis of primal-dual algorithms for total variation image restoration. Technical Report 2790, Optimization Online, November 2010 (available at www.optimization-online.org )
[17] Korpelevič, G.M.: An extragradient method for finding saddle points and for other problems. Èkon. Mat. Metody 12(4), 747–756 (1976) · Zbl 0342.90044
[18] Lawlor, G., Morgan, F.: Paired calibrations applied to soap films, immiscible fluids, and surfaces or networks minimizing other norms. Pac. J. Math. 166(1), 55–83 (1994) · Zbl 0830.49028
[19] Lions, P.-L., Mercier, B.: Splitting algorithms for the sum of two nonlinear operators. SIAM J. Numer. Anal. 16(6), 964–979 (1979) · Zbl 0426.65050
[20] Michelot, C.: A finite algorithm for finding the projection of a point onto the canonical simplex of R n . J. Optim. Theory Appl. 50(1), 195–200 (1986) · Zbl 0571.90074
[21] Mumford, D., Shah, J.: Optimal approximation by piecewise smooth functions and associated variational problems. Commun. Pure Appl. Math. 42, 577–685 (1989) · Zbl 0691.49036
[22] Nedić, A., Ozdaglar, A.: Subgradient methods for saddle-point problems. J. Optim. Theory Appl. 142(1), 1 (2009) · Zbl 1188.90190
[23] Nemirovski, A.: Prox-method with rate of convergence O(1/t) for variational inequalities with Lipschitz continuous monotone operators and smooth convex-concave saddle point problems. SIAM J. Optim. 15(1), 229–251 (2004) (electronic) · Zbl 1106.90059
[24] Nemirovski, A., Yudin, D.: Problem Complexity and Method Efficiency in Optimization. A Wiley-Interscience Publication. Wiley, New York (1983). Translated from the Russian and with a preface by E.R. Dawson, Wiley-Interscience Series in Discrete Mathematics
[25] Nesterov, Yu.: A method for solving the convex programming problem with convergence rate O(1/k 2). Dokl. Akad. Nauk SSSR 269(3), 543–547 (1983)
[26] Nesterov, Yu.: Introductory Lectures on Convex Optimization. Applied Optimization, vol. 87. Kluwer Academic, Boston (2004). A basic course · Zbl 1086.90045
[27] Nesterov, Yu.: Smooth minimization of non-smooth functions. Math. Program., Ser. A 103(1), 127–152 (2005) · Zbl 1079.90102
[28] Nesterov, Yu.: Gradient methods for minimizing composite objective function. Technical report, CORE DISCUSSION PAPER (2007) · Zbl 1136.65051
[29] Pock, T., Cremers, D., Bischof, H., Chambolle, A.: An algorithm for minimizing the Mumford-Shah functional. In: ICCV Proceedings, LNCS. Springer, Berlin (2009)
[30] Popov, L.D.: A modification of the Arrow-Hurwitz method of search for saddle points. Mat. Zametki 28(5), 777–784, 803 (1980) · Zbl 0456.90068
[31] Rockafellar, R.T.: Monotone operators and the proximal point algorithm. SIAM J. Control Optim. 14(5), 877–898 (1976) · Zbl 0358.90053
[32] Rockafellar, R.T.: Convex Analysis, Princeton Landmarks in Mathematics. Princeton University Press, Princeton (1997). Reprint of the 1970 original, Princeton Paperbacks
[33] Rudin, L., Osher, S.J., Fatemi, E.: Nonlinear total variation based noise removal algorithms. Physica D 60, 259–268 (1992) [also in Experimental Mathematics: Computational Issues in Nonlinear Science (Proc. Los Alamos Conf. 1991)] · Zbl 0780.49028
[34] Shulman, D., Hervé, J.-Y.: Regularization of discontinuous flow fields. In: Proceedings Workshop on Visual Motion, pp. 81–86 (1989)
[35] Zach, C., Gallup, D., Frahm, J.M., Niethammer, M.: Fast global labeling for real-time stereo using multiple plane sweeps. In: Vision, Modeling, and Visualization 2008, pp. 243–252. IOS Press, Amsterdam (2008)
[36] Zach, C., Pock, T., Bischof, H.: A duality based approach for realtime TV-L 1 optical flow. In: 29th DAGM Symposium on Pattern Recognition, pp. 214–223. Heidelberg, Germany (2007)
[37] Zhu, M., Chan, T.: An efficient primal-dual hybrid gradient algorithm for total variation image restoration. CAM Reports 08-34, UCLA, Center for Applied Math. (2008)
This reference list is based on information provided by the publisher or from digital mathematics libraries. Its items are heuristically matched to zbMATH identifiers and may contain data conversion errors. It attempts to reflect the references listed in the original paper as accurately as possible without claiming the completeness or perfect precision of the matching.