×

Strong convergence of over-relaxed multi-parameter proximal scaled gradient algorithm and superiorization. (English) Zbl 07339870

Summary: In this paper, we propose an over-relaxed proximal scaled gradient algorithm for solving the non-smooth composite optimization problem in Hilbert space. We prove the strong convergence and the bounded perturbation resilience of this algorithm under some mild conditions. We also list the superiorized version of the algorithm. We apply the algorithms to the \(I_1-I_2\) problem to illustrate the performance and validity. The numerical examples show that the over-relaxed algorithm can achieve fewer iteration steps and less running times in comparison to the under-relaxed algorithm, and that the superiorization algorithm has the minimum iterative steps among the three algorithms.

MSC:

47H09 Contraction-type mappings, nonexpansive mappings, \(A\)-proper mappings, etc.
49J45 Methods involving semicontinuity and convergence; relaxation
49M25 Discrete approximations in optimal control
65J22 Numerical solution to inverse problems in abstract spaces
90C46 Optimality conditions and duality in mathematical programming
PDFBibTeX XMLCite
Full Text: DOI

References:

[1] Atchadé, YF; Fort, G.; Moulines, E., On perturbed proximal gradient algorithms, J Mach Learn Res, 18, 1-33 (2017) · Zbl 1433.90199
[2] Bach, F.; Jenatton, R.; Mairal, J., Optimization with sparsity-inducing penalties, Found Trends Mach Learn, 4, 1, 1-106 (2011) · Zbl 06064248 · doi:10.1561/2200000015
[3] Combettes, PL; Pesquet, JC., A proximal decomposition method for solving convex variational inverse problems, Inverse Probl, 24 (2008) · Zbl 1154.49025 · doi:10.1088/0266-5611/24/6/065014
[4] Daubechies, I.; Defrise, M.; De Mol, C., An iterative thresholding algorithm for linear inverse problems with a sparsity constraint, Commun Pure Appl Math, 57, 1413-1457 (2004) · Zbl 1077.65055 · doi:10.1002/cpa.20042
[5] Dinh, QT; Kyrillidis, A.; Cevher, V., Composite self-concordant minimization, J Mach Learn Res, 16, 71-416 (2015) · Zbl 1337.68231
[6] Parikh, N.; Boyd, S., Proximal algorithms, Found Trends Optim, 1, 127-239 (2014) · doi:10.1561/2400000003
[7] Lions, PL; Mercier, B., Splitting algorithms for the sum of two nonlinear operators, SIAM J Numer Anal, 16, 6, 964-979 (1979) · Zbl 0426.65050 · doi:10.1137/0716071
[8] Schmidt, M.; Roux, NL; Bach, F., Convergence rates of inexact proximal-gradient methods for convex optimization, Adv Neural Inf Process Syst, 24, 1-9 (2011)
[9] Bauschke, HH; Combettes, PL., Convex analysis and monotone operator theory in Hilbert spaces (2011), New York, NY: Springer, New York, NY · Zbl 1218.47001
[10] Xu, HK., Properties and iterative methods for the lasso and its variants, Chinese Ann Math, 35, 3, 501-518 (2014) · Zbl 1295.47064 · doi:10.1007/s11401-014-0829-9
[11] Jin, W.; Censor, Y.; Jiang, M., Bounded perturbation resilience of projected scaled gradient methods, Comput Optim Appl, 63, 2, 365-392 (2016) · Zbl 1339.90262 · doi:10.1007/s10589-015-9777-x
[12] Guo, YN; Cui, W., Perturbation resilience of proximal gradient algorithm for composite objectives, J Sci Appl, 10, 5566-5575 (2017) · Zbl 1412.47123
[13] Guo, YN; Cui, W., Strong convergence and bounded perturbation resilience of modified proximal gradient algorithm, J Inequal Appl, 103 (2018) · Zbl 1497.49018
[14] Wang, Y.; Wang, F.; Xu, HK., Error sensitivity for strongly convergent modifications of the proximal point algorithm, J Optim Theory Appl, 168, 901-916 (2016) · Zbl 1345.47052 · doi:10.1007/s10957-015-0835-4
[15] Cui, HH; Ceng, LC., Convergence of over-relaxed contraction-proximal point algorithm in Hilbert spaces, Optimization, 66, 793-809 (2017) · Zbl 1400.90300 · doi:10.1080/02331934.2017.1296838
[16] Eckstein, J.; Ferris, MC., Operator-splitting methods for monotone affine variational inequalities with a parallel application to optimal control, INFORMS J Comput, 10, 2, 218-235 (1998) · Zbl 1034.90531 · doi:10.1287/ijoc.10.2.218
[17] Cholamjiak, P., A generalized forward-backward splitting method for solving quasi inclusion problems in Banach spaces, Numer Algor, 71, 915-932 (2016) · Zbl 1342.47079 · doi:10.1007/s11075-015-0030-6
[18] Wang, YM; Wang, FH., Strong convergence of the forward-backward splitting method with multiple parameters in Hilbert spaces, Optimization, 67, 4, 493-505 (2018) · Zbl 06865949 · doi:10.1080/02331934.2017.1411485
[19] Censor, Y.; Davidi, R.; Herman, GT., Perturbation resilience and superiorization of iterative algorithms, Inverse Probl, 26, 6 (2010) · Zbl 1193.65019 · doi:10.1088/0266-5611/26/6/065008
[20] Davidi, R.; Schulte, RW; Censor, Y., Fast superiorization using a dual perturbation scheme for proton computed tomography, Trans Amer Nuclear Soc, 106, 73-76 (2012)
[21] Davidi, R.; Herman, GT; Censor, Y., Perturbation-resilient block-iterative projection methods with application to image reconstruction from projections, Int Trans Oper Res, 16, 505-524 (2009) · Zbl 1191.94013 · doi:10.1111/j.1475-3995.2009.00695.x
[22] Nikazad, T.; Davidi, R.; Herman, GT., Accelerated perturbation-resilient block-iterative projection methods with application to image reconstruction, Inverse Probl, 28, 3 (2012) · Zbl 1261.94014 · doi:10.1088/0266-5611/28/3/035005
[23] Censor, Y.; Chen, W.; Combettes, PL, On the effectiveness of projection methods for convex feasibility problems with linear inequality constraints, Comput Optim Appl, 51, 3, 1065-1088 (2012) · Zbl 1244.90155 · doi:10.1007/s10589-011-9401-7
[24] Censor, Y.; Zaslavski, AJ., Strict fejér monotonicity by superiorization of feasibility – seeking projection methods, J Optim Theory Appl, 165, 172-187 (2015) · Zbl 1322.90063 · doi:10.1007/s10957-014-0591-x
[25] Danvidi, R.; Censor, Y.; Schulte, RW, Feasibility-seeking and superiorization algorithm applied to inverse treatment planning in radiation therapy, Contemp Math, 636, 83-92 (2015) · Zbl 1326.65067 · doi:10.1090/conm/636/12729
[26] Xu, HK., Iterative methods for the split feasibility problem in infinite-dimensional Hilbert space, Inverse Probl, 26, 10 (2010) · Zbl 1213.65085 · doi:10.1088/0266-5611/26/10/105018
[27] Moreau, JJ., Proximite et dualite dans ha espace Hilbertien, Bull Soc Math France, 93, 273-299 (1965) · Zbl 0136.12101 · doi:10.24033/bsmf.1625
[28] Xu, HK., Viscosity approximation methods for nonexpansive mappings, J Math Anal Appl, 298, 1, 279-291 (2004) · Zbl 1061.47060 · doi:10.1016/j.jmaa.2004.04.059
[29] Geobel, K.; Kirk, WA., Topics in metric fixed point theory, 104-115 (1990), Cambridge: Cambridge University Press, Cambridge · Zbl 0708.47031
[30] Xu, HK., Iterative algorithms for nonlinear operators, J Lond Math Soc, 66, 1, 240-256 (2002) · Zbl 1013.47032 · doi:10.1112/S0024610702003332
[31] Xu, HK., Error sensitivity for strongly convergent modifications of the proximal point algorithm, J Optim Theory Appl, 168, 901-906 (2015) · Zbl 1345.47052
[32] Censor, Y.; Davidi, R.; Herman, GT., Projected subgradient minimization versus superiorization, J Optim Theory Appl, 160, 3, 730-747 (2014) · Zbl 1298.90104 · doi:10.1007/s10957-013-0408-3
[33] Herman, GT; Garduño, E.; Davidi, R., Superiorization: an optimization heuristic for medical physics, AAPM, 39, 9, 5532-5546 (2012)
This reference list is based on information provided by the publisher or from digital mathematics libraries. Its items are heuristically matched to zbMATH identifiers and may contain data conversion errors. In some cases that data have been complemented/enhanced by data from zbMATH Open. This attempts to reflect the references listed in the original paper as accurately as possible without claiming completeness or a perfect matching.