zbMATH — the first resource for mathematics

The minimal-norm Gauss-Newton method and some of its regularized variants. (English) Zbl 1448.65030
Summary: Nonlinear least-squares problems appear in many real-world applications. When a nonlinear model is used to reproduce the behavior of a physical system, the unknown parameters of the model can be estimated by fitting experimental observations by a least-squares approach. It is common to solve such problems by Newton’s method or one of its variants such as the Gauss-Newton algorithm. In this paper, we study the computation of the minimal-norm solution to a nonlinear least-squares problem, as well as the case where the solution minimizes a suitable semi-norm. Since many important applications lead to severely ill-conditioned least-squares problems, we also consider some regularization techniques for their solution. Numerical experiments, both artificial and derived from an application in applied geophysics, illustrate the performance of the different approaches.
65F22 Ill-posedness and regularization problems in numerical linear algebra
65H10 Numerical computation of solutions to systems of equations
65F20 Numerical solutions to overdetermined systems, pseudoinverses
Full Text: DOI Link
[1] J. BAGLAMA ANDL. REICHEL,Augmented implicitly restarted Lanczos bidiagonalization methods, SIAM J. Sci. Comput., 27 (2005), pp. 19-42. · Zbl 1087.65039
[2] ,Restarted block Lanczos bidiagonalization methods, Numer. Algorithms, 43 (2006), pp. 251-272. · Zbl 1110.65027
[3] ,An implicitly restarted block Lanczos bidiagonalization method using Leja shifts, BIT, 53 (2013), pp. 285-310. · Zbl 1269.65038
[4] A. K. BJÖRCK,Numerical Methods for Least Squares Problems, SIAM, Philadelphia, 1996. · Zbl 0847.65023
[5] J. BOAGA, M. GHINASSI, A. D’ALPAOS, G. P. DEIDDA, G. RODRIGUEZ,ANDG. CASSIANI,Geophysical investigations unravel the vestiges of ancient meandering channels and their dynamics in tidal landscapes, Sci. Rep., 8 (2018), Art. 1708, 8 pages.
[6] D. CALVETTI, B. LEWIS,ANDL. REICHEL,A hybrid GMRES and TV-norm-based method for image restoration, in Advanced Signal Processing Algorithms, Architectures, and Implementations XII, F. T. Luk, ed., Proceedings of SPIE 4791, SPIE, Bellingham, 2002, pp. 192-200.
[7] J. CHUNG ANDJ. G. NAGY,An efficient iterative approach for large-scale separable nonlinear inverse problems, SIAM J. Sci. Comput., 31 (2009/10), pp. 4654-4674. · Zbl 1205.65160
[8] C. CLASON ANDV. H. NHU,Bouligand-Levenberg-Marquardt iteration for a non-smooth ill-posed inverse problem, Electron. Trans. Numer. Anal., 51 (2019), pp. 274-314. http://etna.ricam.oeaw.ac.at/vol.51.2019/pp274-314.dir/pp274-314.pdf · Zbl 1431.65078
[9] G. P. DEIDDA, P. DÍAZ DEALBA, C. FENU, G. LOVICU,ANDG. RODRIGUEZ,FDEMtools: a MATLAB package for FDEM data inversion, Numer. Algorithms, published online 2019. https://doi.org/10.1007/s11075-019-00843-2 · Zbl 07221101
[10] G. P. DEIDDA, P. DÍAZ DEALBA,ANDG. RODRIGUEZ,Identifying the magnetic permeability in multifrequency EM data inversion, Electron. Trans. Numer. Anal., 47 (2017), pp. 1-17. http://etna.ricam.oeaw.ac.at/vol.47.2017/pp1-17.dir/pp1-17.pdf · Zbl 1372.65118
[11] G. P. DEIDDA, P. DÍAZ DEALBA, G. RODRIGUEZ,ANDG. VIGNOLI,Inversion of multiconfiguration complex EMI data with minimum gradient support regularization: a case study, Math. Geosci., published online 2020.https://doi.org/10.1007/s11004-020-09855-4
[12] G. P. DEIDDA, C. FENU,ANDG. RODRIGUEZ,Regularized solution of a nonlinear problem in electromagnetic sounding, Inverse Problems, 30 (2014), Art. 125014, 27 pages. · Zbl 1308.35294
[13] G. DRAGONETTI, A. COMEGNA, A. AJEEL, G. P. DEIDDA, N. LAMADDALENA, G. RODRIGUEZ, G. VIGNOLI,ANDA. COPPOLA,Calibrating electromagnetic induction conductivities with time-domain reflectometry measurements, Hydrol. Earth. Syst. Sci., 22 (2018), pp. 1509-1523.
[14] H. W. ENGL, M. HANKE,ANDA. NEUBAUER,Regularization of Inverse Problems, Kluwer, Dordrecht, 1996. · Zbl 0859.65054
[15] J. ERIKSSON,Optimization and Regularization of Nonlinear Least Squares Problems, PhD. Thesis, Umeå University, Umeå, 1996.
[16] J. ERIKSSON ANDP. WEDIN,Regularization methods for nonlinear least squares problems. Part I: exactly rank-deficient problems, Tech. Rep. 1996.03, Umeå University, Umeå, 1996.
[17] J. ERIKSSON, P. A. WEDIN, M. E. GULLIKSSON,ANDI. SÖDERKVIST,Regularization methods for uniformly rank-deficient nonlinear least-squares problems, J. Optim. Theory Appl., 127 (2005), pp. 1-26. · Zbl 1152.65437
[18] M. HANKE,A regularizing Levenberg-Marquardt scheme, with applications to inverse groundwater filtration problems, Inverse Problems, 13 (1997), pp. 79-95. · Zbl 0873.65057
[19] M. HANKE, J. G. NAGY,ANDC. VOGEL,Quasi-Newton approach to nonnegative image restorations, Linear Algebra Appl., 316 (2000), pp. 223-236. · Zbl 0960.65071
[20] P. C. HANSEN,Analysis of discrete ill-posed problems by means of theL-curve, SIAM Rev., 34 (1992), pp. 561-580. · Zbl 0770.65026
[21] ,Rank-Deficient and Discrete Ill-Posed Problems, SIAM, Philadelphia, 1998.
[22] ,Regularization Tools version 4.0 for Matlab 7.3, Numer. Algorithms, 46 (2007), pp. 189-194.
[23] P. C. HANSEN, T. K. JENSEN,ANDG. RODRIGUEZ,An adaptive pruning algorithm for the discrete L-curve criterion, J. Comput. Appl. Math., 198 (2007), pp. 483-492. · Zbl 1101.65044
[24] P. C. HANSEN ANDD. P. O’LEARY,The use of theL-curve in the regularization of discrete ill-posed problems, SIAM J. Sci. Comput., 14 (1993), pp. 1487-1503.
[25] P. C. HANSEN, V. PEREYRA,ANDG. SCHERER,Least Squares Data Fitting with Applications, Johns Hopkins University Press, Baltimore, 2013. · Zbl 1270.65008
[26] M. HOCHBRUCK ANDM. HÖNIG,On the convergence of a regularizing Levenberg-Marquardt scheme for nonlinear ill-posed problems, Numer. Math., 115 (2010), pp. 71-79. · Zbl 1189.65114
[27] M. E. HOCHSTENBACH,Harmonic and refined extraction methods for the singular value problem, with applications in least squares problems, BIT, 44 (2004), pp. 721-754. · Zbl 1079.65047
[28] M. E. HOCHSTENBACH, L. REICHEL,ANDG. RODRIGUEZ,Regularization parameter determination for discrete ill-posed problems, J. Comput. Appl. Math., 273 (2015), pp. 132-149. · Zbl 1295.65046
[29] Q. JIN,On a class of frozen regularized Gauss-Newton methods for nonlinear inverse problems, Math. Comp., 79 (2010), pp. 2191-2211. · Zbl 1208.65073
[30] M. E. KILMER, P. C. HANSEN,ANDM. I. ESPAÑOL,A projection-based approach to general-form Tikhonov regularization, SIAM J. Sci. Comput., 29 (2007), pp. 315-330. · Zbl 1140.65030
[31] S. LU, S. V. PEREVERZEV,ANDR. RAMLAU,An analysis of Tikhonov regularization for nonlinear ill-posed problems under a general smoothness assumption, Inverse Problems, 23 (2007), pp. 217-230. · Zbl 1118.65056
[32] P. MAHALE ANDM. T. NAIR,A simplified generalized Gauss-Newton method for nonlinear ill-posed problems, Math. Comp., 78 (2009), pp. 171-184. · Zbl 1198.65101
[33] V. A. MOROZOV,The choice of parameter in solving functional equations by regularization, Dokl. Akad. Nauk SSSR, 175 (1967), pp. 1225-1228.
[34] J. M. ORTEGA ANDW. C. RHEINBOLDT,Iterative Solution of Nonlinear Equations in Several Variables, Academic Press, New York, 1970.
[35] Y. PARK, L. REICHEL, G. RODRIGUEZ,ANDX. YU,Parameter determination for Tikhonov regularization problems in general form, J. Comput. Appl. Math., 343 (2018), pp. 12-25. · Zbl 1391.65100
[36] R. RAMLAU,TIGRA—an iterative algorithm for regularizing nonlinear ill-posed problems, Inverse Problems, 19 (2003), pp. 433-465. · Zbl 1029.65059
[37] R. RAMLAU ANDG. TESCHKE,Tikhonov replacement functionals for iteratively solving nonlinear operator equations, Inverse Problems, 21 (2005), pp. 1571-1592. · Zbl 1078.47030
[38] ,A Tikhonov-based projection iteration for nonlinear ill-posed problems with sparsity constraints, Numer. Math., 104 (2006), pp. 177-203. · Zbl 1101.65056
[39] L. REICHEL ANDG. RODRIGUEZ,Old and new parameter choice rules for discrete ill-posed problems, Numer. Algorithms, 63 (2013), pp. 65-87. · Zbl 1267.65045
[40] A. RUHE,Accelerated Gauss-Newton algorithms for nonlinear least squares problems, BIT, 19 (1979), pp. 356-367. · Zbl 0435.65062
[41] H. D. SIMON ANDH. ZHA,Low-rank matrix approximation using the Lanczos bidiagonalization process with applications, SIAM J. Sci. Comput., 21 (2000), pp. 2257-2274. · Zbl 0962.65038
[42] J.
This reference list is based on information provided by the publisher or from digital mathematics libraries. Its items are heuristically matched to zbMATH identifiers and may contain data conversion errors. It attempts to reflect the references listed in the original paper as accurately as possible without claiming the completeness or perfect precision of the matching.