×

From linear to nonlinear iterative methods. (English) Zbl 1022.65060

Authors’ abstract: This paper constitutes an effort towards the generalization of the most common classical iterative methods used for the solution of linear systems (like Gauss-Seidl, successive overrelaxation, Jacobi, and others) to the solution of systems of nonlinear algebraic and/or transcendental qeuations, as well as to unconstrained optimization of nonlinear functions. Convergence and experimental results are presented. The proposed algorithms have also been implemented and tested on classical test problems and on real-line artificial neural network applications and the results to date appear to be very promising.

MSC:

65H10 Numerical computation of solutions to systems of equations
65K05 Numerical mathematical programming methods
90C30 Nonlinear programming

Software:

PVM; minpack; CHABIS; BRENT
PDFBibTeX XMLCite
Full Text: DOI

References:

[1] Axelsson, O., Iterative Solution Methods (1996), Cambridge University Press: Cambridge University Press New York · Zbl 0845.65011
[2] Brent, R. P., Algorithms for Minimization Without Derivatives (1973), Prentice-Hall: Prentice-Hall Englewood Cliffs, NJ · Zbl 0245.65032
[3] Brodatz, P., Textures—A Photographic Album for Artists and Designer (1966), Dover: Dover New York
[4] Gilbert, J. C.; Nocedal, J., Global convergence properties of conjugate gradient methods for optimization, SIAM J. Optimization, 2, 21-42 (1992) · Zbl 0767.90082
[5] Haralick, R.; Shanmugan, K.; Dinstein, I., Textural features for image classification, IEEE Trans. System Man. Cybernetics, 3, 610-621 (1973)
[6] Haykin, S., Neural Networks: A Comprehensive Foundation (1994), Macmillan: Macmillan New York · Zbl 0828.68103
[7] Geist, A.; Beguelin, A.; Dongarra, J.; Jiang, W.; Manchek, R.; Sunderam, V., PVM: Parallel Virtual Machine. A User’s Guide and Tutorial for Networked Parallel Computing (1994), MIT Press: MIT Press Cambridge, MA · Zbl 0849.68032
[8] Kelley, C. T., Iterative Methods for Linear and Nonlinear Equations (1995), SIAM: SIAM Philadelphia, PA · Zbl 0832.65046
[9] Magoulas, G. D.; Vrahatis, M. N.; Androulakis, G. S., A new method in neural network supervised training with imprecision, (Proceedings of the IEEE 3rd International Conference on Electronics, Circuits and Systems (1996), IEEE Press: IEEE Press Piscataway, NJ), 287-290 · Zbl 0884.68106
[10] Magoulas, G. D.; Vrahatis, M. N.; Androulakis, G. S., Improving the convergence of the back-propagation algorithm using learning rate adaptation methods, Neural Comput., 11, 1769-1796 (1999)
[11] Moré, B. J.; Garbow, B. S.; Hillstrom, K. E., Testing unconstrained optimization, ACM Trans. Math. Software, 7, 17-41 (1981) · Zbl 0454.65049
[12] Ortega, J. M.; Rheinboldt, W. C., Iterative Solution of Nonlinear Equations in Several Variables (1970), Academic Press: Academic Press New York · Zbl 0241.65046
[13] Plagianakos, V. P.; Nousis, N. K.; Vrahatis, M. N., Locating and computing in parallel all the simple roots of special functions using PVM, J. Comput. Appl. Math., 133, 545-554 (2001) · Zbl 0988.65024
[14] Polak, E., Optimization: Algorithms and Consistent Approximations (1997), Springer: Springer New York · Zbl 0899.90148
[15] Powell, M. J.D., An efficient method for finding the minimum of a function of several variables without calculating derivatives, Computer J., 7, 155-162 (1964) · Zbl 0132.11702
[16] Ralston, A.; Rabinowitz, P., A First Course in Numerical Analysis (1978), McGraw-Hill: McGraw-Hill New York · Zbl 0408.65001
[17] Reinboldt, W. C., Methods for Solving Systems of Nonlinear Equations (1974), SIAM: SIAM Philadelphia, PA
[18] Rumelhart, D. E.; Hinton, G. E.; Williams, R. J., Learning internal representations by error propagation, (Rumelhart, D. E.; McClelland, J. L., Parallel Distributed Processing: Explorations in the Microstructure of Cognition, Vol. 1 (1986), MIT Press: MIT Press Cambridge, MA), 318-362
[19] Sikorski, K., Bisection is optimal, Numer. Math., 40, 111-117 (1982) · Zbl 0492.65027
[20] Sperduti, A.; Starita, A., Speed up learning and network optimization with extended back-propagation, Neural Networks, 6, 365-383 (1993)
[21] Stewart, G. W., Introduction to Matrix Computations (1973), Academic Press: Academic Press New York · Zbl 0302.65021
[22] Varga, R., Matrix Iterative Analysis (2000), Springer: Springer Berlin · Zbl 0998.65505
[23] Van der Smagt, P. P., Minimization methods for training feedforward neural networks, Neural Networks, 7, 1-11 (1994)
[24] Vrahatis, M. N., Solving systems of nonlinear equations using the nonzero value of the topological degree, ACM Trans. Math. Software, 14, 312-329 (1988) · Zbl 0665.65052
[25] Vrahatis, M. N., CHABIS: A mathematical software package for locating and evaluating roots of systems of nonlinear equations, ACM Trans. Math. Software, 14, 330-336 (1988) · Zbl 0709.65518
[26] Vrahatis, M. N., A short proof and a generalization of Miranda’s existence theorem, Proc. Amer. Math. Soc., 107, 701-703 (1989) · Zbl 0695.55001
[27] Vrahatis, M. N.; Androulakis, G. S.; Lambrinos, J. N.; Magoulas, G. D., A class of gradient unconstrained minimization algorithms with adaptive stepsize, J. Comput. Appl. Math., 114, 367-386 (2000) · Zbl 0958.65072
[28] Vrahatis, M. N.; Androulakis, G. S.; Manousakis, G. E., A new unconstrained optimization method for imprecise function and gradient values, J. Math. Anal. Appl., 197, 586-607 (1996) · Zbl 0887.90166
[29] Vrahatis, M. N.; Grapsa, T. N.; Ragos, O.; Zafiropoulos, F. A., On the localization and computation of zeros of Bessel functions, Z. Angew. Math. Mech., 77, 467-475 (1997) · Zbl 0915.33001
[30] Vrahatis, M. N.; Iordanidis, K. I., A rapid generalized method of bisection for solving systems of nonlinear equations, Numer. Math., 49, 123-138 (1986) · Zbl 0604.65031
[31] Vrahatis, M. N.; Ragos, O.; Zafiropoulos, F. A.; Grapsa, T. N., Locating and computing zeros of Airy functions, Z. Angew. Math. Mech., 76, 419-422 (1996) · Zbl 0878.33001
[32] Vogl, T. P.; Mangis, J. K.; Rigler, A. K.; Zink, W. T.; Alkon, D. L., Accelerating the convergence of the back-propagation method, Biol. Cybern., 59, 257-263 (1988)
[33] Voigt, R. G., Rates of convergence for a class of iterative procedures, SIAM J. Numer. Anal., 8, 127-134 (1971) · Zbl 0232.65044
[34] Young, D., Iterative methods for solving partial difference equations of elliptic type, Trans. Amer. Math. Soc., 76, 92-111 (1954) · Zbl 0055.35704
[35] Wolfe, P., Convergence conditions for ascent methods, SIAM Rev., 11, 226-235 (1969) · Zbl 0177.20603
[36] Wolfe, P., Convergence conditions for ascent methods. II: Some corrections, SIAM Rev., 13, 185-188 (1971) · Zbl 0216.26901
[37] Zangwill, W. I., Minimizing a function without calculating derivatives, Computer J., 10, 293-296 (1967) · Zbl 0189.48004
[38] Zoutendijk, G., Nonlinear programming, computational methods, (Abadie, J., Integer and Nonlinear Programming (1970), North-Holland: North-Holland Amsterdam), 37-86 · Zbl 0336.90057
This reference list is based on information provided by the publisher or from digital mathematics libraries. Its items are heuristically matched to zbMATH identifiers and may contain data conversion errors. In some cases that data have been complemented/enhanced by data from zbMATH Open. This attempts to reflect the references listed in the original paper as accurately as possible without claiming completeness or a perfect matching.