zbMATH — the first resource for mathematics

On the acceleration of the backpropagation training method. (English) Zbl 0891.68090
We present a new method for dynamically adapting the stepsize in FNN training. This method allows us to give an individual stepsize for each weight. We illustrate its performance in training FNNs for two problems and we compare it with some popular training methods.

68T05 Learning and adaptive systems in artificial intelligence
Full Text: DOI
[1] Rumelhart, D.E.; Hinton, G.E.; Williams, R.J., Learning internal representations by error propagation, (), 318-362
[2] Goldstein, A.A., Cauchy’s method of minimization, Numer. math., 4, 146-150, (1962) · Zbl 0105.10201
[3] Baldi, P.; Hornik, K., Neural networks and principal component analysis: teaming from examples and local minima, Neural networks, 2, 53-58, (1989)
[4] Chan, L.W.; Fallside, F., An adaptive training algorithm for backpropagation networks, Comput. speech language, 2, 205-218, (1987)
[5] Jacobs, R.A., Increased rates of convergence through learning rate adaptation, Neural networks, 1, 295-307, (1988)
[6] Vogl, T.P.; Mangis, J.K.; Rigler, A.K.; Zink, W.T.; Alkon, D.L., Accelerating the convergence of the back-propagation method, Biological cybernetics, 59, 257-263, (1988)
[7] Ortega, J.M.; Rheinboldt, W.C., ()
[8] Voigt, R.G., Rates of convergence for a class of iterative procedures, SIAM J. numer. anal., 8, 127-134, (1971) · Zbl 0232.65044
[9] Varga, R., ()
[10] Armijo, L., Minimization of function having Lipschitz continous first partial derivatives, Pacific J. math., 16, 1-3, (1966) · Zbl 0202.46105
[11] Blum, E.K., Approximation of Boolean functions by sigmoidal networks: part I: XOR and other two variable functions, Neural computation, 1, 532-540, (1989)
[12] Nguyen, D.; Widrow, B., Improving the learning speed of 2-layer neural networks by choosing initial values of the adaptive weights, IEEE first international joint conference on neural networks, 3, 21-26, (1990)
[13] Sperduti, A.; Starita, A., Speed up learning and network optimization with extended back-propagation, Neural networks, 6, 365-383, (1993)
This reference list is based on information provided by the publisher or from digital mathematics libraries. Its items are heuristically matched to zbMATH identifiers and may contain data conversion errors. It attempts to reflect the references listed in the original paper as accurately as possible without claiming the completeness or perfect precision of the matching.