Training the random neural network using quasi-Newton methods. (English) Zbl 0960.90094

Summary: Training in the Random Neural Network (RNN) is generally specified as the minimization of an appropriate error function with respect to the parameters of the network (weights corresponding to positive and negative connections). We propose here a technique for error minimization that is based on the use of quasi-Newton optimization techniques. Such techniques offer more sophisticated exploitation of the gradient information compared to simple gradient descent methods, but are computationally more expensive and difficult to implement. In this work we specify the necessary details for the application of quasi-Newton methods to the training of the RNN, and provide comparative experimental results from the use of these methods to some well-known test problems, which confirm the superiority of the approach.


90C53 Methods of quasi-Newton type
92B20 Neural networks for/in biological studies, artificial life and related topics
90B10 Deterministic network models in operations research


Full Text: DOI


[1] Atalay, V.; Gelenbe, E.; Yalabik, N., The random neural network model for texture generation, International journal of pattern recognition and artificial intelligence, 6, 131-141, (1992)
[2] Atalay, V.; Gelenbe, E., Parallel algorithm for colour texture generation using the random neural network model, International journal of pattern recognition and artificial intelligence, 6, 437-446, (1992)
[3] Barnard, E., Optimization for training neural nets, IEEE transactions on neural networks, 3, 2, 232-240, (1992)
[4] Bishop, C., (1995), Neural Networks for Pattern Recognition Oxford University Press
[5] Fletcher, R., Practical methods of optimization, (1987), Wiley New York · Zbl 0905.65002
[6] Gelenbe, E., Random neural networks with negative and positive signals and product form solution, Neural computation, 1, 502-510, (1989)
[7] Gelenbe, E., Stability of the random neural network model, Neural computation, 2, 239-247, (1990)
[8] Gelenbe, E., Product form queueing networks with negative and positive customers, Journal of applied probability, 28, 656-663, (1991) · Zbl 0741.60091
[9] E. Gelenbe, A. Stafylopatis, A. Likas, Associative memory operation of the random network model, in: T. Kohonen, et al. (Eds.), Artificial Neural Networks, vol. 1, North-Holland, Amsterdam, 1991, pp. 307-312
[10] Gelenbe, E., G-networks with triggered customer movement, Journal of applied probability, 30, 742-748, (1993) · Zbl 0781.60088
[11] Gelenbe, E., Learning in the recurrent random neural network, Neural computation, 5, 154-164, (1993)
[12] E. Gelenbe, Hopfield energy of the random neural network, in: Proceedings of the IEEE International Conference on Neural Networks, vol. VII, Orlando, FL, June 1994, pp. 4681-4686
[13] Gelenbe, E.; Feng, Y.; Ranga, K.; Krishnan, R., Neural network methods for volumetric magnetic resonance imaging of the human brain, Proceedings of the IEEE, 84, 1488-1496, (1996)
[14] Gelenbe, E.; Sungur, M.; Cramer, C.; Gelenbe, P., Traffic and video quality with adaptive neural compression, Multimedia systems, 4, 357-369, (1996)
[15] Gelenbe, E.; Ghanwani, A.; Srinivasan, V., Improved neural heuristics for multicast routing, IEEE journal on selected areas in communications, 15, 147-155, (1997)
[16] Gill, P.; Murray, W.; Wright, M., Practical optimization, (1997), Academic Press New York
[17] Henderson, W., Queueing networks with negative customers and negative queue lengths, Journal of applied probability, 30, 931-942, (1993) · Zbl 0787.60115
[18] A. Likas, A. Stafylopatis, An investigation of the analogy between and the random network and the Hopfield network, in: Proceedings of the ISCIS VI, North-Holland, 1991
[19] Likas, A.; Stafylopatis, A., High capacity associative memory based on the random neural network model, International journal of pattern recognition and artificial intelligence, 10, 919-937, (1996)
[20] Likas, A.; Karras, D.A.; Lagaris, I.E., Neural network training and simulation using a multidimensional optimization system, International journal of computer mathematics, 67, 33-46, (1998) · Zbl 0892.68086
[21] Papageorgiou, D.G.; Demetropoulos, I.N.; Lagaris, I.E., Merlin 3.0, a multidimensional optimization environment, Computer physics communications, 109, 227-249, (1998) · Zbl 0936.65072
[22] Stafylopatis, A.; Likas, A., Pictorial information retrieval using the random neural network, IEEE transactions on software engineering, 18, 590-600, (1992)
This reference list is based on information provided by the publisher or from digital mathematics libraries. Its items are heuristically matched to zbMATH identifiers and may contain data conversion errors. It attempts to reflect the references listed in the original paper as accurately as possible without claiming the completeness or perfect precision of the matching.