×

Artificial neural networks with quasipolynomial synapses and product synaptic contacts. (English) Zbl 0786.92002

Summary: A neural network model with quasipolynomial synapses and product contacts is investigated. The model further generalizes the sigma-pi and product unit models. What and how many quasipolynomial terms, both for individual variables and for cross-product terms, are learned, is not predetermined, subject to hardware constraints. Three possible cases are considered.
In case 1, the number of learnable parameters needed is determined in learning. It can be considered another method of “growing” a network for a given task, although the graph of the network is fixed. Mechanisms preventing the network from growing too many parameters are designed. In cases 2 and 3, the number of parameters allowed or available is fixed. Cases 2 and 3 may offer both some control on the generalizability of learning and flexibility in functional representation, and may provide a compromise between the complexity of loading and generalizability of learning.
Gradient-descent algorithms for training feedforward networks with polynomial synapses and product contacts are developed. Hardware issues are considered, and experimental results are presented.

MSC:

92B20 Neural networks for/in biological studies, artificial life and related topics
68T05 Learning and adaptive systems in artificial intelligence
PDFBibTeX XMLCite
Full Text: DOI

References:

[1] Abu-Mostafa Y (1989) Complexity in neural systems. In: Mead C (eds) Analog VLSI and neural systems. Addison-Wesley, Reading, Mass
[2] Baum EB, Haussler D (1989) What size net gives valid generalization? Neural Comput 1:151–160
[3] Blum A, Rivest RL (1988) Training a 3-node neural network is NP-complete. In: Proceedings 1988 Workshop on computational learning theory. Morgan Kaufmann, San Mateo, Calif, pp 9–18
[4] Bullock TH, Orkand R, Grinnell A (1977) Introduction to nervous systems. Freeman, San Francisco
[5] Carroll SM, Dickinson BW (1989) Construction of neural nets using the random transform. In. Proceedings of International Joint Conference on Neural Networks, vol 1. pp 607–611
[6] Cybenko G (1989) Approximation by superpositions of a sigmoidal function. Math Control Signals Syst 2:303–314 · Zbl 0679.94019
[7] Durbin R, Rumelhart DE (1989) Product units: a computationally powerful and biologically plausible extension to the backpropagation networks. Neural Comput. 1:133–142
[8] Engelmore R, Morgan T (eds) (1988) Blackboard systems. Addison-Wesley, Reading, Mass
[9] Frean M (1990) The upstart algorithm: a method for constructing and training feedforward networks. Neural Comput. 2:198–209
[10] Giles CL, Maxwell T (1987) Learning, invariance, and generalization in high-order neural networks. Appl. Opt 26:4972–4978
[11] Hornik K, Stinchcombe M, White H (1989) Multilayer feedfoward networks are universal approximators. Neural Networks 2:359–366 · Zbl 1383.92015
[12] Judd JS (1990) Neural network design and the complexity of learning. MIT Press, Cambridge, Mass
[13] Liang P (1990) Problem decomposition and subgoaling in artificial neural networks. In: Proceedings 1990 IEEE Conference on Systems, Man, and Cybernetics, Nov. 4–7, 1990, Los Angeles, Calif, pp 178–181
[14] Marchand M, Golea M, Rujan P (1990) A convergence theorem for sequential learning in two-layer Perceptrons. Europhys Lett 11:487–492
[15] McCulloch WS, Pitts W (1943) A logical calculus of ideas immanent in nervous activity. Bull Math Biophys 5:115–133 · Zbl 0063.03860
[16] Mezard M, Nadal J (1990) Learning in feedforward layered networks: the tiling algorithm. J. Phys A 22:2191
[17] Rumelhart DE, Hinton GE, Williams RJ (1986) Learning internal representations by error propagation. In: Rumelhart DE, McClelland JL (eds) Parallel distributed processing. MIT Press,Cambridge, Mass, pp 318–362
[18] Schürmann J (1976) Multifont word recognition system with application to postal address reading. In: Proceedings of International Joint Conference on Pattern Recognition, pp 658–662
[19] Schürmann J (1977) Polynomklassifikatoren für die Zeichenerkennung. Oldenbourg-Verlag, München. · Zbl 0413.62044
[20] Sethi IK (1990) Entropy nets: from decision trees to neural networks. roc IEEE 78:1605–1613
[21] Shepherd GM (1983) Neurobiology. Oxford University Press, New York
[22] Yin HF, Liang P (1991) A connectionist expert system combining production system and associative memory. Intl J Pattern Recognition Art Intell 5:523–544
This reference list is based on information provided by the publisher or from digital mathematics libraries. Its items are heuristically matched to zbMATH identifiers and may contain data conversion errors. In some cases that data have been complemented/enhanced by data from zbMATH Open. This attempts to reflect the references listed in the original paper as accurately as possible without claiming completeness or a perfect matching.