Self-organizing maps: Stationary states, metastability and convergence rate. (English) Zbl 0747.92005

Erwin, E.; Obermayer, K.; Schulten, K.

Biol. Cybern. 67, No. 1, 35-45 (1992).

Summary: We investigate the effect of various types of neighborhood function on the convergence rates and the presence or absence of metastable stationary states of T. Kohonen’s [see “Self-organization and associative memory”, 2nd ed. (1988; Zbl 0659.68100)] self-organizing feature map algorithm in one dimension. We demonstrate that the time necessary to form a topographic representation of the unit interval \([0,1]\) may vary over several orders of magnitude depending on the range and also the shape of the neighborhood function, by which the weight changes of the neurons in the neighborhood of the winning neuron are scaled.
We will prove that for neighborhood functions which are convex on an interval given by the length of the Kohonen chain there exist no metastable states. For all other neighborhood functions, metastable states are present and may trap the algorithm during the learning process. For the widely-used Gaussian function there exists a threshold for the width above which metastable states cannot exist. Due to the presence or absence of metastable states, convergence time is very sensitive to slight changes in the shape of the neighborhood function. Fastest convergence is achieved using neighborhood functions which are “convex” over a large range around the winner neuron and yet have large differences in value at neighboring neurons.

MSC:

92B20	Neural networks for/in biological studies, artificial life and related topics
68T05	Learning and adaptive systems in artificial intelligence

Keywords:

SOFM algorithm; convergence rates; metastable stationary states; self- organizing feature map algorithm; topographic representation of the unit interval; neighborhood functions; Gaussian function; threshold

Citations:

Zbl 0659.68100

References:

[1]	Erwin E, Obermayer K, Schulten K (1991) Convergence properties of self-organizing maps. In: Kohonen T et al. (ed) Artificial neural networks, vol I, North Holland, Amsterdam, pp 409–414 · Zbl 0747.92006
[2]	Erwin E, Obermayer K, Schulten K (1992) Self-organizing maps: ordering, convergence properties and energy functions. Biol Cybern (this issue) · Zbl 0747.92006
[3]	Geszti T (1990) Physical models of neural networks. World Scientific, Singapore · Zbl 0743.92004
[4]	Geszti T, Csabai I, Czakó F, Szakács T, Serneels R, Vattay G (1990) Dynamics of the Kohonen map. In: Statistical mechanics of neural networks: Proceedings, Sitges, Barcelona, Spain, Springer, Berlin Heidelberg New York, pp 341–349
[5]	Kohonen T (1982a) Analysis of a simple self-organizing process. Biol Cybern 44:135–140 · Zbl 0495.93038 · doi:10.1007/BF00317973
[6]	Kohonen T (1982b) Self-organized formation of topologically correct feature maps. Biol Cybern 43:59–69 · Zbl 0466.92002 · doi:10.1007/BF00337288
[7]	Kohonen T (1988) Self-organization and associative memory. Springer, New York Berlin Heidelberg · Zbl 0659.68100
[8]	Lo ZP, Bavarian B (1991) On the rate of convergence in topology preserving neural networks. Biol Cybern 65:55–63 · Zbl 0731.92002 · doi:10.1007/BF00197290
[9]	Ritter H, Schulten K (1986) On the stationary state of Kohonen’s self-organizing sensory mapping. Biol Cybern 54:99–106 · Zbl 0586.92004 · doi:10.1007/BF00320480
[10]	Ritter H, Martinetz T, Schulten K (1989) Topology conserving maps for learning visuomotor-coordination. Neural Networks 2:159–168 · doi:10.1016/0893-6080(89)90001-4