N-dimensional coulomb neural network which provides for cumulative learning of internal representations

• US 4,897,811 A
• Filed: 01/19/1988
• Issued: 01/30/1990
• Est. Priority Date: 01/19/1988
• Status: Expired due to Term

• Current Assignee: Nestor, Inc.
• Original Assignee: Nestor, Inc.
• Inventors: Scofield, Christopher L.
• Primary Examiner(s): NOT, DEFINED
• Application Number:
  US07/145,630
• Time in Patent Office: 742 Days
• Field of Search: 364/200, 364/300, 364/900, 364/513
• US Class Current: 706/25
• CPC Class Codes: G06F 18/24143 Distances to neighbourhood prototypes, e.g. restricted Coulomb energy networks [RCEN]
• CPC Class Codes: G06N 3/04 Architecture, e.g. interconnection topology
• CPC Class Codes: G06N 3/08 Learning methods

Abstract

A learning algorithm for the N-dimensional Coulomb network is disclosed which is applicable to multi-layer networks. The central concept is to define a potential energy of a collection of memory sites. Then each memory site is an attractor of other memory sites. With the proper definition of attractive and repulsive potentials between various memory sites, it is possible to minimize the energy of the collection of memories. By this method, internal representations may be "built-up" one layer at a time. Following the method of Bachmann et al. a system is considered in which memories of events have already been recorded in a layer of cells. A method is found for the consolidation of the number of memories required to correctly represent the pattern environment. This method is shown to be applicable to a supervised or unsupervised learning paradigm in which pairs of input and output patterns are presented sequentially to the network. The resulting learning procedure develops internal representations in an incremental or cumulative fashion, from the layer closest to the input, to the output layer.

Claims

1. An N-dimensional Coulomb neural network comprising, in combination:

(a) a plurality K of input terminals, each terminal (m) for receiving one of K input signals f_m (t);

(b) a plurality N of neural cells, each neural cell (n) having K inputs and one output, and for producing a first output signal x_n (t) at its output representing a sum of K signal representations applied to its inputs;

(c) a plurality N×

K of input connection elements, each input connection element (mn) coupling one of said input terminals (m) with one of said neural cells (n) and providing a transfer of information from a respective input terminal (m) to a respective neural cell (n) in dependence upon a signal f_m (t) appearing at an input terminal thereof and upon a connection strength ω

_nm of said connection element;

(d) a plurality N of output connection elements, each output connection element (n) being coupled to said output of a respective one (n) of said neural cells and including;

(1) means for storing said first output signal x_n (t) of the neural cell (n) to which it is coupled; and

(2) means for subtracting a next received first output signal x_n (t+1) from a previously stored first output signal x_n (t) to form a difference, and for producing a second output signal (x_n (t)-x_n (t+1))² representing a square of said difference;

(e) an effective cell connected to said output connection elements for receiving said second output signals and having means for computing a function of a state space distance, where L is an integer greater or equal to N-2,

and for producing a third output signal representative thereof,

wherein each of said neural cells adjusts a connection strength (ω

_nm) in accordance with the formula;

space="preserve" listing-type="equation">δ

ω

.sub.nm =(+/-)|x(t)-x(t+1)|-(L+2)Δ

.sub.nm (f(t),f(t+1))

where Δ

_nm (f(t), f(t+1)) is given by;

##EQU21##

2. The neural network of claim 1, wherein for supervised learning the negative sign of δ

ω

_nm is taken for subsequent patterns of the same class and the positive sign of δ

ω

_nm is taken for subsequent patterns of a different class.

3. The neural network of claim 2, wherein for unsupervised learning only the positive sign is taken for δ

ω

_nm for all subsequent patterns.