Temporal sequences with neural networks

• US 4,752,906 A
• Filed: 12/16/1986
• Issued: 06/21/1988
• Est. Priority Date: 12/16/1986
• Status: Expired due to Term

• Current Assignee: American Telephone & Telegraph Company (AT&T, Inc.), Bell Telephone Laboratories, Inc. (Nokia Corporation)
• Original Assignee: American Telephone & Telegraph Company (AT&T, Inc.)
• Inventors: Kleinfeld, David
• Primary Examiner(s): Fleming, Michael R.
• Application Number:
  US06/942,431
• Time in Patent Office: 553 Days
• Field of Search: 364/807, 364/824, 364/724, 364/482, 364/200, 364/900, 364/300, 364/811, 364/815
• US Class Current: 708/801
• CPC Class Codes: G06N 3/049 Temporal neural networks, e.g. delay elements, oscillating neurons or pulsed inputs

Abstract

A sequence generator employing a neural network having its output coupled to at least one plurality of delay elements. The delayed outputs are fed back to an input interconnection network, wherein they contribute to the next state transition through an appropriate combination of interconnections.

Claims

1. A sequence generator comprising:

a neural network responsive to input signals applied to a plurality of inputs that are internally connected to a cross-connect network, where said neural network develops a plurality of output signals responsive to said input signals and to interconnection values within said cross-connect network;

a delay/averaging network responsive to said plurality of output signals of said neural network, for developing a delayed plurality of output signals that are related to said plurality of output signals of said neural network; and

an input interconnection network responsive to said plurality of delayed output signals of said delay network for developing said plurality of input signals of said neural network;

WHERE

the interconnections within said cross-connect network develop specified local minima in the energy function describing the response of said neural network to said input signals, and said input interconnection network determines the sequence of neural network output signals developed by the sequence generator.

2. The generator of claim 1, wherein each input signal in said plurality of input signals is related to said delayed output signals through a summing interconnection of preselected summing strengths.

3. The arrangement of claim 1 wherein each i^th input signal of said plurality of input signals, where i is a number, is related to ##EQU7## where ##EQU8## is the j^th delayed output signal of said plurality of delayed output signals and j is a number, D(x) is a preselected delay function characterizing said delay/averaging network as a function of the integration variable x, V_j (t-x) is the j^th output voltage at time (t-x), each Q_ij is a preselected constant, and the summation is over all values of j from 1 to N where N is the number of delayed output signals in said plurality output signals.

4. The arrangement of claim 3 for developing a sequence M¹, M², . . . M.sup.μ

, M.sup.μ

+1, . . . M^m, of said plurality of output signals, with each M.sup.μ

representing said plurality of output signals at a point in time, wherein ##EQU9## for i≠

j and 0 otherwise.

5. The arrangement of claim 3 for developing a cyclic sequence M¹, M², . . . M.sup.μ

, M.sup.μ

+1, . . . M^m, M¹, . . . , of said plurality of output signals, with each M.sup.μ

representing said plurality of output signals at a point in time, wherein ##EQU10## for i≠

j and 0 otherwise, and M^m+1 =M¹.

6. The arrangement of claim 1 further comprising a gating network interposed in the signal loop created by said neural network, said delay network and said input interconnection network.

7. The arrangement of claim 6 wherein said gating means, under influence of an applied control signal, either transmits its input signal to its output or inhibits said transmitting.

8. The arrangement of claim 6 wherein said gating means, under influence of an applied control signal, transmits to its output either its input signal or a preselected different signal.

9. The arrangement of claim 1 further comprising a gating network interposed between said delay network and said input interconnection network.

10. A sequence generator comprising:

a neural network, developing a plurality of output signals in response to a plurality of input signals and characterized by an internal cross-connect network with interconnections that develop specified local minima in the energy function describing the response of said neural network to said input signals;

a delay network responsive to said plurality of output signals of said neural network, developing K delayed pluralities of output signals with each delayed plurality of said K delayed pluralities having a different delay from said plurality of output signals of said neural network; and

an input interconnection network responsive to said K pluralities of delayed output signals of said delay network for supplying a plurality of input signals to said neural network and for providing the connectivity that defines the sequence of signals generated by the sequence generator.

11. The arrangement of claim 10 further comprising a gating network interposed in the signal loop created by said neural network, said delay network and said input interconnection network.

12. A sequence generator comprising:

a neural network, developing a plurality of output signals in response to a plurality of input signals and characterized by an internal cross-connect network with interconnections that develop specified local minima in the energy function describing the response of said neural network to said input signals;

at least one delay/averaging network responsive to said plurality of output signals of said neural network, each developing a set of delayed plurality of output signals; and

at least one input interconnection network each responsive to a set of said plurality of delayed output signals for supplying a plurality of input signals to said neural network and for providing the connectivity that defines the sequence of signals generated by the sequence generator.