Neural network for learning, recognition and recall of pattern sequences
First Claim
1. A system for processing data consisting of a temporal sequence of patterns, said system comprising:
- a plurality of external and internal input means for receiving and transmitting data signals representing an element of one of said patterns;
a plurality of prediction means coupled to said external input means for producing an output signal value that is a prediction of a future data signal value in said temporal sequence pattern;
each prediction means including a plurality of interconnected dendritic compartments connected to an associated one of said internal input means for storing and transforming said input data signals;
said prediction means including a processing element producing an output activation value at its output based on its input, said processing element connected to at least one of said dendritic compartments, wherein said processing element includes means for producing an activating value which is a function of said transformed input signals from said plurality of dendritic compartments;
recognition means coupled to said external input means, and also to one associated prediction means output for producing an output signal only when a current prediction means output in response to a current input from said temporal pattern matches the next external input data signal from a subsequent input from said temporal pattern; and
gate means with inputs coupled to the output of an associated prediction means and also coupled to the output of said recognition means, said gate means also coupled between said external input means and said internal input means, the gate means operative to selectively direct either the external input means or the prediction means output to said internal input means depending on the state of the recognition means output, whereby when said current prediction means output matches the next external data signal, the prediction means produces its next output without an additional external input.
0 Assignments
0 Petitions
Accused Products
Abstract
A sequence processor for rapidly learning, recognizing and recalling temporal sequences. The processor, called the Katamic system, is a biologically inspired artificial neural network based on a model of the functions of the cerebellum in the brain. The Katamic system utilizes three basic types of neuron-like elements with different functional characteristics called predictrons, recognitrons and bi-stable switches. The Katamic System is clock operated, processing input sequences pattern by pattern to produce an output pattern which is a prediction of the next pattern in the input sequence. The Katamic System learns rapidly, has a large memory capacity, exhibits sequence completion and sequence recognition capability, and is fault and noise tolerant. The system'"'"'s modular construction permits straightforward scaleability.
74 Citations
19 Claims
-
1. A system for processing data consisting of a temporal sequence of patterns, said system comprising:
-
a plurality of external and internal input means for receiving and transmitting data signals representing an element of one of said patterns; a plurality of prediction means coupled to said external input means for producing an output signal value that is a prediction of a future data signal value in said temporal sequence pattern; each prediction means including a plurality of interconnected dendritic compartments connected to an associated one of said internal input means for storing and transforming said input data signals; said prediction means including a processing element producing an output activation value at its output based on its input, said processing element connected to at least one of said dendritic compartments, wherein said processing element includes means for producing an activating value which is a function of said transformed input signals from said plurality of dendritic compartments; recognition means coupled to said external input means, and also to one associated prediction means output for producing an output signal only when a current prediction means output in response to a current input from said temporal pattern matches the next external input data signal from a subsequent input from said temporal pattern; and gate means with inputs coupled to the output of an associated prediction means and also coupled to the output of said recognition means, said gate means also coupled between said external input means and said internal input means, the gate means operative to selectively direct either the external input means or the prediction means output to said internal input means depending on the state of the recognition means output, whereby when said current prediction means output matches the next external data signal, the prediction means produces its next output without an additional external input. - View Dependent Claims (2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12)
-
-
13. A method for processing data consisting of a temporal sequence of patterns, said method comprising:
-
receiving and transmitting data signals representing an element of one of said patterns; producing an output signal in a prediction unit that is a prediction of a future data signal value in said temporal sequence pattern; storing and transforming said input signals in a plurality of dendritic compartments in said prediction unit; producing an output activation value in a processing element in said prediction unit that is a function of said input signal from said plurality of dendritic compartments; providing an external input network and an internal input network which is connected to at least one of said dendritic compartments in an associated prediction means; providing a recognition unit coupled to said external input network, and also to one associated prediction unit; producing an output signal in said recognition unit when a current prediction unit output in response to a current input from said temporal pattern matches the next input data signal in said eternal input network from a subsequent input from said temporal pattern; and selectively directing in a gate unit the external input network or an associated prediction means output to said internal input network depending on the state of the recognition unit output, whereby when said current prediction means output matches the next external data signal, the prediction unit produces a next output without an additional external input. - View Dependent Claims (14, 15, 16, 17, 18)
-
-
19. A neural network for recognition of sequences of patterns, said neural network comprising:
-
a plurality of external and internal input means for receiving and transmitting data signals representing an element of one of said patterns; a plurality of prediction means coupled to said external input means for producing an output signal value that is a prediction of a future data signal value in said temporal sequence pattern; each prediction means including a plurality of interconnected dendritic compartments connected to an associated one of said internal input means for storing and transforming said input data signal values; said prediction means including a processing element producing an output activation value at its output based on its inputs said processing element connected to at least one of said dendritic compartments, wherein said processing element includes means for producing an activation value which is a function of said transformed input signals form said plurality of dendritic compartments; recognition means coupled to said external input means, and also to one associated prediction means output for producing an output signal only when a current prediction means output in response to a current input from said pattern matches the next external input data signal form a subsequent input from said pattern; gate means with inputs coupled to the output of an associated prediction means and also coupled to the output of said recognition means, said gate means also coupled between said external input means and said internal input means, the gate means operative to selectively direct either the external input means or the prediction means output to said internal input means depending on the sate of the recognition means output, whereby when said current prediction means output matches the next external data signal, the prediction means produces its next output without an additional external input; said internal input means providing a weighted connection form one of said gate means to one of said dendritic compartments in said associated prediction means and also to dendritic compartments in a plurality of other prediction means, said internal input means also provided a non-weighted connection to each of the other dendritic compartments in said associated prediction means; said interconnected dendritic compartments in a given prediction means being arranged in a column-like structure with the bottom dendritic compartment in said column being connected to said processing element, and each dendritic compartment is connected to the one below it in the column, each dendritic compartment also including a positive long term memory means, a negative long term memory means and a short term memory means; wherein said function used by said processing element to determine its activation value is proportional to the dot product of the difference between the data values in said positive and negative long term memory and the data values in said short term memories in the dendritic compartment.
-
Specification