Methods, architecture, and apparatus for implementing machine intelligence and hierarchical memory systems
First Claim
1. A memory system, said memory system comprising:
- a hierarchy comprising a plurality of cortical processing units, each of said cortical processing units comprising a first output, said first output for outputting a first set of pattern information, a first input for receiving a second set of pattern information, said first input coupled to a sensory unit or to a first output from another cortical processing unit, a memory, said memory storing information about patterns that may appear in said second set of pattern information on said first input, and a processing method, said processing method comparing said second set of pattern information on said first input with said information about patterns stored in said memory.
1 Assignment
0 Petitions
Accused Products
Abstract
Sophisticated memory systems and intelligent machines may be constructed by creating an active memory system with a hierarchical architecture. Specifically, a system may comprise a plurality of individual cortical processing units arranged into a hierarchical structure. Each individual cortical processing unit receives a sequence of patterns as input. Each cortical processing unit processes the received input sequence of patterns using a memory containing previously encountered sequences with structure and outputs another pattern. As several input sequences are processed by a cortical processing unit, it will therefore generate a sequence of patterns on its output. The sequence of patterns on its output may be passed as an input to one or more cortical processing units in next higher layer of the hierarchy. A lowest layer of cortical processing units may receive sensory input from the outside world. The sensory input also comprises a sequence of patterns.
104 Citations
43 Claims
-
1. A memory system, said memory system comprising:
a hierarchy comprising a plurality of cortical processing units, each of said cortical processing units comprising a first output, said first output for outputting a first set of pattern information, a first input for receiving a second set of pattern information, said first input coupled to a sensory unit or to a first output from another cortical processing unit, a memory, said memory storing information about patterns that may appear in said second set of pattern information on said first input, and a processing method, said processing method comparing said second set of pattern information on said first input with said information about patterns stored in said memory. - View Dependent Claims (2, 3, 4, 5, 6, 7, 8, 9, 10)
-
11. (canceled)
-
12. A system, comprising:
-
a first level of modules, the modules each arranged to store a sequence of patterns; and
a second level of at least one module, the at least one module arranged to store a sequence of patterns, wherein the first level of modules and the second level of at least one module form at least part of a hierarchical network structure, wherein, at least partially dependent on the sequences of patterns stored in the first level of modules and in the second level of at least one module, an object causing at least one pattern is determinable by information passing through the hierarchical network structure. - View Dependent Claims (13, 14, 15, 16, 17, 18, 19)
-
-
20. A method, comprising:
-
accessing a first module having a stored sequence of patterns;
accessing a second module having a stored sequence of patterns, wherein the first module and the second module form at least part of a hierarchical network; and
determining an object causing a pattern, the determining comprising at least one of;
passing first information from the first module to the second module dependent on the sequence stored in the first module, wherein the first information is indicative of at least one possible cause of the pattern, and passing second information from the second module to the first module dependent on the sequence stored in the second module, wherein the second information is indicative of an expectation of a next pattern caused by the object. - View Dependent Claims (21, 22, 23, 24, 25, 26)
-
-
27. A method, comprising:
-
inputting a first sensed pattern to a first processing unit;
inputting a second sensed pattern to a second processing unit;
in response to inputting the first sensed pattern, determining a first distribution having at least one possible cause of the first sensed pattern dependent on at least one sequence of patterns stored in the first processing unit;
in response to inputting the second sensed pattern, determining a second distribution having at least one possible cause of the second sensed pattern dependent on at least one sequence of patterns stored in the second processing unit;
passing the first distribution and the second distribution to a third processing unit; and
determining a cause of the first sensed pattern and the second sensed pattern at least partially dependent on at least one operation performed on the first distribution and the second distribution by the third processing unit. - View Dependent Claims (28, 29, 30)
-
-
31. A system, comprising:
-
a first processing unit arranged to input a sensed pattern and further arranged to generate an inference as to at least one possible cause of the sensed pattern dependent on a plurality of sequences stored in the first processing unit, wherein the inference comprises a probability distribution; and
a second processing unit arranged to, dependent on the inference, generate a prediction as to a next inputted sensed pattern dependent on a plurality of sequences stored in the second processing unit, wherein a subsequent inference by the first processing unit is dependent on the prediction. - View Dependent Claims (32, 33)
-
-
34. An apparatus, comprising:
-
a representation of a lower level of processing units, wherein the lower level processing units are arranged to output information regarding possible causes of sequences of sensed input patterns dependent on stored sequences previously learned by the lower level processing units; and
a representation of a higher level of at least one processing unit, wherein the at least one higher level processing unit is arranged to input the output information from the lower level processing units and output information regarding at least one possible cause of the sequences dependent on stored sequences previously learned by the higher level processing unit, wherein the lower level of processing units and the higher level of at least one processing unit collectively represent at least part of a hierarchical network. - View Dependent Claims (35, 36, 37)
-
-
38. A method, comprising:
-
storing sequences associated with an invariant structure of an object in a hierarchy of modules forming a hierarchical network;
receiving a sensed pattern;
first comparing the sensed pattern with a first part of the stored sequences;
determining at least one possible cause of the sensed pattern at least partially dependent on the comparing;
second comparing information associated with the at least one possible cause with a second part of the stored sequences, the second part representing a larger part of the invariant structure than the first part; and
determining the object as a cause of the sensed pattern at least partially dependent on the second comparing. - View Dependent Claims (39)
-
-
40. A system, comprising:
-
a hierarchical network of processing units, the hierarchical network having;
a first level of processing units each arranged to store at least one sequence of patterns associated with an object, and a second level of at least one processing unit arranged to store at least one sequence of patterns associated with the object, wherein a cause of input data received by the hierarchical network is determinable at least partially dependent on belief propagation through the hierarchical network, the belief propagation being at least partially dependent on the sequences stored in the first level of processing units and the second level of at least one processing unit. - View Dependent Claims (41, 42, 43)
-
Specification