BELIEF PROPAGATION IN A HIERARCHICAL TEMPORAL MEMORY BASED SYSTEM

• US 20070192271A1
• Filed: 01/11/2007
• Published: 08/16/2007
• Est. Priority Date: 02/10/2006
• Status: Active Grant

• Current Assignee: Numenta, Inc.
• Original Assignee: Numenta, Inc.
• Inventors: Hawkins, Jeffrey, George, Dileep
• Granted Patent: US 7,899,775 B2
• Time in Patent Office: Days
• US Class Current: 706/52
• CPC Class Codes: G06N 20/00 Machine learning
• CPC Class Codes: G06N 3/04 Architecture, e.g. interconnection topology
• CPC Class Codes: G06N 3/045 Combinations of networks
• CPC Class Codes: G06N 3/047 Probabilistic or stochastic networks
• CPC Class Codes: G06N 3/049 Temporal neural networks, e.g. delay elements, oscillating neurons or pulsed inputs
• CPC Class Codes: G06N 3/08 Learning methods
• CPC Class Codes: G06N 5/02 Knowledge representation; Symbolic representation
• CPC Class Codes: G06N 7/01 Probabilistic graphical models, e.g. probabilistic networks

Abstract

A hierarchy of computing modules is configured to (i) learn a cause of input data sensed over space and time, and (ii) determine a cause of novel sensed input data dependent on the learned cause. The hierarchy has a first level of computing modules and a second level of at least one computing module, wherein a computing module in the first level is configured to output to the computing module in the second level a first set of values representing probabilities of possible causes of input data received by the system.

Claims

1. A system, comprising:

a hierarchy of computing modules configured to learn a cause of input data sensed over space and time, the hierarchy further configured to determine a cause of novel sensed input data dependent on the learned cause, and the hierarchy having a first level of computing modules and a second level of at least one computing module,

wherein a computing module in the first level is configured to output to the computing module in the second level a first set of values representing probabilities of possible causes of input data received by the system.

2. The system of claim 1, wherein the first set of values is determined by the computing module in the first level dependent on the input data received by the system.

3. The system of claim 1, wherein the possible causes comprise causes previously learned by the computing module in the first level.

4. The system of claim 1, wherein the computing module in the second level is configured to determine a possible cause of the input data received by the system dependent on the set of values output by the computing module in the first level.

5. The system of claim 1, wherein the computing module in the second level is configured to determine a second set of values representing probabilities of possible causes of the input data received by the system.

6. The system of claim 5, wherein the computing module in the second level is further configured to determine the second set of values dependent on the first set of values output by the computing module in the first level.

7. The system of claim 6, wherein the computing module in the second level is further configured to output the second set of values to the computing module in the first level.

8. The system of claim 6, wherein the computing module in the second level is further configured to determine the second set of values dependent on a conditional probability table stored in association with the computing module in the second level.

9. The system of claim 5, wherein the computing module in the second level is further configured to output the second set of values to the computing module in the first level.

10. The system of claim 1, wherein at least one of the computing module in the first level and the computing module in the second level is implemented at least in part in at least one of software and hardware.

11. The system of claim 1, wherein an input range of the computing module in the second level is greater than an input range of the computing module in the first level.

12. A computer-implemented method of recognizing an object, comprising:

learning a cause of a first set of input data received over space and time by a hierarchy of computing modules having a first level of a plurality of computing modules and a second level of at least one computing module;

receiving a second set of input data;

determining at the first level a first set of probabilities representing possible causes of the second set of input data, wherein the determination is dependent on the learned cause;

outputting the first set of probabilities to the second level; and

determining a likely cause of the second set of input data dependent on the first set of probabilities received at the second level.

13. The computer-implemented method of claim 12, wherein the second set of input data is distinct from the first set of input data, and wherein the second set of input data is received by the hierarchy after the cause of the first set of input data has been learned.

14. The computer-implemented method of claim 12, further comprising:

determining at the second level a second set of probabilities representing possible causes of the second set of input data.

15. The computer-implemented method of claim 14, further comprising:

determining the second set of probabilities dependent on the first set of probabilities received at the second level.

16. The computer-implemented method of claim 15, further comprising:

outputting the second set of probabilities to the first level.

17. The computer-implemented method of claim 14, further comprising:

outputting the second set of probabilities to the first level.

18. The computer-implemented method of claim 12, further comprising:

determining the first set of probabilities dependent on an output from the second level.

19. A computer-readable medium having instructions therein that are executable by a processor, the instructions comprising instructions to:

learn a cause of a first set of input data received over space and time by a hierarchy of computing modules having a first level of a plurality of computing modules and a second level of at least one computing module;

receive a second set of input data;

determine at the first level a first set of probabilities representing possible causes of the second set of input data, wherein the determination is dependent on the learned cause;

output the first set of probabilities to the second level; and

determine a likely cause of the second set of input data dependent on the first set of probabilities received at the second level.

20. The computer-readable medium of claim 19, further comprising instructions to:

determine at the second level a second set of probabilities representing possible causes of the second set of input data.

21. The computer-readable medium of claim 20, further comprising instructions to:

determine the second set of probabilities dependent on the first set of probabilities received at the second level.

22. The computer-readable medium of claim 21, further comprising instructions to:

determine the first set of probabilities dependent on the second set of probabilities.