Message passing in a hierarchical temporal memory based system

• US 7,904,412 B2
• Filed: 01/11/2007
• Issued: 03/08/2011
• Est. Priority Date: 02/10/2006
• Status: Active Grant

• Current Assignee: Numenta, Inc.
• Original Assignee: Numenta, Inc.
• Inventors: Saphir, William, Marianetti, Ronald II, Hawkins, Jeffrey
• Primary Examiner(s): Holmes; Michael B.
• Assistant Examiner(s): Kennedy; Adrian L
• Application Number:
  US11/622,455
• Publication Number:
  US 20070192269A1
• Time in Patent Office: 1,517 Days
• Field of Search: None
• US Class Current: 706/52
• CPC Class Codes: G06N 20/00 Machine learning
• CPC Class Codes: G06N 3/04 Architecture, e.g. interconnection topology
• CPC Class Codes: G06N 3/045 Combinations of networks
• CPC Class Codes: G06N 3/047 Probabilistic or stochastic networks
• CPC Class Codes: G06N 3/049 Temporal neural networks, e.g. delay elements, oscillating neurons or pulsed inputs
• CPC Class Codes: G06N 3/08 Learning methods
• CPC Class Codes: G06N 5/02 Knowledge representation; Symbolic representation
• CPC Class Codes: G06N 7/01 Probabilistic graphical models, e.g. probabilistic networks

Abstract

A hierarchy of computing modules is configured to learn a cause of input data sensed over space and time, and is further configured to determine a cause of novel sensed input data dependent on the learned cause. Further, the hierarchy has a first level of computing modules and a second level of at least one computing module, where at least one of the computing modules in the first level operates on a first server, and where the at least one computing module in the second level operates on a second server. The hierarchy also includes a message manager module configured to relay information between the first server and the second server.

Claims

1. A computer-implemented system, comprising:

a hierarchy of computing modules configured to receive first input data to learn spatial patterns and temporal sequences in the first input data in a learning stage, the hierarchy in an inference stage subsequent to the learning stage configured to receive second input data and generate identity information about spatial patterns and temporal sequences in the second input data, the hierarchy comprising;

a first level of computing modules configured to output first information about patterns and sequences in the second input data corresponding to the learned spatial patterns and temporal sequences, wherein at least one of the computing modules in the first level operates on a first server; and

at least one computing module at a second level associated with the first level of computing modules, the at least one computing module configured to receive the first information and output second information about patterns and sequences in the first information corresponding to the learned spatial patterns and temporal sequences, wherein the at least one computing module in the second level operates on a second server; and

at least one message manager module configured to relay at least part of the first information or part of the second information between the first server and the second server.

2. The system of claim 1, wherein the message manager module operates on the first server.

3. The system of claim 1, wherein the message manager module operates on the second server.

4. The system of claim 1, wherein the message manager module is further configured to receive the first information generated from one or more computing modules in the first level and distribute the the first information to the second server.

5. The system of claim 1, wherein the message manager module is further configured to receive the second information generated from the at least one computing module in the second level and distribute the second information to the first server.

6. The system of claim 1, wherein the message manager module is further configured to relay the at least part of the first information dependent on topology information about the hierarchy.

7. The system of claim 1, wherein at least another of the computing modules in the first level operates on the second server, and wherein the message manager module is further configured to relay information between the at least one other computing module in the first level and the computing module in the second level.

8. The system of claim 1, wherein an input range of the computing module in the second module is greater than an input range of the computing module in the first level.

9. The system of claim 1, wherein the message manager module is further configured to operate according to at least one of a message passing interface (MPI) protocol and a zero-copy protocol using shared memory.

10. The system of claim 1, wherein the message manager module is further configured to operate dependent on at least one of a socket connection and a shared memory buffer.

11. A computer-implemented method, comprising:

learning spatial patterns and temporal sequences in first input data in a learning stage responsive to receiving the first input data in a learning stage at a hierarchy of computing modules comprising a first level of computing modules and at least one computing module at a second level, wherein at least one of the computing modules in the first level of computing modules operates on a first server and the at least one computing module at the second level operates on a second server;

the first level of computing modules in an inference stage subsequent to the learning stage outputting first information responsive to receiving the second input data, the first information representing information about spatial patterns and temporal sequences in the second input data corresponding to the learned spatial patterns and temporal sequences;

passing at least part of the first information to the at least one computing module at the second level; and

the at least one computing module at the second level in the inference stage generating second information responsive to receiving the at least part of the first information, the second information representing information about spatial patterns and temporal sequences in the at least part of the first information.

12. The computer-implemented method of claim 11, wherein passing the at least part of the first information comprises:

at message manager module, receiving the at least part of the first information generated from the at least one computing module in the first level; and

distributing the at least part of the first information to the second server.

13. The computer-implemented method of claim 11, further comprising:

at message manager module, receiving at least part of the second information generated from the at least one computing module in the second level; and

distributing the at least part of the second information to the first server.

14. The computer-implemented method of claim 11, wherein the passing of the at least part of the first information is dependent on topology information about the hierarchy.

15. The computer-implemented method of claim 11, wherein at least another of the computing modules in the first level operates on the second server, the method further comprising:

relaying information between the at least one other computing module in the first level and the at least one computing module in the second level.

16. The computer-implemented method of claim 11, wherein an input range of the computing module in the second module is greater than an input range of the at least one computing module in the first level.

17. A computer-readable medium having instructions therein that are executable by a processor, the instructions comprising instructions to:

learn spatial patterns and temporal sequences in first input data in a learning stage responsive to receiving the first input data in a learning stage at a hierarchy of computing modules comprising a first level of computing modules and at least one computing module at a second level, wherein at least one of the computing modules in the first level of computing modules operates on a first server and the at least one computing module at the second level operates on a second server;

output first information from the first level of computing modules in an inference stage subsequent to the learning stage responsive to receiving the second input data, the first information representing information about spatial patterns and temporal sequences in the second input data corresponding to the learned spatial patterns and temporal sequences;

pass at least part of the first information to the at least one computing module at the second level; and

generate by the at least one computing module at the second level in the inference stage second information responsive to receiving the at least part of the first information, the second information representing information about spatial patterns and temporal sequences in the at least part of the first information.

18. The computer-readable medium of claim 17, wherein the instructions to pass the at least part of the first information comprise instructions to:

at message manager module, receive the at least part of the first information generated from the at least one computing module in the first level; and

distribute the at least part of the first information to the second server.

19. The computer-readable medium of claim 17, further comprising instructions to:

at message manager module, receive at least part of the second information generated from the at least one computing module in the second level; and

distribute the at least part of the second information to the first server.

20. The computer-readable medium of claim 17, wherein the instructions to pass the at least part of the first information are dependent on topology information about the hierarchy.

21. The computer-readable medium of claim 17, wherein at least another of the computing modules in the first level operates on the second server, the computer-readable medium further comprising instructions to:

relay information between the at least one other computing module in the first level and the at least one computing module in the second level.

22. The computer-readable medium of claim 17, wherein an input range of the computing module in the second module is greater than an input range of the at least one computing module in the first level.