Hierarchical based system for identifying object using spatial and temporal patterns

• US 7,624,085 B2
• Filed: 01/11/2007
• Issued: 11/24/2009
• Est. Priority Date: 02/10/2006
• Status: Active Grant

• Current Assignee: Numenta, Inc.
• Original Assignee: Numenta, Inc.
• Inventors: Hawkins, Jeffrey, George, Dileep
• Primary Examiner(s): Vincent; David R
• Assistant Examiner(s): Kennedy; Adrian L
• Application Number:
  US11/622,456
• Publication Number:
  US 20070192264A1
• Time in Patent Office: 1,048 Days
• Field of Search: 706/52, 706/16, 706/23, 700/44, 700/47
• US Class Current: 706/52
• CPC Class Codes: G06N 20/00 Machine learning
• CPC Class Codes: G06N 3/04 Architecture, e.g. interconnection topology
• CPC Class Codes: G06N 3/045 Combinations of networks
• CPC Class Codes: G06N 3/047 Probabilistic or stochastic networks
• CPC Class Codes: G06N 3/049 Temporal neural networks, e.g. delay elements, oscillating neurons or pulsed inputs
• CPC Class Codes: G06N 3/08 Learning methods
• CPC Class Codes: G06N 5/02 Knowledge representation; Symbolic representation
• CPC Class Codes: G06N 7/01 Probabilistic graphical models, e.g. probabilistic networks

Abstract

A hierarchy of computing modules is configured to learn a cause of input data sensed over space and time, and is further configured to determine a cause of novel sensed input data dependent on the learned cause. Further, the hierarchy has a first level of computing modules and a second level of at least one computing module, where at least one of the computing modules in the first level is configured to receive a portion of the novel sensed input data, and where the computing module in the first level is further capable of determining a possible cause of the novel sensed input data dependent on analyzing only a subset of the portion of the novel sensed input data.

Claims

1. A computer-implemented system for determining an identity of a sensed object or a state of sensed object associated with spatial patterns and temporal sequences in input data provided by a sensor, the input data representing at least one of image, video, audio, text, weather conditions, tactile data or data associated with operation of a machine, comprising:

a hierarchy of computing modules configured to receive first input data to learn spatial patterns and temporal sequences in the first input data associated with a sensed object or a state of a sensed object in a learning stage, the hierarchy in an inference stage subsequent to the learning stage further configured to receive second input data and generate identity information about spatial patterns and temporal sequences in the second input data corresponding to the spatial patterns and temporal sequences learned by the hierarchy in the learning stage, the hierarchy of computing modules comprising;

a plurality of first computing modules, each first computing module configured to receive a portion of the second input data and generate first information about sequences in the received portion of the second input data corresponding to learned spatial patterns and temporal sequences;

a relay module configured to receive and select the first information generated by the plurality of the first computing modules; and

a second computing module configured to generate the identity information based on the first information selected by the relay module, wherein the identity information is one of displayed, output for controlling a physical object or stored in a storage medium, for identification of a sensed object or a state of a sensed object.

2. The system of claim 1, wherein the first information generated by the first computing module comprises probability distributions associated with the sensed object or the state of the sensed object.

3. The system of claim 1, wherein the relay module is configured to modify the selected information before sending the selected information to the at least one second computing module.

4. The system of claim 1, wherein the plurality of first computing modules are further configured to receive the second input data across a first input range, the selected information covering a second input range of the second input data, the first input range greater than the second input range.

5. The system of claim 1, wherein the relay module selects the first information based on a control signal received at the relay module.

6. The system of claim 5, wherein the control signal is generated by the first computing modules.

7. The system of claim 5, wherein the control signal is generated by a computing module in the hierarchy of computing modules other than the plurality of the first computing modules or the at least one second computing module.

8. The system of claim 5, wherein the control signal is generated by the at least one second computing module.

9. The system of claim 5, wherein the control signal is received from a source outside the hierarchy of computing modules.

10. The system of claim 1, wherein the relay module is configured to select a subset of the first information generated from the first computing modules.

11. A computer-implemented system for determining an identity of a sensed object or a state of sensed object associated with spatial patterns and temporal sequences in input data provided by a sensor, the input data representing at least one of image, video, audio, text, weather conditions, tactile data or data associated with operation of a machine, comprising:

a hierarchy of computing modules configured to receive first input data to learn spatial patterns and temporal sequences in the first input data associated with a sensed object or a state of a sensed object in a learning stage, the hierarchy in an inference stage subsequent to the learning stage further configured to receive second input data and generate identity information about spatial patterns and temporal sequences in the second input data corresponding to the spatial patterns and temporal sequences learned by the hierarchy in the learning stage, the hierarchy of computing modules comprising;

a plurality of first computing modules, each first computing module configured to receive a portion of the second input data and generate first information about sequences in the received portion of the second input data corresponding to learned spatial patterns and temporal sequences;

a relay module configured to receive and modify the first information generated by the plurality of the first computing modules; and

a second computing module configured to generate the identity information based on the first information selected by the relay module, wherein the identity information is one of displayed, output for controlling a physical object or stored in a storage medium, for identification of a sensed object or a state of a sensed object.

12. The system of claim 11, wherein the relay module is configured to select modified information, the relay module sending the selected information to the at least one second computing module.

13. The system of claim 12, wherein the plurality of first computing modules are further configured to receive the second input data across a first input range, the selected information covering a second input range of the second input data, the first input range greater than the second input range.

14. The system of claim 11, wherein the relay module modifies the first information based on a control signal received at the relay module.

15. A computer-implemented method in a hierarchy of computing modules for determining an identity of a sensed object or a state of a sensed object associated with spatial patterns and temporal sequences in input data provided by a sensor, the hierarchy of computing modules structured to include a plurality of first computing modules and at least one second computing module, the input data representing at least one of image, video, audio, text, weather conditions, tactile data or data associated with operation of a machine, comprising:

the hierarchy of computing modules, in a learning stage, learning spatial patterns and temporal sequences in a first input data associated with a sensed object or a state of a sensed object responsive to receiving the first input data;

each of the first computing module, in an inference stage subsequent to the learning stage, generating first information responsive to receiving a portion of a second input data, the first information indicative of temporal sequences in the second input data corresponding to learned spatial patterns and temporal sequences;

a relay module, in the inference stage, receiving and selecting the first information generated by the plurality of the first computing modules; and

the at least one second computing module, in the inference stage, generating identity information responsive to receiving the selected first information, the identity information is one of displayed, output for controlling a physical object or stored in a storage medium, for identification of a sensed object or a state of a sensed object.

16. The computer-implemented method of claim 15, wherein the plurality of first computing modules are further configured to receive the input data across a first input range, the subset of the first information generated based on a second input range of the input data, the first input range greater than the second input range.

17. The computer-implemented method of claim 15, further comprising:

the relay module modifying the selected information responsive to receiving a control signal.

18. The computer-implemented method of claim 17, wherein the control signal is generated by the first computing modules.

19. The computer-implemented method of claim 17, wherein the control signal is generated by a computing module in the hierarchy of computing modules other than the plurality of the first computing modules or the at least one second computing module.

20. The computer-implemented method of claim 17, wherein the control signal is received from a source outside the hierarchy of computing modules.

21. A computer readable storage medium structured to store instructions executable by a processor in a computing device, the instructions adapted to implement a hierarchy of computing modules including a plurality of first computing modules and at least one second computing module, the hierarchy of computing modules determining an identity of a sensed object or a state of sensed object associated with spatial patterns and temporal sequences in input data provided by a sensor, the input data representing at least one of image, video, audio, text, weather conditions, tactile data or data associated with operation of a machine, the instructions when executed cause the processor to:

the hierarchy of computing modules, in a learning stage, learn spatial patterns and temporal sequences in a first input data associated with a sensed object or a state of a sensed object responsive to receiving the first input data;

at each of the first computing module, in an inference stage subsequent to the learning stage, generate first information responsive to receiving a portion of a second input data, the first information indicative of temporal sequences in the second input data corresponding to learned spatial patterns and temporal sequences;

at a relay module, in the inference stage, receive and select the first information generated by the plurality of the first computing modules; and

at the second computing module, in the inference stage, generate identity information responsive to receiving the selected first information, the identity information is one of displayed, output for controlling a physical object or stored in a storage medium, for identification of a sensed object or a state of a sensed object.

22. The computer readable storage medium of claim 21, further comprising instructions to:

selectively relay the first information from the first computing modules to the second computing modules responsive to receiving a control signal at the relay module.

23. The computer readable storage medium of claim 22, wherein the control signal is generated by the first computing modules.

24. The computer readable storage medium of claim 22, wherein the control signal is generated by a computing module in the hierarchy of computing modules other than the plurality of the first computing modules or the second computing module.

25. The computer readable storage medium of claim 22, wherein the control signal is received from a source outside the hierarchy of computing modules.

26. The computer readable storage medium of claim 22, wherein the relay module modifies the information based on a control signal received at the relay module.

27. A computer-implemented system for determining an identity of a sensed object or a state of a sensed object associated with spatial patterns and temporal sequences in input data provided by a sensor, the input data representing at least one of image, video, audio, text, weather conditions, tactile data or data associated with operation of a machine, comprising:

a hierarchy of computing modules configured to receive first input data to learn spatial patterns and temporal sequences in the first input data associated with a sensed object or a state of a sensed object in a learning stage, the hierarchy in an inference stage subsequent to the learning stage further configured to receive second input data and generate identity information about spatial patterns and temporal sequences in the second input data corresponding to the spatial patterns and temporal sequences learned by the hierarchy in the learning stage, the hierarchy of computing modules comprising;

a plurality of first computing modules, at least one of the first computing modules configured to receive a portion of the input data and generate first information indicative of sequences in the input data corresponding to learned spatial patterns and temporal sequences; and

at least one second computing module configured to generate the identity information based on the first information received from the plurality of first computing modules, wherein the identity information is one of displayed, output for controlling a physical object or stored in a storage medium, for identification of a sensed object or a state of a sensed object; and

a category attention module for switching on or off a subset of the learned spatial patterns and temporal sequences associated with the identity of one or more sensed objects or states of sensed object.

28. A computer-implemented method for determining an identity of a sensed object or a state of a sensed object associated with spatial patterns and temporal sequences in input data provided by a sensor, the input data representing at least one of image, video, audio, text, weather conditions, tactile data or data associated with operation of a machine, the method performed by a hierarchy of computing modules structured to include a plurality of first computing modules and at least one second computing module, comprising:

switching on or off a subset of spatial patterns and temporal sequences associated with the identity of one or more sensed objects or states of a sensed object;

each of the first computing module, generating first information indicative of temporal sequences in the input data corresponding to the spatial patterns and temporal sequences that are switched on, responsive to receiving a portion of the input data; and

the at least one second computing module, generating identity information based on the first information received from the plurality of first computing modules, wherein the identity information is one of displayed, output for controlling a physical object or stored in a storage medium, for identification of a sensed object or a state of a sensed object.

29. A computer readable storage medium structured to store instructions executable by a processor in a computing device to determine an identity of a sensed object or a state of a sensed object associated with spatial patterns and temporal sequences in input data provided by a sensor, the input data representing at least one of image, video, audio, text, weather conditions, tactile data or data associated with operation of a machine, the instructions adapted to implement a hierarchy of computing modules including a plurality of first computing modules and at least one second computing module, the instructions when executed cause the processor to:

switch on or off a subset of spatial patterns and temporal sequences associated with the identity of one or more sensed objects or states of sensed objects;

each of the second computing module, generate first information indicative of temporal sequences in the input data corresponding to the spatial patterns and temporal sequences that are switched on, responsive to receiving a portion of the input data; and

the second computing module, generate identity information based on the first information received from the plurality of first computing modules, wherein the identity information is one of displayed, output for controlling a physical object or stored in a storage medium, for identification of a sensed object or a state of a sensed object.