Pattern detection feedback loop for spatial and temporal memory systems

• US 9,552,551 B2
• Filed: 06/27/2014
• Issued: 01/24/2017
• Est. Priority Date: 08/25/2011
• Status: Active Grant

• Current Assignee: Numenta, Inc.
• Original Assignee: Numenta, Inc.
• Inventors: Marianetti, Ronald II, Raj, Anosh, Ahmad, Subutai
• Primary Examiner(s): Chaki, Kakali
• Assistant Examiner(s): PELLETT, DANIEL T
• Application Number:
  US14/316,805
• Publication Number:
  US 20140310227A1
• Time in Patent Office: 942 Days
• US Class Current: 1/1
• CPC Class Codes: G06N 20/00 Machine learning
• CPC Class Codes: G06N 5/047 Pattern matching networks; Rete networks

Abstract

A spatial and temporal memory system (STMS) processes input data to detect whether spatial patterns and/or temporal sequences of spatial patterns exist within the data, and to make predictions about future data. The data processed by the STMS may be retrieved from, for example, one or more database fields and is encoded into a distributed representation format using a coding scheme. The performance of the STMS in predicting future data is evaluated for the coding scheme used to process the data as performance data. The selection and prioritization of STMS experiments to perform may be based on the performance data for an experiment. The best fields, encodings, and time aggregations for generating predictions can be determined by an automated search and evaluation of multiple STMS systems.

Claims

1. A method of searching for spatial patterns or temporal sequences of spatial patterns in input data, comprising:

configuring each of a plurality of spatial and temporal memory systems, each spatial and temporal memory system configured to encode a first subset of the input data based on a first encoder configuration to generate encoded data;

identifying, by each of the spatial and temporal memory systems, spatial patterns or temporal sequences of spatial patterns within the encoded data;

evaluating the predictive performance of each of the spatial and temporal memory systems based on the identified spatial patterns or temporal sequences of spatial patterns; and

responsive to one or more of the spatial and temporal memory systems failing to satisfy one or more predetermined predictive performance criteria, reconfiguring the one or more spatial and temporal memory systems to encode a second subset of the input data based on a second encoder configuration.

2. The method of claim 1, wherein each of the spatial and temporal memory systems is further configured to time aggregate a corresponding subset of the input data before encoding the subset of input data.

3. The method of claim 1, wherein each of the spatial and temporal memory systems is further configured to:

analyze a corresponding subset of the input data before encoding the subset of input to determine at least one of;

the input data types, the distribution of input data values, and the range of input data values; and

encode the subset of the input data based on the analysis.

4. The method of claim 1, wherein each of the spatial and temporal memory systems is configured based on at least one of:

input data previously provided to each of the spatial and temporal memory systems, a previous configuration of each of the spatial and temporal memory systems, analysis of the input data, and user-selected parameters.

5. The method of claim 1, wherein evaluating the predictive performance of a spatial and temporal memory system comprises determining a predicted spatial pattern based on an output of the spatial and temporal memory system, the output generated based on the spatial patterns or temporal sequences of spatial patterns identified at the spatial and temporal memory system.

6. The method of claim 1, wherein at least one predetermined predictive performance criteria comprises a threshold level of predictive accuracy of a spatial and temporal memory system in identifying spatial patterns or temporal sequences of spatial patterns within the encoded data.

7. The method of claim 1, further comprises:

identifying, by each re-configured spatial and temporal memory system, spatial patterns or temporal sequences of spatial patterns within the second subset of encoded input data;

evaluating the predictive performance of each re-configured spatial and temporal memory system based on the identified spatial patterns or temporal sequences of spatial patterns within the second subset of encoded input data; and

responsive to one or more of the re-configured spatial and temporal memory systems failing to satisfy one or more predetermined predictive performance criteria, configuring the one or more of the re-configured spatial and temporal memory systems to encode a third subset of the input data based on a third encoder configuration.

8. A system for searching for spatial patterns or temporal sequences of spatial patterns in input data, comprising:

a non-transitory computer-readable storage medium storing executable computer instructions configured to, when executed, perform steps comprising;

configuring each of a plurality of spatial and temporal memory systems, each spatial and temporal memory system configured to encode a first subset of the input data based on a first encoder configuration to generate encoded data;

identifying, by each of the spatial and temporal memory systems, spatial patterns or temporal sequences of spatial patterns within the encoded data;

evaluating the predictive performance of each of the spatial and temporal memory systems based on the identified spatial patterns or temporal sequences of spatial patterns; and

responsive to one or more of the spatial and temporal memory systems failing to satisfy one or more predetermined predictive performance criteria, re-configuring the one or more spatial and temporal memory systems to encode a second subset of the input data based on a second encoder configuration; and

a processor configured to execute the instructions.

9. The system of claim 8, wherein each of the spatial and temporal memory systems is further configured to time aggregate a corresponding subset of the input data before encoding the subset of input data.

10. The system of claim 8, wherein each of the spatial and temporal memory systems is further configured to:

analyze a corresponding subset of the input data before encoding the subset of input to determine at least one of;

the input data types, the distribution of input data values, and the range of input data values; and

encode the subset of the input data based on the analysis.

11. The system of claim 8, wherein each of the spatial and temporal memory systems is configured based on at least one of:

input data previously provided to each of the spatial and temporal memory systems, a previous configuration of each of the spatial and temporal memory systems, analysis of the input data, and user-selected parameters.

12. The system of claim 8, wherein evaluating the predictive performance of a spatial and temporal memory system comprises determining a predicted spatial pattern based on an output of the spatial and temporal memory system, the output generated based on the spatial patterns or temporal sequences of spatial patterns identified at the spatial and temporal memory system.

13. The system of claim 8, wherein at least one predetermined predictive performance criteria comprises a threshold level of predictive accuracy of a spatial and temporal memory system in identifying spatial patterns or temporal sequences of spatial patterns within the encoded data.

14. The system of claim 8, wherein the instructions are configured to, when executed, perform further steps comprising:

identifying, by each re-configured spatial and temporal memory system, spatial patterns or temporal sequences of spatial patterns within the second subset of encoded input data;

evaluating the predictive performance of each re-configured spatial and temporal memory system based on the identified spatial patterns or temporal sequences of spatial patterns within the second subset of encoded input data; and

responsive to one or more of the re-configured spatial and temporal memory systems failing to satisfy one or more predetermined predictive performance criteria, configuring the one or more of the re-configured spatial and temporal memory systems to encode a third subset of the input data based on a third encoder configuration.

15. A non-transitory computer-readable storage medium storing executable computer program instructions for searching for spatial patterns or temporal sequences of spatial patterns in input data, the instructions comprising instructions for:

configuring each of a plurality of spatial and temporal memory systems, each spatial and temporal memory system configured to encode a first subset of the input data based on a first encoder configuration to generate encoded data;

identifying, by each of the spatial and temporal memory systems, spatial patterns or temporal sequences of spatial patterns within the encoded data;

evaluating the predictive performance of each of the spatial and temporal memory systems based on the identified spatial patterns or temporal sequences of spatial patterns; and

responsive to one or more of the spatial and temporal memory systems failing to satisfy one or more predetermined predictive performance criteria, re-configuring the one or more spatial and temporal memory systems to encode a second subset of the input data based on a second encoder configuration.

16. The non-transitory computer-readable storage medium of claim 15, wherein each of the spatial and temporal memory systems is further configured to time aggregate a corresponding subset of the input data before encoding the subset of input data.

17. The non-transitory computer-readable storage medium of claim 15, wherein each of the spatial and temporal memory systems is further configured to:

analyze a corresponding subset of the input data before encoding the subset of input to determine at least one of;

the input data types, the distribution of input data values, and the range of input data values; and

encode the subset of the input data based on the analysis.

18. The non-transitory computer-readable storage medium of claim 15, wherein each of the spatial and temporal memory systems is configured based on at least one of:

input data previously provided to each of the spatial and temporal memory systems, a previous configuration of each of the spatial and temporal memory systems, analysis of the input data, and user-selected parameters.

19. The non-transitory computer-readable storage medium of claim 15, wherein evaluating the predictive performance of a spatial and temporal memory system comprises determining a predicted spatial pattern based on an output of the spatial and temporal memory system, the output generated based on the spatial patterns or temporal sequences of spatial patterns identified at the spatial and temporal memory system.

20. The non-transitory computer-readable storage medium of claim 15, wherein at least one predetermined predictive performance criteria comprises a threshold level of predictive accuracy of a spatial and temporal memory system in identifying spatial patterns or temporal sequences of spatial patterns within the encoded data.