Temporal memory using sparse distributed representation

• US 9,189,745 B2
• Filed: 03/11/2011
• Issued: 11/17/2015
• Est. Priority Date: 03/15/2010
• Status: Active Grant

• Current Assignee: Numenta, Inc.
• Original Assignee: Numenta, Inc.
• Inventors: Hawkins, Jeffrey C., Marianetti, Ronald II, Raj, Anosh, Ahmad, Subutai
• Primary Examiner(s): Gaffin, Jeffrey A
• Assistant Examiner(s): FINK, THOMAS ANDREW
• Application Number:
  US13/046,464
• Publication Number:
  US 20110225108A1
• Time in Patent Office: 1,712 Days
• US Class Current: 1/1
• CPC Class Codes: G06N 20/00 Machine learning
• CPC Class Codes: G06N 3/047 Probabilistic or stochastic networks
• CPC Class Codes: G06N 3/049 Temporal neural networks, e.g. delay elements, oscillating neurons or pulsed inputs
• CPC Class Codes: G06N 3/08 Learning methods
• CPC Class Codes: G06N 7/01 Probabilistic graphical models, e.g. probabilistic networks

Abstract

A processing node in a temporal memory system includes a spatial pooler and a sequence processor. The spatial pooler generates a spatial pooler signal representing similarity between received spatial patterns in an input signal and stored co-occurrence patterns. The spatial pooler signal is represented by a combination of elements that are active or inactive. Each co-occurrence pattern is mapped to different subsets of elements of an input signal. The spatial pooler signal is fed to a sequence processor receiving and processed to learn, recognize and predict temporal sequences in the input signal. The sequence processor includes one or more columns, each column including one or more cells. A subset of columns may be selected by the spatial pooler signal, causing one or more cells in these columns to activate.

Claims

1. A processing node in a computing device, comprising:

a processor;

a spatial pooler configured to;

detect spatial patterns in an input signal, the spatial patterns in the input signal changing over time; and

generate a series of spatial pooler outputs, each spatial pooler output having a plurality of output elements and in sparse distributed representation to indicate that a subset of the output elements is active, each active output element of the spatial pooler output indicating detection of one of a plurality of spatial pattern in the input signal at a time; and

a sequence processor comprising a plurality of groups of cells that are selectively activated responsive to receiving the series of spatial pooler outputs, the sequence processor configured to generate a sequence processor output based on activation of each cell in the groups, each cell in each of the groups activated based at least on;

(a) activation of one of the output elements associated with each of the groups; and

(b) predictive activation of each cell in the groups connected to a subset of other cells in the groups, the predictive activation of each cell based on comparison of (i) activation of the subset of other cells connected to each cell at a first time after a second time and (ii) activation of the subset of other cells at the second time when each cell previously became activated.

2. The processing node of claim 1, wherein the sequence processor output represents a temporal sequence of spatial patterns in the input signal detected or predicted.

3. The processing node of claim 1, wherein the spatial pooler comprises a plurality of co-occurrence detectors for detecting the spatial patterns, each co-occurrence detector configured to generate a match score representing a degree of match between a co-occurrence pattern assigned to each co-occurrence detector and a spatial pattern in the input signal, wherein each of the series of spatial pooler outputs represent selection of a subset of co-occurrence detectors based on match scores generated by the co-occurrence detectors.

4. The processing node of claim 3, wherein a spatial pattern in the input signal is represented by a vector with a plurality of input elements in binary values to indicate whether each input element is active or inactive, and each co-occurrence detector is mapped to a subset of the plurality of input elements, wherein the match score of each co-occurrence detector represents a number of active input elements in the vector mapped to the co-occurrence detector.

5. The processing node of claim 3, wherein each of the plurality of co-occurrence detectors is assigned to different input space of the input signal, and wherein the assignment of the input space corresponds to topological relationship between the plurality of co-occurrence detectors.

6. The processing node of claim 5, wherein the plurality of co-occurrence detectors comprise a first co-occurrence detector and a second co-occurrence detector adjacent to the first co-occurrence detector, the first co-occurrence detector mapped to input elements within first input space and the second co-occurrence detector mapped to input elements within second input space adjacent to the first input space.

7. The processing node of claim 3, wherein the spatial pooler is configured to

select a first co-occurrence detector at a third time;

and

retain, at a fourth time subsequent to the third time, selection of the first co-occurrence detector over a second co-occurrence detector responsive to a match score of the second co-occurrence detector at the fourth time not exceeding a match score of the first co-occurrence detector by a threshold difference.

8. The processing node of claim 3, wherein the spatial pooler is configured to modify a co-occurrence pattern assigned to a co-occurrence detector generating a match score lower than other co-occurrence detectors or generating a match score lower than a threshold for a predetermined amount of time.

9. The processing node of claim 3, wherein the spatial pooler is configured to:

adjust a permanent value for each potential mapping of an input element in the input signal to the co-occurrence detector responsive to receiving the spatial patterns, the permanence value representing whether mapping of an input element to the co-occurrence detector will contribute to the selection of the co-occurrence detector; and

replace potential mapping of an input element with assigned mapping of another input element in the input signal to the co-occurrence detector responsive to a permanence value of the potential mapping exceeding a threshold.

10. The processing node of claim 9, wherein each assigned mapping of an input element to a co-occurrence detector has a permanence value that is reduced by a decay function over time unless each assigned mapping contributes to the selection of the co-occurrence detector.

11. The processing node of claim 3, wherein the spatial pooler is configured to:

determine a frequency at which each of the plurality of co-occurrence detectors is selected, wherein the plurality of co-occurrence detectors include a first co-occurrence detector and a second co-occurrence detector selected more frequently than the first co-occurrence detector over a predetermined amount of time;

increase likelihood of selecting the first co-occurrence detector relative to the second co-occurrence detector by processing match scores of the first and second co-occurrence detectors using an activation function; and

select the co-occurrence detectors based on the processed match scores.

12. The processing node of claim 3, wherein the co-occurrence patterns for the plurality of co-occurrence detectors are copied from a plurality of master co-occurrence patterns stored in memory of the computing device.

13. The processing node of claim 1, wherein the spare distributed representation includes 1 to 10% of output elements that are active.

14. The processing node of claim 1, wherein each of the series of spatial pooler outputs has the same number of active output elements.

15. The processing node of claim 3, wherein co-occurrence detectors are assigned with a distance relative to each other, and wherein selection of a co-occurrence detector inhibits selection of other co-occurrence detectors within a predetermined distance from the selected co-occurrence detector.

16. The processing node of claim 15, wherein the spatial pooler is configured to:

process match scores of the plurality of co-occurrence detectors with convolution max function; and

select the co-occurrence detectors with highest processed match scores.

17. The processing node of claim 3, wherein a subset of co-occurrence detectors with highest match scores are selected.

18. The processing node of claim 1, wherein each of the groups in the sequence processor correspond to an input element in the input signal, at least one cell in each column activated responsive to the corresponding input element in the input signal being active.

19. The processing node of claim 1, wherein each cell comprises one or more temporal memory segments for storing the activation of the subset of other cells at a single point of time or different times before the first time, the cell activated responsive to (i) detecting the activation of the subset of other cells stored in the one or more temporal memory segments of the cell or (ii) activation of a group including the cell and selection of the cell in the group for activation.

20. The processing node of claim 19, wherein each cell generates a sequence output sent to another cell for activating the other cell responsive to predictive activation of the cell.

21. The processing node of claim 19, wherein the sequence processor is configured to predictively activate each cell responsive to a threshold number of the subset of other cells connected to each cell being active.

22. The processing node of claim 21, wherein the threshold number is dynamically adjusted.

23. The processing node of claim 21, wherein each cell is activated when a threshold number of cells within a window sliding across a plurality of temporal memory segments are activated.

24. The processing node of claim 20, wherein more than one set of activation states of the subset of other cells at different times are stored in a single temporal memory segment.

25. The processing node of claim 19, wherein the sequence processor further comprises a monitor configured to monitor frequency that the activation states of the subset of other cells as stored in the one or more temporal memory segments are detected, the sequence processor removing a temporal memory segment responsive to the frequency dropping below a threshold.

26. The processing node of claim 1, wherein the sequence processor is configured to associate temporal sequences of different lengths.

27. The processing node of claim 1, wherein the processing node forms a hierarchy of processing nodes in conjunction with a parent processing node higher in the hierarchy than the processing node, the parent processing node generating a parent output that is more stable or abstract that the sequence processor output based on the sequence processor output received from the processing node.

28. The processing node of claim 1, wherein the sequence processor is configured to perform online learning of temporal sequences.

57. The processing node of claim 1, wherein each cell belongs to a first group of cells and the subset of other cells belongs to groups other than the first group of cells.

29. A method of detecting and processing sequences of spatial patterns in an input, comprising:

at a spatial pooler, detecting spatial patterns in an input signal, the spatial patterns in the input signal changing over time;

at the spatial pooler, generating a series of spatial pooler outputs, each spatial pooler output having a plurality of output elements and in sparse distributed representation to indicate that a subset of the output elements is active, each active output element of the spatial pooler output indicating detection of one of a plurality of spatial patterns in the input signal at a time;

selectively activating each cell in each of groups of cells in a sequence processor based at least on;

(a) activation of one of the output elements associated with each of the groups; and

(b) predictive activation of each cell in the groups connected to a subset of other cells in the groups, the predictive activation of each cell based on comparison of (i) activation of the subset of other cells connected to each cell at a first time after a second time and (ii) activation of the subset of other cells at the second time when each cell previously became activated; and

at the sequence processor, generating a sequence processor output based on activation of each cell in the groups.

30. The method of claim 29, wherein the sequence processor output represents a temporal sequence of spatial patterns in the input signal detected or predicted.

31. The method of claim 29, further comprising:

generating match scores for a plurality of co-occurrence detectors, each match score representing a degree of match between a co-occurrence pattern assigned to each of the plurality of co-occurrence detectors and a spatial pattern in the input signal;

selecting a subset of the plurality of co-occurrence detectors based on the match scores; and

generating a spatial pooler output indicating the selected subset of the plurality of co-occurrence detectors.

32. The method of claim 31, wherein a spatial pattern in the input signal is represented by a vector with a plurality of input elements in binary values to indicate whether each input element is active or inactive, and each co-occurrence detector is mapped to a subset of the plurality of input elements, wherein the match score of each co-occurrence detector represents a number of active input elements in the vector mapped to the co-occurrence detector.

33. The method of claim 31, wherein each of the plurality of co-occurrence detectors is assigned to different input space of the input signal, wherein the assignment of the input space corresponds to topological relationship between the plurality of co-occurrence detectors.

34. The method of claim 33, wherein the plurality of co-occurrence detectors comprise a first co-occurrence detector and a second co-occurrence detector adjacent to the first co-occurrence detector, the first co-occurrence detector mapped to input elements within first input space and the second co-occurrence detector mapped to input elements within second input space adjacent to the first input space.

35. The method of claim 31, further comprising:

selecting a first co-occurrence detector by the spatial pooler at a third time; and

retaining, at a fourth time subsequent to the third time, selection of the first co-occurrence detector over a second co-occurrence detector responsive to a match score of the second co-occurrence detector at the fourth time not exceeding a match score of the first co-occurrence detector by a threshold difference.

36. The method of claim 31, further comprising modifying a co-occurrence pattern assigned to a co-occurrence detector that generates a match score lower than other co-occurrence detectors or generates a match score lower than a threshold for a predetermined amount of time.

37. The method of claim 31, further comprising:

adjusting a permanent value for each potential mapping of an input element in the input signal to the co-occurrence detector responsive to receiving the spatial patterns, the permanence value representing whether mapping of an input element to the co-occurrence detector will contribute to the selection of the co-occurrence detector; and

replacing potential mapping of an input element with assigned mapping of another input element in the input signal to the co-occurrence detector responsive to a permanence value of the potential mapping exceeding a threshold.

38. The method of claim 37, further comprising reducing a permanence value of each assigned mapping of an input element to a co-occurrence detector by a decay function over time unless each assigned mapping contributes to the selection of the co-occurrence detector.

39. The method of claim 31, further comprising:

determining a frequency at which each of the plurality of co-occurrence detectors is selected, wherein the plurality of co-occurrence detectors include a first co-occurrence detector and a second co-occurrence detector selected more frequently than the first co-occurrence detector over a predetermined amount of time;

increasing likelihood of selecting the first co-occurrence detector relative to the second co-occurrence detector by processing match scores of the first and second co-occurrence detectors using an activation function; and

selecting the co-occurrence detectors based on the processed match scores.

40. The method of claim 31, further comprising copying co-occurrence patterns for the plurality of co-occurrence detectors from a plurality of master co-occurrence patterns.

41. The method of claim 29, wherein the spare distributed representation includes 1 to 10% of output elements that are active.

42. The method of claim 29, wherein each of the series of spatial pooler outputs has the same number of active output elements.

43. The method of claim 31, wherein co-occurrence detectors are assigned with a distance relative to each other, wherein selection of a co-occurrence detector inhibits selection of other co-occurrence detectors within a predetermined distance from the selected co-occurrence detector.

44. The method of claim 43, further comprising:

processing match scores of the plurality of co-occurrence detectors with convolution max function; and

selecting the co-occurrence detectors with highest processed match scores.

45. The method of claim 31, wherein a subset of co-occurrence detectors with highest match scores are selected.

46. The method of claim 29, further comprising activating at least one cell in a group of cells activated responsive to an input element in the input signal corresponding to the group of cells being active.

47. The method of claim 29, wherein each cell is predictively activated responsive to detecting activation states of the subset of other cells stored in one or more temporal memory segments of each cell.

48. The method of claim 47, further comprising generating at each cell a sequence output sent to another cell for activating the other cell responsive to predictive activation of the cell.

49. The method of claim 47, further comprising predictively activating the cell at the sequence processor responsive to a threshold number of the subset of other cells connected to the cell becoming activate.

50. The method of claim 49, wherein the threshold number is dynamically adjusted.

51. The method of claim 49, further comprising activating each cell responsive to a threshold number of cells within a window sliding across a plurality of temporal memory segments being activate.

52. The method of claim 48, wherein more than one set of activation states of the subset of other cells at different times are stored in a single temporal memory segment of the cell.

53. The method of claim 47, further comprising:

monitoring frequency that the activation states of cells as stored in the one or more temporal memory segments are detected; and

removing a temporal memory segment responsive to the frequency dropping below a threshold.

54. The method of claim 29, wherein associating temporal sequences comprises associating temporal sequences of different lengths.

55. The method of claim 29, further comprising performing online learning of the temporal sequences.

58. The method of claim 29, wherein each cell belongs to a first group of cells and the subset of other cells belongs to groups other than the first group of cells.

56. A non-transitory computer readable storage medium structured to store instructions executable by a processor in a computing device, the instructions, when executed cause the processor to:

detect spatial patterns in an input signal, the spatial patterns in the input signal changing over time;

generate a series of spatial pooler outputs, each spatial pooler output having a plurality of output elements and in sparse distributed representation to indicate that a subset of the output elements is active, each active output element of the spatial pooler output indicating detection of one of a plurality of spatial patterns in the input signal at a time;

selectively activate each cell in each of groups of cells in a sequence processor based at least on;

(a) activation of one of the output elements associated with each of the groups; and

(b) predictive activation of each cell in the groups connected to a subset of other cells in the groups, the predictive activation of each cell based on comparison of (i) activation of the subset of other cells connected to each cell at a first time after a second time and (ii) activation of the subset of other cells at the second time when each cell previously became activated; and

generate a sequence processor output based on activation of each cell in the groups.