Artificial neurons including power series of weights and counts that represent prior and next association

• US 6,581,049 B1
• Filed: 11/08/1999
• Issued: 06/17/2003
• Est. Priority Date: 11/08/1999
• Status: Expired due to Term

• Current Assignee: Intel Corporation
• Original Assignee: Saffron Technology, Inc. (Intel Corporation)
• Inventors: Fleming, James S., Aparicio, Manuel IV, Ariely, Dan
• Primary Examiner(s): Follansbee, John
• Assistant Examiner(s): Hirl, Joseph P.
• Application Number:
  US09/436,925
• Time in Patent Office: 1,317 Days
• Field of Search: 706/39, 706/41, 706/37
• US Class Current: 706/39
• CPC Class Codes: G06N 3/063 using electronic means

Abstract

An artificial neuron includes inputs and dendrites, a respective one of which is associated with a respective one of the inputs. Each dendrite includes a power series of weights, and each weight in a power series includes an associated count for the associated power. The power series of weights preferably is a base-two power series of weights, each weight in the base-two power series including an associated count that represents a bit position. The counts for the associated power preferably are statistical counts. More particularly, the dendrites preferably are sequentially ordered, and the power series of weights preferably includes a pair of first and second power series of weights. Each weight in the first power series includes a first count that is a function of associations of prior dendrites, and each weight of the second power series includes a second count that is a function of associations of next dendrites. More preferably, a first and second power series of weights is provided for each of multiple observation phases. In order to propagate an input signal into the artificial neuron, a trace preferably also is provided that is responsive to an input signal at the associated input. The trace preferably includes a first trace count that is a function of associations of the input signal at prior dendrites, and a second trace count that is a function of associations of the input signal at next dendrites. The first and second power series are responsive to the respective first and second trace counts. The input signal preferably is converted into the first and second trace counts, and a trace wave propagator propagates the respective first and second trace counts into the respective first and second power series of weights.

Claims

1. An artificial neuron comprising:

a plurality of inputs; and

a plurality of dendrites, a respective one of which is uniquely associated with a respective one of the plurality of inputs, each dendrite comprising a power series of weights, each weight in a power series including an associated count for the associated power.

2. An artificial neuron according to claim 1 wherein the power series of weights is a base two power series of weights, each weight in the base two power series including an associated count that represents a bit position of the associated count.

3. An artificial neuron according to claim 1 wherein the counts are statistical counts.

4. An artificial neuron according to claim 1 wherein the dendrites are sequentially ordered and wherein the power series of weights comprises a pair of first and second power series of weights, each weight in the first power series including a first count that is a function of associations of prior dendrites and each weight in the second power series including a second count that is a function of associations of next dendrites.

5. An artificial neuron according to claim 4 wherein each dendrite further comprises:

a trace that is responsive to an input signal at the associated input, the trace including a first trace count that is a function of associations of the input signal at prior dendrites and a second trace count that is a function of associations of the input signal at next dendrites, the respective first and second power series being responsive to the respective first and second trace counts.

6. An artificial neuron according to claim 5 wherein each trace comprises a power series of first trace counts that is a function of associations of the input signal at prior dendrites and a power series of second trace counts that is a function of associations of the input signal at next dendrites.

7. An artificial neuron according to claim 6 wherein the trace wave propagator propagates the trace along the sequentially ordered dendrites in a forward direction and in a reverse direction.

8. An artificial neuron according to claim 6 wherein the trace wave propagator further propagates carry results of the trace along the power series of weights in the plurality of dendrites to provide memorization of the input signal.

9. An artificial neuron according to claim 8 wherein the trace wave propagator further comprises a Double Match/Filter that identifies carry results for a weight in a dendrite, for propagation to a next higher power weight.

10. An artificial neuron according to claim 9 wherein the Double Match/Filter identifies carry results for a weight in a dendrite based upon co-occurrence of a weight and a trace.

11. An artificial neuron according to claim 5 further comprising:

a converter that converts the input signal into the first and second trace counts.

12. An artificial neuron according to claim 5 further comprising:

a trace wave propagator that propagates the respective first and second trace counts into the respective first and second power series of weights.

13. An artificial neuron according to claim 5 wherein the function of associations of the input signal at prior dendrites is a sum of associations of the input signal at prior dendrites and wherein the function of associations of the input signal at next dendrites is a sum of associations of the input signal at next dendrites.

14. An artificial neuron according to claim 5 further comprising an accumulator that accumulates matches between the first and second trace counts and the first and second power series of weights to provide a reading operation.

15. An artificial neuron according to claim 14 wherein the accumulator accumulates matches between the first and second trace counts to all of the counts in the first and second power series of weights, regardless of whether carry results are produced.

16. An artificial neuron according to claim 14 further comprising a summer that is responsive to the accumulator to sum results of the accumulations of matches of the first and second trace counts to the first and second power series of weights.

17. An artificial neuron according to claim 4 wherein the function of associations of prior dendrites is a sum of associations of prior dendrites and the function of associations of next dendrites is a sum of associations of next dendrites.

18. An artificial neuron according to claim 4 wherein the pair of first and second power series of weights is a first pair of first and second power series of weights corresponding to a first observation phase of the plurality of inputs and wherein each dendrite further comprises a second pair of first and second power series of weights that correspond to a second observation phase of the plurality of inputs.

19. An artificial neuron according to claim 8 wherein each dendrite further comprises:

a trace that is responsive to an input signal at the associated input, the trace including a first trace count that is a function of associations of the input signal at prior dendrites and a second trace count that is a function of associations of the input signal at next dendrites, the respective first and second power series in the first and second pairs being responsive to the respective first and second trace counts.

20. An artificial neuron according to claim 19 wherein each trace comprises a power series of first trace counts that is a function of associations of the input signal at prior dendrites and a power series of second trace counts that is a function of associations of the input signal at next dendrites.

21. An artificial neuron according to claim 20 further comprising:

means for generating the first and second trace counts from the input signal.

22. An artificial neuron according to claim 20 further comprising:

means for propagating the respective first and second trace counts into the respective first and second pairs of power series of weights.

23. An artificial neuron according to claim 1 wherein the power series of weights is a first power series of weights corresponding to a first observation phase and wherein each dendrite further comprises a second power series of weights corresponding to a second observation phase, each weight in the second power series including a count for the associated power in the second observation phase.

24. An artificial neuron comprising:

a plurality of inputs that are sequentially ordered; and

a plurality of pairs of first and second weights, a respective pair being uniquely associated with a respective one of the plurality of inputs, each first weight being a function of associations of prior inputs of the plurality of sequentially ordered inputs and each second weight being a function of associations of next inputs in the plurality of sequentially ordered inputs.

25. An artificial neuron according to claim 24 wherein the function of associations of prior and next inputs is a statistical function of associations of prior and next inputs.

26. An artificial neuron according to claim 24 wherein the function of associations of prior and next inputs is a sum of associations of prior and next inputs.

27. An artificial neuron according to claim 24 further comprising:

a trace that is responsive to an input signal at the associated input, the trace including a first trace count that is a function of associations of the input signal at prior inputs of the plurality of sequentially ordered inputs and a second trace count that is a function of associations of the input signal at next inputs in the plurality of sequentially ordered inputs, the respective first and second weights being responsive to the respective first and second trace counts.

28. An artificial neuron comprising:

a plurality of inputs that are sequentially ordered; and

a plurality of pairs of first and second weights, a respective pair being uniquely associated with a respective one of the plurality of inputs, each first weight being a first statistical function of the plurality of sequentially ordered inputs and each second weight being a second statistical function of the plurality of sequentially ordered inputs.

29. An artificial neuron according to claim 28 further comprising:

a trace that is responsive to an input signal at the associated input, the trace including a first trace count that is the first statistical function of the input signal and a second trace count that is the second statistical function of the input signal, the respective first and second weights being responsive to the respective first and second trace counts.

30. An artificial neuron comprising:

a plurality of inputs; and

a plurality of power series of weights, a respective power series of weights being uniquely responsive to a respective one of the plurality of inputs, each weight in a power series including an associated count for the associated power of the associated input.

31. An artificial neuron according to claim 30 wherein the plurality of power series of weights is a plurality of base two power series of weights, each weight in a base two power series including an associated count that represents a bit position of the associated count.

32. An artificial neuron according to claim 30 wherein the counts are statistical counts.

33. An artificial neuron according to claim 30 wherein the inputs are sequentially ordered and wherein the plurality of power series of weights comprises first and second power series of weights that are responsive to a respective one of the plurality of inputs, each weight in the first power series including an associated first count that represents a sum of prior inputs for the associated power of the associated input in the sequentially ordered inputs and each weight in the second power series including an associated second count that represents a sum of next inputs in the sequentially ordered inputs for the associated power of the associated input.

34. An artificial neuron according to claim 33 further comprising:

a plurality of traces, a respective one of which is responsive to a respective one of the plurality of inputs, each trace including a first trace count that represents a sum of prior inputs and a second trace count that represents a sum of next inputs, the respective first and second power series being responsive to the respective first and second trace counts.

35. An artificial neuron according to claim 30 wherein the plurality of power series of weights is a plurality of first power series of weights corresponding to a first observation phase and wherein the artificial neuron further comprises a plurality of second power series of weights corresponding to a second observation phase, a respective second power series of weights being responsive to a respective one of the plurality of inputs, each weight in a power series including an associated count for the associated power of the associated input for the second observation phase.

36. A method of memorizing a plurality of inputs into an artificial neuron that includes a plurality of dendrites, a respective one of which is uniquely associated with a respective one of the plurality of inputs, the method comprising the steps of:

providing a power series of weights for each dendrite, each weight in a power series including an associated count for the associated power;

converting the input signal into a power series of trace counts that is a function of associations of the input signal; and

propagating the respective first and second trace counts into the respective power series of weights.

37. A method according to claim 36:

wherein the dendrites are sequentially ordered and wherein the power series of weights comprises a pair of first and second power series of weights, each weight in the first power series including a first count that is a function of associations of prior dendrites and each weight in the second power series including a second count that is a function of associations of next dendrites; and

wherein each trace comprises a power series of first trace counts that is a functions of associations of the input signal at prior dendrites and a power series of second trace counts that is a function of associations of the input signal at next dendrites.

38. A method according to claim 37 wherein the step of propagating comprises the step of propagating the respective first and second trace counts into the respective power series of weights in a forward direction and in a reverse direction.

39. A method according to claim 36 wherein the step of propagating further comprises the step of propagating carry results of the trace along the power series of weights in the plurality of dendrites.

40. A method according to claim 39 wherein the step of propagating carry results comprises the step of double match filtering the trace to identify carry results for a weight in a dendrite, for propagation to a next higher power weight.

41. A method according to claim 40 wherein the step of double match filtering comprises the step of identifying carry results for a weight in a dendrite based upon co-occurrence of a weight and a trace.

42. A method according to claim 36 further comprising the step of accumulating matches between the first and second trace counts and the first and second power series of weights to read the artificial neuron.

43. A method according to claim 42 wherein the accumulating step comprises the step of accumulating matches between the first and second trace counts and all of the counts in the first and second power series of weights, regardless of whether carry results are produced.

44. A method according to claim 42 further comprising the step of summing results of the accumulating step.

45. A method of reading an artificial neuron that includes a plurality of dendrites, the method comprising the steps of:

providing a power series of weights for each dendrite, each weight in a power series including an associated count for the associated power for a plurality of dendrites, a respective one of which is uniquely associated with a respective one of the plurality of inputs;

converting an input signal into a power series of trace counts that is a function of associations of the input signal; and

accumulating matches between the first and second trace counts and the first and second power series of weights.

46. A method according to claim 45 wherein the accumulating step comprises the step of accumulating matches between the first and second trace counts and all of the counts in the first and second power series of weights, regardless of whether carry results are produced.

47. A method according to claim 45 further comprising the step of summing results of the accumulating step.

48. A method according to claim 45:

wherein the dendrites are sequentially ordered and wherein the power series of weights comprises a pair of first and second power series of weights, each weight in the first power series including a first count that is a function of associations of prior dendrites and each weight in the second power series including a second count that is a function of associations of next dendrites; and

wherein each trace comprises a power series of first trace counts that is a function of associations of the input signal at prior dendrites and a power series of second trace counts that is a function of associations of the input signal at next dendrites.

49. A computer program product for providing an artificial neuron, the computer program product comprising a computer-readable storage medium having computer-readable program code embodied in the medium, the computer-readable program code comprising:

computer-readable program code that provides a plurality of inputs; and

computer-readable program code that provides a plurality of dendrites, a respective one of which is uniquely associated with a respective one of the plurality of inputs, each dendrite comprising a power series of weights, each weight in a power series including an associated count for the associated power.

50. A computer program product according to claim 49 wherein the power series of weights is a base two power series of weights, each weight in the base two power series including an associated count that represents a bit position of the associated count.

51. A computer program product according to claim 49 wherein the counts are statistical counts.

52. A computer program product according to claim 49 wherein the dendrites are sequentially ordered and wherein the power series of weights comprises a pair of first and second power series of weights, each weight in the first power series including a first count that is a function of associations of prior dendrites and each weight in the second power series including a second count that is a function of associations of next dendrites.

53. A computer program product according to claim 52 wherein each dendrite further comprises:

a trace that is responsive to an input signal at the associated input, the trace including a first trace count that is a function of associations of the input signal at prior dendrites and a second trace count that is a function of associations of the input signal at next dendrites, the respective first and second power series being responsive to the respective first and second trace counts.

54. A computer program product according to claim 53 wherein each trace comprises a power series of first trace counts that is a function of associations of the input signal at prior dendrites and a power series of second trace counts that is a function of associations of the input signal at next dendrites.

55. A computer program product according to claim 54 wherein the computer-readable program code provides a trace wave propagator that propagates the trace along the sequentially ordered dendrites in a forward direction and in a reverse direction.

56. A computer program product according to claim 54 wherein the computer-readable program code that provides a trace wave propagator that further propagates carry results of the trace along the power series of weights in the plurality of dendrites to provide memorization of the input signal.

57. A computer program product according to claim 56 wherein the computer-readable program code that provides a trace wave propagator that further comprises computer-readable program code that provides a Double Match/Filter that identifies carry results for a weight in a dendrite, for propagation to a next higher power weight.

58. A computer program product according to claim 57 wherein the Double Match/Filter identifies carry results for a weight in a dendrite based upon co-occurrence of a weight and a trace.

59. A computer program product according to claim 53 further comprising:

computer-readable program code that provides a converter that converts the input signal into the first and second trace counts.

60. A computer program product according to claim 53 further comprising:

computer-readable program code that provides a trace wave propagator that propagates the respective first and second trace counts into the respective first and second power series of weights.

61. A computer program product according to claim 53 wherein the function of associations of the input signal at prior dendrites is a sum of associations of the input signal at prior dendrites and wherein the function of associations of the input signal at next dendrites is a sum of associations of the input signal at next dendrites.

62. A computer program product according to claim 53 further comprising computer-readable program code that provides an accumulator that accumulates matches between the first and second trace counts and the first and second power series of weights to provide a reading operation.

63. A computer program product according to claim 62 wherein the computer-readable program code that provides an accumulator that accumulates matches between the first and second trace counts to all of the counts in the first and second power series of weights, regardless of whether carry results are produced.

64. A computer program product according to claim 62 further comprising computer-readable program code that provides a summer that is responsive to the accumulator to sum results of the accumulations of matches of the first and second trace counts to the first and second power series of weights.

65. A computer program product according to claim 52 wherein the function of associations of prior dendrites is a sum of associations of prior dendrites and the function of associations of next dendrites is a sum of associations of next dendrites.

66. A computer program product according to claim 52 wherein the pair of first and second power series of weights is a first pair of first and second power series of weights corresponding to a first observation phase of the plurality of inputs and wherein each dendrite further comprises a second pair of first and second power series of weights that correspond to a second observation phase of the plurality of inputs.

67. A computer program product according to claim 66 wherein each dendrite further comprises:

a trace that is responsive to an input signal at the associated input, the trace including a first trace count that is a function of associations of the input signal at prior dendrites and a second trace count that is a function of associations of the input signal at next dendrites, the respective first and second power series in the first and second pairs being responsive to the respective first and second trace counts.

68. A computer program product according to claim 67 wherein each trace comprises a power series of first trace counts that is a function of associations of the input signal at prior dendrites and a power series of second trace counts that is a function of associations of the input signal at next dendrites.

69. A computer program product according to claim 68 further comprising:

computer-readable program code means for generating the first and second trace counts from the input signal.

70. A computer program product according to claim 68 further comprising:

computer-readable program code means for propagating the respective first and second trace counts into the respective first and second pairs of power series of weights.

71. A computer program product according to claim 49 wherein the power series of weights is a first power series of weights corresponding to a first observation phase and wherein each dendrite further comprises a second power series of weights corresponding to a second observation phase, each weight in the second power series including a count for the associated power in the second observation phase.

72. A computer program product for providing an artificial neuron, the computer program product comprising a computer-readable storage medium having computer-readable program code embodied in the medium, the computer-readable program code comprising:

computer-readable program code that provides a plurality of inputs that are sequentially ordered; and

computer-readable program code that provides a plurality of pairs of first and second weights, a respective pair being uniquely associated with a respective one of the plurality of inputs, each first weight being a function of associations of prior inputs of the plurality of sequentially ordered inputs and each second weight being a function of associations of next inputs in the plurality of sequentially ordered inputs.

73. A computer program product according to claim 72 wherein the function of associations of prior and next inputs is a statistical function of associations of prior and next inputs.

74. A computer program product according to claim 72 wherein the function of associations of prior and next inputs is a sum of associations of prior and next inputs.

75. A computer program product according to claim 72 further comprising:

computer-readable program code that provides a trace that is responsive to an input signal at the associated input, the trace including a first trace count that is a function of associations of the input signal at prior inputs of the plurality of sequentially ordered inputs and a second trace count that is a function of associations of the input signal at next inputs in the plurality of sequentially ordered inputs, the respective first and second weights being responsive to the respective first and second trace counts.

76. A computer program product for providing an artificial neuron, the computer program product comprising a computer-readable storage medium having computer-readable program code embodied in the medium, the computer-readable program code comprising:

computer-readable program code that provides a plurality of inputs that are sequentially ordered; and

computer-readable program code that provides a plurality of pairs of first and second weights, a respective pair being uniquely associated with a respective one of the plurality of inputs, each first weight being a first statistical function of the plurality of sequentially ordered inputs and each second weight being a second statistical function of the plurality of sequentially ordered inputs.

77. A computer program product according to claim 76 further comprising:

computer-readable program code that provides a trace that is responsive to an input signal at the associated input, the trace including a first trace count that is the first statistical function of the input signal and a second trace count that is the second statistical function of the input signal, the respective first and second weights being responsive to the respective first and second trace counts.

78. A computer program product for providing an artificial neuron, the computer program product comprising a computer-readable storage medium having computer-readable program code embodied in the medium, the computer-readable program code comprising:

computer-readable program code that provides a plurality of inputs; and

computer-readable program code that provides a plurality of power series of weights, a respective power series of weights being uniquely responsive to a respective one of the plurality of inputs, each weight in a power series including an associated count for the associated power of the associated input.

79. A computer program product according to claim 78 wherein the plurality of power series of weights is a plurality of base two power series of weights, each weight in a base two power series including an associated count that represents a bit position of the associated count.

80. A computer program product according to claim 78 wherein the counts are statistical counts.

81. A computer program product according to claim 78 wherein the inputs are sequentially ordered and wherein the plurality of power series of weights comprises first and second power series of weights that are responsive to a respective one of the plurality of inputs, each weight in the first power series including an associated first count that represents a sum of prior inputs for the associated power of the associated input in the sequentially ordered inputs and each weight in the second power series including an associated second count that represents a sum of next inputs in the sequentially ordered inputs for the associated power of the associated input.

82. A computer program product according to claim 81 further comprising:

a plurality of traces, a respective one of which is responsive to a respective one of the plurality of inputs, each trace including a first trace count that represents a sum of prior inputs and a second trace count that represents a sum of next inputs, the respective first and second power series being responsive to the respective first and second trace counts.

83. A computer program product according to claim 78 wherein the plurality of power series of weights is a plurality of first power series of weights corresponding to a first observation phase and wherein the artificial neuron further comprises a plurality of second power series of weights corresponding to a second observation phase, a respective second power series of weights being responsive to a respective one of the plurality of inputs, each weight in a power series including an associated count for the associated power of the associated input for the second observation phase.