Artificial neurons including weights that define maximal projections

• US 7,016,886 B2
• Filed: 08/09/2002
• Issued: 03/21/2006
• Est. Priority Date: 08/10/2001
• Status: Expired due to Fees

• Current Assignee: Intel Corporation
• Original Assignee: Saffron Technology, Inc. (Intel Corporation)
• Inventors: Fleming, James S., Aparicio, Manuel IV, Cabana, David R.
• Primary Examiner(s): Knight, Anthony
• Assistant Examiner(s): ALLEN, NICOLE L
• Application Number:
  US10/216,341
• Publication Number:
  US 20030033265A1
• Time in Patent Office: 1,320 Days
• Field of Search: 706/39, 706/15
• US Class Current: 706/39
• CPC Class Codes: G06N 3/04 Architecture, e.g. interconnection topology
• CPC Class Codes: G06N 3/08 Learning methods

Abstract

An artificial neuron includes inputs and dendrites, a respective one of which is associated with a respective one of the inputs. A respective dendrite includes a respective power series of weights. The weights in a given power of the power series represent a maximal projection. A respective power also may include at least one switch, to identify holes in the projections. By providing maximal projections, linear scaling may be provided for the maximal projections, and quasi-linear scaling may be provided for the artificial neuron, while allowing a lossless compression of the associations. Accordingly, hetero-associative and/or auto-associative recall may be accommodated for large numbers of inputs, without requiring geometric scaling as a function of input.

Claims

1. An artificial neuron comprising:

a plurality of inputs; and

a plurality of dendrites, a respective one of which is associated with a respective one of the plurality of inputs, at least some of the dendrites comprising a power series of weights, at least some of the weights in at least one power of the power series defining a maximal projection.

2. An artificial neuron according to claim 1 wherein at least one power of the power series also includes at least one switch.

3. An artificial neuron according to claim 2 wherein the power series of weights is a base two power series of weights, the at least some of the weights in the at least one power of the power series defining a maximal row projection and/or a maximal column projection.

4. An artificial neuron according to claim 3 wherein the maximal row projection and/or the maximal column projection is sorted by sums of row projections and/or column projections, respectively.

5. An artificial neuron according to claim 3 further comprising:

a trace that is responsive to an input signal at the associated input, the trace being configured to map activations of the input signal to locations in the maximal row projection and/or maximal column projection.

6. An artificial neuron according to claim 5 further comprising:

a router that is configured to map the at least some of the weights in the at least one power of the power series to define the maximal row projection and/or column projection.

7. An artificial neuron according to claim 5 further comprising:

a reader that is configured to determine an activated subset in the maximal row projection and/or maximal column projection in response to the trace and to select, for each column of the activated subset, a smallest row projection that is greater than or equal to an index for the column.

8. An artificial neuron according to claim 7 wherein the reader is further configured to accumulate the smallest row projections that are greater than or equal to the index for the associated column, for the columns of the activated subset, to thereby obtain an association of the input signal with the weights.

9. An artificial neuron according to claim 5 further comprising:

a writer that is configured to embed the trace into the maximal projection and the switch.

10. An artificial neuron according to claim 9 wherein the writer comprises a matrix compressor that is configured to find redundant projections between the trace and the maximal projection.

11. An artificial neuron according to claim 3 further comprising:

a router that is configured to map the at least some of the weights in the at least one power of the power series to define the maximal row projection and/or column projection.

12. An artificial neuron according to claim 1 wherein a respective dendrite comprises a power series of weights, the respective weights in a respective power of the power series defining a maximal projection.

13. An artificial neuron comprising:

a plurality of inputs; and

a plurality of dendrites, a respective one of which is associated with a respective one of the plurality of inputs, at least some of the dendrites including a weight, at least some of the weights defining a maximal projection.

14. An artificial neuron according to claim 13 further comprising at least one switch.

15. An artificial neuron according to claim 14 wherein the maximal projection comprises a maximal row projection and/or a maximal column projection.

16. An artificial neuron according to claim 15 wherein the maximal row projection and/or the maximal column projection is sorted by sums of row projections and/or column projections, respectively.

17. An artificial neuron according to claim 15 further comprising:

a trace that is responsive to an input signal at the associated input, the trace being configured to map activations of the input signal to locations in the maximal row projection and/or maximal column projection.

18. An artificial neuron according to claim 17 further comprising:

a router that is configured to map the at least some of the weights to define the maximal row projection and/or column projection.

19. An artificial neuron according to claim 17 further comprising:

a reader that is configured to determine an activated subset in the maximal row projection and/or maximal column projection in response to the trace and to select, for each column of the activated subset, a smallest row projection that is greater than or equal to an index for the column.

20. An artificial neuron according to claim 19 wherein the reader is further configured to accumulate the smallest row projections that are greater than or equal to an index for the associated column, for the columns of the activated subset, to thereby obtain an association of the input signal with the dendrites.

21. An artificial neuron according to claim 17 further comprising:

a writer that is configured to embed the trace into the maximal projection and the switch.

22. An artificial neuron according to claim 21 wherein the writer comprises a matrix compressor that is configured to find redundant projections between the trace and the maximal projection.

23. An artificial neuron according to claim 15 further comprising:

a router that is configured to map the at least some of the weights to define the maximal row projection and/or column projection.

24. An artificial neuron according to claim 13 wherein a respective dendrite includes a respective weight, the respective weights defining a maximal projection.

25. A method of operating an artificial neuron that includes a plurality of inputs and a plurality of dendrites, a respective one of which is associated with a respective one of the plurality of inputs, the method comprising:

generating a power series of weights for at least some of the dendrites, at least some of the weights in at least one power of the power series defining a maximal projection.

26. A method according to claim 25 wherein the generating further comprises generating a switch for at least one power of the power series.

27. A method according to claim 26 wherein the power series of weights is a base two power series of weights and wherein the generating further comprises generating a maximal row projection and/or a maximal column projection for the at least one power of the power series.

28. A method according to claim 27 wherein the generating further comprises generating the maximal row projection and/or the maximal column projection by sorting by sums of row projections and/or column projections, respectively.

29. A method according to claim 27 further comprising:

mapping activations of the plurality of inputs to locations in the maximal row projection and/or maximal column projection.

30. A method according to claim 29 further comprising:

mapping the at least some of the weights in the at least one power of the power series to define the maximal row projection and/or column projection.

31. A method according to claim 29 further comprising:

determining an activated subset in the maximal row projection and/or maximal column projection in response to the mapping; and

selecting, for each column of the activated subset, a smallest row projection that is greater than or equal to an index for the column.

32. A method according to claim 31 further comprising:

accumulating the smallest row projections that are greater than or equal to the index for the associated column, for the columns of the activated subset, to thereby obtain an association of the plurality of inputs with the weights.

33. A method according to claim 29 further comprising:

embedding the activations that are mapped into the maximal projection and the switch.

34. A method according to claim 33 wherein the embedding comprises:

finding redundant projections between the activations that are mapped and the maximal projection.

35. A method according to claim 27 further comprising:

mapping the at least some of the weights in the at least one power of the power series to define the maximal row projection and/or column projection.

36. A method according to claim 25 wherein the generating comprises:

generating a respective power series of weights for a respective dendrite, the respective weights in a respective power of the power series defining a maximal projection.

37. A method of operating an artificial neuron that includes a plurality of inputs and a plurality of dendrites, a respective one of which is associated with a respective one of the plurality of inputs, the method comprising:

generating a maximal projection for at least some of the dendrites.

38. A method according to claim 37 wherein the generating further comprises generating a switch for the dendrites.

39. A method according to claim 38 wherein the generating further comprises generating a maximal row projection and/or a maximal column projection for the at least some of the dendrites.

40. A method according to claim 39 wherein the generating further comprises generating the maximal row projection and/or the maximal column projection by sorting by sums of row projections and/or column projections, respectively.

41. A method according to claim 39 further comprising:

mapping activations of the plurality of inputs to locations in the maximal row projection and/or maximal column projection.

42. A method according to claim 41 further comprising:

mapping at least some the projections to define the maximal row projection and/or column projection.

43. A method according to claim 41 further comprising:

determining an activated subset in the maximal row projection and/or maximal column projection in response to the mapping; and

selecting, for each column of the activated subset, a smallest row projection that is greater than or equal to an index for the column.

44. A method according to claim 43 further comprising accumulating the smallest row projections that are greater than or equal to an index for the associated column, for the columns of the activated subset, to thereby obtain an association of the input signal with the dendrites.

45. A method according to claim 41 further comprising:

embedding the activations that are mapped into the maximal projection and the switch.

46. A method according to claim 45 wherein the embedding comprises:

finding redundant projections between the activations that are mapped and the maximal projection.

47. A method according to claim 39 further comprising:

mapping at least some the projections to define the maximal row projection and/or column projection.

48. A method according to claim 37 wherein the generating comprises:

generating a maximal projection for the plurality of dendrites.

49. A computer program product for providing an artificial neuron, the computer program product comprising a computer-readable storage medium having computer-readable program code embodied in the medium, the computer-readable program code comprising:

computer-readable program code that is configured to provide a plurality of inputs; and

computer-readable program code that is configured to provide a plurality of dendrites, a respective one of which is associated with a respective one of the plurality of inputs, at least some of the dendrites comprising a power series of weights, at least some of the weights in at least one power of the power series defining a maximal projection.

50. A computer program product according to claim 49 wherein at least one power of the power series also includes at least one switch.

51. A computer program product according to claim 50 further comprising:

computer-readable program code that is configured to provide a router that is configured to map the at least some of the weights in the at least one power of the power series to define the maximal row projection and/or column projection.

52. A computer program product according to claim 50 wherein the power series of weights is a base two power series of weights, the at least one of the weights in the at least one power of the power series defining a maximal row projection and/or a maximal column projection.

53. A computer program product according to claim 52 further comprising:

computer-readable program code that is configured to provide a trace that is responsive to an input signal at the associated input, the trace being configured to map activations of the input signal to locations in the plurality of maximal row projection and/or maximal column projection.

54. A computer program product according to claim 52 wherein the maximal row projection and/or the maximal column projection is sorted by sums of row projections and/or column projections, respectively.

55. A computer program product according to claim 54 further comprising:

computer-readable program code that is configured to provide a router that is configured to map the at least some of the weights in the at least one power of the power series to define the maximal row projection and/or column projection.

56. A computer program product according to claim 54 further comprising:

computer-readable program code that is configured to provide a reader that is configured to determine an activated subset in the maximal row projection and/or maximal column projection in response to the trace and to select, for each column of the activated subset, a smallest row projection that is greater than or equal to an index for the column.

57. A computer program product according to claim 56 wherein the computer-readable program code that is configured to provide a reader is further configured to accumulate the smallest row projections that are greater than or equal to the index for the associated column, for the columns of the activated subset, to thereby obtain an association of the input signal with the weights.

58. A computer program product according to claim 54 further comprising:

computer-readable program code that is configured to provide a writer that is configured to embed the trace into the maximal projection and the switch.

59. A computer program product according to claim 58 wherein the computer-readable program code that is configured to provide a writer comprises computer-readable program code that is configured to provide a matrix compressor that is configured to find redundant projections between the trace and the maximal projections.

60. A computer program product according to claim 49 wherein the computer-readable program code that is configured to provide a plurality of dendrites comprises computer-readable program code that is configured to provide a plurality of dendrites, a respective one of which is associated with a respective one of the plurality of inputs, a respective dendrite comprising a power series of weights, the respective weights in a respective power of the power series defining a maximal projection.

61. A computer program product for providing an artificial neuron, the computer program product comprising a computer-readable storage medium having computer-readable program code embodied in the medium, the computer-readable program code comprising:

computer-readable program code that is configured to provide a plurality of inputs; and

computer-readable program code that is configured to provide a plurality of dendrites, a respective one of which is associated with a respective one of the plurality of inputs, at least some of the dendrites including a weight, at least some of the weights defining a maximal projection.

62. A computer program product according to claim 61 further comprising:

computer-readable program code that is configured to provide at least one switch.

63. A computer program product according to claim 62 wherein the maximal projection comprises a maximal row projection and/or a maximal column projection.

64. A computer program product according to claim 63 wherein the maximal row projection and/or the maximal column projection is sorted by sums of row projections and/or column projections, respectively.

65. A computer program product according to claim 63 further comprising:

computer-readable program code that is configured to provide a trace that is responsive to an input signal at the associated input, the trace being configured to map activations of the input signal to locations in the maximal row projection and/or maximal column projection.

66. A computer program product according to claim 65 further comprising:

computer-readable program code that is configured to provide a router that is configured to map the at least some of the weights to define the maximal row projection and/or column projection.

67. A computer program product according to claim 65 further comprising:

computer-readable program code that is configured to provide a reader that is configured to determine an activated subset in the maximal row projection and/or maximal column projection in response to the trace and to select, for each column of the activated subset, a smallest row projection that is greater than or equal to an index for the column.

68. A computer program product according to claim 67 wherein the computer-readable program code that is configured to provide a reader is further configured to accumulate the smallest row projections that are greater than or equal to an index for the associated column, for the columns of the activated subset, to thereby obtain an association of the input signal with the dendrites.

69. A computer program product according to claim 65 further comprising:

computer-readable program code that is configured to provide a writer that is configured to embed the trace into the maximal projection and the switch.

70. A computer program product according to claim 69 wherein the computer-readable program code that is configured to provide a writer comprises computer-readable program code that is configured to provide a matrix compressor that is configured to find redundant projections between the trace and the maximal projection.

71. A computer program product according to claim 63 further comprising:

computer-readable program code that is configured to provide a router that is configured to map the at least some of the weights to define the maximal row projection and/or column projection.

72. A computer program product according to claim 61 wherein the computer-readable code that is configured to provide a plurality of dendrites comprises computer-readable program code that is configured to provide a plurality of dendrites, a respective one of which is associated with a respective one of the plurality of inputs, a respective one of the dendrites including a respective weight, the respective weights defining a maximal projection.