Artificial synapse component using multiple distinct learning means with distinct predetermined learning acquisition times

• US 7,904,398 B1
• Filed: 03/22/2007
• Issued: 03/08/2011
• Est. Priority Date: 10/26/2005
• Status: Active Grant

• Current Assignee: Dominic John Repici
• Original Assignee: Dominic John Repici
• Inventors: Repici, Dominic John
• Primary Examiner(s): Holmes; Michael B.
• Assistant Examiner(s): Brown, Jr.; Nathan H
• Application Number:
  US11/689,676
• Time in Patent Office: 1,447 Days
• Field of Search: 706/15
• US Class Current: 706/15
• CPC Class Codes: G06N 3/063 using electronic means
• CPC Class Codes: G06N 3/08 Learning methods

Abstract

Neuron component and method for use in artificial neural networks (ANNs) with input synapses (204, 204b . . . 204n), each synapse includes multiple weights called synapse weights (206-1, 206-2, 206-3). Each synapse further includes a facility to modulate, or gate, an input signal connected to the synapses, by each of the respective synapse weights within the synapse, supplying the result of each modulating operation. The neuron also sums the results of all modulating operations, and subjects the results to a transfer function. Each of the multiple weights associated with a given synapse, may be specified to have its own weight-adjustment facility (214, 214b, 214c), with its own error-values (216, 216b, 216c), and its own specified learning and aspect (1000) includes a separate sum (1018, 1018b) and transfer function (1020, 1020b) for each synapse weight.

Claims

1. An artificial neuron component, comprising:

(a.) at least one artificial synapse, comprising;

(1.) a connection means that connects an input signal to be conveyed to said artificial synapse, and

(2.) a plurality of means for storing or supplying a plurality of respective synapse weights or synapse weight values, and

(3.) a synapse modulating means for modulating said input signal by each of said plurality of respective synapse weights or synapse weight values, thereby to produce a plurality of respective results determined by modulating said input signal by said plurality of respective synapse weights or synapse weight values,

(b.) a summing means for accumulating said plurality of respective results of said synapse modulating means as an internal sum, and

(c.) a transfer conversion means for subjecting said internal sum accumulated by said summing means to a predetermined transfer function to provide a resultant output value,

(d.) a plurality of weight adjustment means, each comprising a predetermined learning means for adjusting a respective one of the synapse weight values within said artificial synapse, and for correcting the value of said respective one of the synapse weight values within said artificial synapse as a function of said predetermined learning means,

whereby each of said plurality of weight adjustment means will train each of said plurality of respective synapse weights or synapse weight values using a distinct predetermined learning means, and a distinct predetermined learning rate, providing each of said plurality of respective synapse weights or synapse weight values with a distinct predetermined learning acquisition time, and the output of said transfer conversion means is a representation of said input signal, modulated by each of said plurality of respective synapse weights or synapse weight values within said artificial synapse.

2. The artificial neuron component of claim 1, further including a predetermined forget means for adjusting the value of at least one of said plurality of respective synapse weights or synapse weight values within said artificial synapse closer to zero at a predetermined rate and according to a predetermined set of criteria,

whereby said predetermined forget means will reduce at least one of said plurality of respective synapse weights or synapse weight values according to said predetermined rate, providing each synapse weight with a distinct, predetermined learning retention time.

3. The artificial neuron component of claim 1 wherein at least one of said plurality of weight adjustment means includes a weight-to-weight learning means, comprising:

(a.) a weight error value calculation means for providing a weight-error value proportional to the difference between a respective one of said plurality of respective synapse weights or synapse weight values being adjusted and at least one other of said plurality of respective synapse weights or synapse weight values in said artificial synapse, and

(b.) a means for using a predetermined proportion of said weight-error value for altering the value of the respective one of synapse weights associated with said respective weight adjustment means, so as to bring the value of the respective synapse weight closer to the value of at least one other weight values used to produce said weight-error value,

whereby said weight-to-weight learning means will adjust at least one of said plurality of respective synapse weights or synapse weight values of said artificial synapse, using the value of at least one other of the plurality of weight values within said artificial synapse as a training value for the respective one of said synapse weights associated with the weight adjustment means.

4. The artificial neuron component of claim 3, further including a predetermined forget means for adjusting the value of at least one of said plurality of respective synapse weights or synapse weight values within said artificial synapse closer to zero at a predetermined rate and according to a predetermined set of criteria,

whereby said forget means will reduce at least one of said plurality of respective synapse weights or synapse weight values according to said predetermined rate, providing each synapse weight with a distinct, predetermined learning retention time.

5. The artificial neuron component of claim 3, further comprising a separate weight sum means for each of said plurality of respective synapse weights or synapse weight values, each of said separate weight sum means being arranged to accumulate the results of said synapse modulating means for one of said plurality of respective synapse weights or synapse weight values into a weight sum for the respective synapse weight, and a transfer conversion means for subjecting each of the weight sums accumulated by said separate weight sum means to a predetermined transfer function, producing a resultant value,

whereby a plurality of separate weight sums and results of said transfer conversion means are produced for each of said plurality of respective synapse weights or synapse weight values and made available to said plurality of weight adjustment means.

6. The artificial neuron component of claim 5, further including a predetermined forget means for adjusting one of said plurality of respective synapse weights or synapse weight values for said artificial synapse in said artificial neuron component, and for adjusting the value of the synapse weight within each artificial synapse closer to zero at a predetermined rate and according to a predetermined set of criteria,

whereby said predetermined forget means will reduce each of said plurality of respective synapse weights or synapse weight values according to a predetermined forget rate, providing each of said plurality of respective synapse weights or synapse weight values with a distinct, predetermined learning retention time.

7. The artificial neuron component of claim 1, further comprising a separate weight sum means for each of said plurality of respective synapse weights or synapse weight values, each of said separate weight sum means being arranged to accumulate the results of said synapse modulating means for one of said plurality of respective synapse weights or synapse weight values in said artificial synapse into a weight sum for the respective synapse weight, and a transfer conversion means for subjecting each of the weight sums accumulated by said weight sum means to a predetermined transfer function, producing a resultant value,

whereby a plurality of separate weight sums and results of said transfer conversion means are produced for each of said plurality of respective synapse weights or synapse weight values and made available to said plurality of weight adjustment means.

8. The artificial neuron component of claim 7, further including a predetermined forget means for adjusting one of said plurality of respective synapse weights or synapse weight values for said at least one artificial synapse in said artificial neuron component, and for adjusting the value of the synapse weight within each artificial synapse closer to zero at a predetermined rate and according to a predetermined set of criteria,

whereby said predetermined forget means will reduce each of said plurality of respective synapse weights or synapse weight values according to a predetermined forget rate, providing each of said plurality of respective synapse weights or synapse weight values with a distinct, predetermined learning retention time.

9. A method of emulating a neuron for use in artificial neural networks comprising:

(a.) providing at least one synapse means, comprising;

(1.) a plurality of means for storing a plurality of respective synapse weights or weight values, and

(2.) a synapse modulating means for respectively modulating an input signal by each of said plurality of respective synapse weights or weight values, to produce a plurality of respective results,

(b.) accumulating the results of said synapse modulating means for each of said plurality of respective synapse weights or weight values in said synapse means into an internal sum, and

(c.) subjecting said internal sum to a predetermined transfer function to provide a resultant output,

(d.) providing a plurality of weight-adjustment means each comprising a predetermined learning means, for adjusting a respective one of said plurality of respective synapse weights or weight values within said at least one synapse means, and for correcting the value of said respective one of said plurality of respective synapse weights or weight values within said at least one synapse means as a function of said predetermined learning means,

whereby said resultant output is a representation of at least one input signal, individually modulated by each of said plurality of respective synapse weights or weight values comprising each of said at least one synapse means, and said weight-adjustment means will train each of said synapse weights using a distinct predetermined learning means, and a distinct predetermined learning rate, providing each of said plurality of respective synapse weights or weight values with a distinct, predetermined learning acquisition time.

10. The neuron method of claim 9, further including adjusting said plurality of respective synapse weights or weight values within each synapse means closer to zero at a predetermined rate and according to a predetermined set of criteria,

whereby at least one of said plurality of respective synapse weights or weight values will be moved closer to zero according to a predetermined forget rate, providing each synapse weight with a distinct, predetermined learning retention time.

11. The neuron method of claim 9 wherein at least one of said plurality of weight-adjustment means includes a weight-to-weight learning means, comprising:

(a.) calculating a weight-error value proportional to the difference between said respective one of said plurality of respective synapse weights or weight values being adjusted and at least one other of said plurality of respective synapse weights or weight values in said synapse means, and

(b.) using a predetermined proportion of said weight-error value for altering the value of said respective one of said plurality of respective synapse weights or weight values associated with said respective weight-adjustment means, so as to bring it closer to the value of at least one other of said plurality of respective synapse weights or weight values used to produce said weight-error value,

whereby said weight-to-weight learning means will adjust at least one of said plurality of weights of said synapse means, using the value of at least one other of the plurality of synapse weights within said at least one synapse means as a training value for the respective one of said synapse weights associated with the weight adjustment means.

12. The neuron method of claim 11, further including providing a predetermined forget means adjusting one of said plurality of respective synapse weights or weight values for said at least one synapse means, and for adjusting the value of the synapse weight within each synapse means closer to zero at a predetermined rate and according to a predetermined set of criteria,

whereby said predetermined forget means will reduce each of said plurality of respective synapse weights or weight value according to a predetermined forget rate, providing each synapse weight with a distinct, predetermined learning retention time.

13. The neuron method of claim, 11 further including a separate weight sum means for each of said plurality of respective synapse weights or weight values, said weight sum means being arranged to:

(a.) accumulate the results of said respective synapse modulating means for one of said plurality of respective synapse weights or weight values in each of said at least one synapse means into a weight sum for each of said plurality of respective synapse weights or weight values, and

(b.) subject each of said weight sum accumulated to a predetermined transfer function, producing a resultant value,

whereby a plurality of separate weight sums and results of said transfer conversion means are produced for each of said plurality of respective synapse weights or weight values and made available to said weight-adjustment means.

14. The neuron method of claim 13, further including providing a predetermined forget process for adjusting one of said plurality of respective synapse weights or weight values for all the synapse means in the neuron, and for adjusting the value of the synapse weight within said at least one synapse means closer to zero at a predetermined rate and according to a predetermined set of criteria,

whereby said predetermined forget process will reduce each of said plurality of respective synapse weights or weight values according to a predetermined forget rate, providing each synapse weight with a distinct, predetermined, learning retention time.

15. The neuron method of claim 9, further including a separate weight sum means for each of said synapse weights, said weight sum means being arranged to:

(a.) accumulate the results of said respective synapse modulating means for one of said plurality of respective synapse weights or weight values in each of said at least one synapse means into a weight sum for the respective synapse weight, and

(b.) subject each of said weight sum accumulated by said weight sum means to a predetermined transfer function, producing a resultant value,

whereby a plurality of separate weight sums and results of said transfer conversion means are produced for each of said plurality of respective synapse weights or weight values and made available for use by said weight adjustment means.

16. The neuron method of claim 15, further including providing a predetermined forget process for adjusting one of said plurality of respective synapse weights or weight values, for said at least one synapse means, and for adjusting the value of the synapse weight within said at least one synapse closer to zero at a predetermined rate and according to a predetermined set of criteria,

whereby said predetermined forget process will reduce each of said plurality of respective synapse weights or weight values according to a predetermined forget rate, providing each synapse weight with a distinct, predetermined, learning retention time.