Neuron unit

• US 5,671,337 A
• Filed: 12/21/1993
• Issued: 09/23/1997
• Est. Priority Date: 12/24/1992
• Status: Expired due to Term

• Current Assignee: Olympus Optical Corporation Limited (Olympus Corporation)
• Original Assignee: Olympus Optical Corporation Limited (Olympus Corporation)
• Inventors: Yoshihara, Takafumi
• Primary Examiner(s): Davis, George B.
• Application Number:
  US08/171,013
• Time in Patent Office: 1,372 Days
• Field of Search: 395/27, 395/24, 395/23, 395/22
• US Class Current: 706/43
• CPC Class Codes: G06N 3/063 using electronic means
• CPC Class Codes: G06N 3/08 Learning methods

Abstract

A neuron unit comprises a synapse section in which input signals from external input means or preceding neuron units and synaptic weights are multiplied, and a neuron section in which output signals of the synapse section are added, subjected to thresholding, subjected to linear or nonlinear conversion, and then outputted to the outside. According to an aspect of the present invention, a synaptic weight used in the synapse section and/or a threshold used in the neuron section, which corresponds in magnitude to an external input, is obtained through interpolation from synaptic weights which have been learned previously, varying in magnitude according to external inputs. According to the other aspect of the present invention, the synaptic weight and/or the threshold is obtained in accordance with a previously set function of an external input.

Claims

1. A neuron unit comprising:

a synapse section including;

synaptic weight generating means for generating a synaptic weight corresponding to the magnitude of an input signal, and

multiplier means for multiplying said input signal and said synaptic weight, and

a neuron section including;

first adder means for adding output signals of said multiplier means together,

threshold value generating means for generating a threshold value,

second adder means for adding an output of said threshold value generating means to an output of said first adder means, and

conversion means for subjecting an output of said second adder means to a predetermined conversion.

2. A neuron unit according to claim 1, in which

said synaptic weight generating means includes:

storage means for storing synaptic weights each corresponding to the magnitude of a respective one of predetermined learning input signals; and

interpolation means for interpolating a synaptic weight, corresponding to said input signal, from said synaptic weights stored in said storage means.

3. A neuron unit according to claim 2, in which

said interpolation means performs polynomial interpolation based on said synaptic weights stored in said storage means.

4. A neuron unit according to claim 2, in which said synaptic weight generating means further includes quantization means for quantizing a magnitude of said input signal in a variable quantization width.

5. A neuron unit according to claim 4, further comprising:

random number generating means for generating predetermined random numbers; and

learning means for setting initial values of synaptic weights and a threshold value on the basis of random numbers generated by said random number generating means, correcting said synaptic weights and said threshold value on the basis of externally input target data, and storing said synaptic weights and threshold value into said storage means.

6. A neuron unit according to claim 4, further comprising:

random number generating means for generating predetermined random numbers; and

learning means for setting initial values of synaptic weights and coefficients of said predetermined functions on the basis of random numbers generated by said random number generating means, correcting coefficients and said synaptic weights stored in said storage means on the basis of externally input target data, and storing said synaptic weights and said coefficients into said storage means.

7. A neuron unit according to claim 6, in which said learning means performs a learning process comprising the steps of:

(a) initializing a number of times learning is performed, fixed coefficients of coefficients of said predetermined functions and a number of said fixed coefficients;

(b) performing predetermined learning using coefficients other than said fixed coefficients while updating the number of times said learning is performed until an error function value of a result of said learning falls within an allowable error range;

(c) checking the number of said fixed coefficients while comparing a degree of decrease of error with a preset value until the degree of decrease of error falls to within said preset value, when said error function value of the result of said learning is outside said allowable error range;

repeating steps (a) to (c) when said checking step indicates all of said fixed coefficients have been used already;

learning enabling a predetermined number of said unused fixed coefficients, when unused fixed coefficients are found in said checking step; and

decrementing the number of said fixed coefficients by said predetermined number.

8. A neuron unit according to claim 2, further comprising delay means for delaying input signals from external input means or preceding neuron units by a predetermined time, and in which said storage means stores synaptic weights corresponding to said predetermined learning input signals Which are input to said delay means and predetermined learning outputs of said delay means corresponding to said predetermined learning input signals, and said interpolation means interpolates synaptic weights corresponding to said input signals and output signals from said delay means by interpolation from said synaptic weights stored in said storage means.

9. A neuron unit according to claim 2, further comprising delay means for delaying a previous output signal of said neuron section by a predetermined time and feeding said previous output signal back to said function processing means when next input signals are input.

10. A neuron unit according to claim 1, in which

said threshold value generating means includes:

storage means for storing threshold values each corresponding to a respective one of predetermined output values of said first adder means; and

interpolation means for interpolating a threshold value, corresponding to an output of said first adder means, from said threshold values stored in said storage means.

11. A neuron unit according to claim 10, in which

said interpolation means performs polynomial interpolation based on said threshold values stored in said storage means.

12. A neuron unit according to claim 10, in which said threshold generating means further includes quantization means for quantizing a magnitude of an output signal of said first adder means in a variable quantization width.

13. A neuron unit according to claim 12, further comprising:

random number generating means for generating predetermined random numbers; and

learning means for setting initial values of synaptic weights and a threshold value on the basis of random numbers generated by said random number generating means, correcting said synaptic weights and said threshold value on the basis of externally input target data, and storing said synaptic weights and threshold value into said storage means.

14. A neuron unit according to claim 1, in which

said synaptic weight generating means includes:

storage means for storing predetermined functions and coefficients of said predetermined functions each defined for a respective one of said multiplier means; and

function calculating means responsive to said functions and said coefficients stored in said storage means for obtaining a synaptic weight as a function value of said input signal.

15. A neuron unit according to claim 14, further comprising delay means for delaying input signals from external input means or preceding neuron units by a predetermined time, and in which said function calculating means obtains synaptic weights as function values of said input signals and output signals from said delay means.

16. A neuron unit according to claim 14, further comprising delay means for delaying a previous output signal of said neuron section by a predetermined time and feeding said previous output signal back to said function processing means when next input signals are input.

17. A neuron unit according to claim 14, further comprising:

random number generating means for generating predetermined random numbers; and

learning means for setting initial values of coefficients of said predetermined functions and a threshold value on the basis of random numbers generated by said random number generating means, correcting coefficients from said coefficient storage means and said threshold values on the basis of externally input target data, and storing said coefficients and a threshold value into said storage means.

18. A neuron unit according to claim 17, in which said learning means performs a learning process comprising the steps of:

(a) initializing a number of times learning is performed, fixed coefficients of coefficients of said predetermined functions and a number of said fixed coefficients;

(b) performing predetermined learning using coefficients other than said fixed coefficients while updating the number of times said learning is performed until an error function value of a result of said learning falls within an allowable error range;

(c) checking the number of said fixed coefficients while comparing a degree of decrease of error with a preset value until the degree of decrease of error falls to within said preset value, when said error function value of the result of said learning is outside said allowable error range;

repeating steps (a) to (c) when said checking step indicates all of said fixed coefficients have been used already;

learning enabling a predetermined number of said unused fixed coefficients, when unused fixed coefficients are found in said checking step; and

decrementing the number of said fixed coefficients by said predetermined number.

19. A neuron unit according to claim 1, in which

said threshold value generating means includes:

storage means for storing predetermined functions and coefficients of said predetermined functions; and

function calculating means responsive to said predetermined functions and said coefficients stored in said storage means for obtaining a threshold value as a function value for an output of said first adder means.

20. A neuron unit comprising:

a synapse section including;

storage means for storing, in advance, a plurality of synaptic weight groups, each including combinations of a plurality of synaptic weights that take discrete values resulting from learning ;

interpolation means for reading out from said storage means one of said plurality of synaptic weight groups which corresponds to an input signal received from an external input means or a preceding neuron unit, and interpolating said one of the synaptic weight groups in order to obtain an interpolated value corresponding to said input signal, to thereby obtain a new synaptic weight;

multiplier means for multiplying said synaptic weights obtained by said interpolation means and said input signal; and

a neuron section including;

first adder means for receiving output signals from said multiplier means and for adding up said output signals;

threshold value generating means for generating a threshold value;

second adder means for adding said threshold value to an output of said first adder means; and

conversion means for subjecting an output of said second adder means to a predetermined conversion.

21. A neuron unit according to claim 20, wherein:

said synapse section has a plurality of multiplier means corresponding to a plurality of input signals from said external input means or said preceding neuron unit,

said interpolation means is connected so that said interpolated new synaptic weight is supplied to each of said multiplier means, and

said plurality of input signals are input to said interpolation means.

22. A neuron unit according to claim 20, wherein:

said synapse section includes one of each of said storage means, said interpolation means and said multiplier means,

a plurality of said input signals are input to said interpolation means and said multiplier means in time sequence order; and

said first adder means receives output signals from said multiplier means, and adds up said output signals.

23. A neuron unit according to claim 22, wherein said first adder means includes count means for:

counting a number of add-up operations carried out for the output signals from said multiplier means in order,

instructing said first adder means to complete the add-up operations when said counted number of add-up operations reaches a predetermined value, and

instructing said second adder means to add up threshold values from said threshold value generating means.

24. A neuron unit according to claim 20, wherein said threshold value generating means includes:

storage means for storing threshold values, each corresponding to a respective one of predetermined learning output values of said first adder means; and

interpolation means for interpolating a threshold value, corresponding to an output of said first adder means, from said threshold values stored in said storage means.

25. A neuron unit according to claim 24, wherein said interpolation means performs polynomial interpolation based on said threshold values stored in said storage means.

26. A neuron unit according to claim 24, wherein said threshold generating means further includes quantization means for quantizing a magnitude of an output signal of said first adder means in a variable quantization width.

27. A neuron unit according to claim 26, further comprising:

random number generating means for generating predetermined random numbers; and

learning means for setting initial values of synaptic weights and a threshold value on the basis of random numbers generated by said random number generating means, correcting said synaptic weights and said threshold value on the basis of externally input target data, and storing said synaptic weights and a threshold value into said storage means.

28. A neuron unit according to claim 20, wherein said threshold value generating means includes:

storage means for storing predetermined functions and coefficients of said predetermined functions; and

function calculating means responsive to said predetermined functions and said coefficients stored in said storage means for obtaining a threshold value as a function value for an output of said first adder means.

29. A neuron unit according to claim 28, further comprising:

random number generating means for generating predetermined random numbers; and

learning means for setting initial values of synaptic weights and coefficients of said predetermined functions on the basis of random numbers generated by said random number generating means, correcting coefficients and said synaptic weights stored in said storage means on the basis of externally input target data, and storing said synaptic weights and said coefficients into said storage means.

30. A neuron unit according to claim 29, wherein said learning means performs a learning process comprising the steps of:

(a) initializing a number of times learning is performed, fixed coefficients of coefficients of said predetermined functions and a number of said fixed coefficients;

(b) performing predetermined learning using coefficients other than said fixed coefficients while updating the number of times said learning is performed until an error function value of a result of said learning falls within an allowable error range;

(c) checking the number of said fixed coefficients while comparing a degree of decrease of error with a preset value until the degree of decrease of error falls to within said preset value, when said error function value of the result of said learning is outside said allowable error range;

repeating steps (a) to (c) when said checking step indicates all of said fixed coefficients have been used already;

learning enabling a predetermined number of said unused fixed coefficients, when unused fixed coefficients are found in said checking step; and

decrementing the number of said fixed coefficients by said predetermined number.

31. A neuron unit according to claim 20, wherein said interpolation means performs polynomial interpolation based on said synaptic weights stored in said storage means.

32. A neuron unit according to claim 20, wherein said synapse section further includes quantization means for quantizing a magnitude of said input signal in a variable quantization width.

33. A neuron unit according to claim 32, further comprising:

random number generating means for generating predetermined random numbers; and

learning means for setting initial values of synaptic weights and a threshold value on the basis of random numbers generated by said random number generating means, correcting said synaptic weights and said threshold value on the basis of externally input target data, and storing said synaptic weights and a threshold value into said storage means.

34. A neuron unit according to claim 20,

further comprising delay means for delaying input signals from external input means or preceding neuron units by a predetermined time, wherein

said storage means stores synaptic weights corresponding to predetermined learning input signals and predetermined learning outputs, and

said interpolation means interpolates synaptic weights corresponding to said input signals and output signals from said delay means by interpolation from said synaptic weights stored in said storage means.

35. A neuron unit according to claim 20, further comprising delay means for delaying a previous output signal of said neuron section by a predetermined time and feeding the previous output signal back to said interpolation means when next input signals are input.