Learning method for multi-level neural network

• US 5,764,860 A
• Filed: 09/04/1997
• Issued: 06/09/1998
• Est. Priority Date: 03/09/1995
• Status: Expired due to Fees

• Current Assignee: KDDI Corporation
• Original Assignee: Kokusai Denshi Denwa Company Limited (KDDI Corporation)
• Inventors: Yatsuzuka, Youtaro
• Primary Examiner(s): Hafiz, Tariq R.
• Application Number:
  US08/923,333
• Time in Patent Office: 278 Days
• Field of Search: 395/20-25, 395/27, 382/155-161
• US Class Current: 706/25
• CPC Class Codes: G06N 3/063 using electronic means
• CPC Class Codes: G06N 3/08 Learning methods

Abstract

A learning method supervised by a binary teacher signal for a binary neural network comprises at least an error signal generator 10 for weighting factor updating, which generates an error signal for weighting factor updating having an opposite polarity to that of a difference signal between an output unit signal of the binary neural network and the binary teacher signal on an output unit whereat a binary output unit signal coincides with the binary teacher signal, and an amplitude which decreases by increase of distance from the binary teacher signal, when an absolute value of the difference signal is smaller than a threshold, generates an error signal which has the same polarity as that of the difference signal and an amplitude smaller than that of the difference signal, when the absolute value of the difference signal is larger than the threshold, or generates an error signal which has an amplitude equal to or smaller than that of the difference signal on an output unit providing a wrong binary output unit signal which is different from the binary teacher signal. Updating the weighting factors by the error signal which is optimally generated according to discrimination between the correct binary output unit signal and the wrong one, can provide a binary neural network which converges very quickly and reliably to obtain a desired binary output and also realizes a high generalization ability.

Claims

1. A neural network having a learning method supervised by a teacher signal, said neural network comprising:

a neural network having input means for inputting at least one input signal, output means for outputting at least one output unit signal for controlling a device, reach output unit signal being obtained from the input signals at least through weighting factors;

means for generating a first error signal for updating said weighting factors of said neural network, wherein said first error signal has an opposite polarity to that of a difference signal between an output unit signal of said neural network and said teacher signal, and an amplitude which decreases accordance to a distance from said teacher signal, when an absolute value of said difference signal is smaller than a first threshold,

means for generating a second error signal for updating said weighting factors, wherein said second error signal has the same polarity as that of said difference signal and an amplitude smaller than that of said difference signal, when said absolute value of said difference signal is in a range between said first threshold and a second threshold,

means for generating a third error signal for updating said weighting factors, wherein said third error signal has an amplitude equal to or smaller than that of said difference signal, when said absolute value of said difference signal is larger than said second threshold, and

means for updating said weighting factors by using said first, second, and third error signals.

8. A neural network according to one of claims 1 to 6, wherein:

after said neural network has converged once by said training input signal such that all multi-level output unit signals coincide with said multi-level teacher signals, updating said weighting factors by using said difference signal as said error signal is continued until conditions of termination of learning are satisfied.

2. A multi-level neural network having a learning method supervised by a multi-level teacher signal, said multi-level neural network comprising:

a multi-level neural network having input means for inputting at least one input signal, output means for outputting at least one output unit signal for controlling a device each output unit signal being obtained from the input signals at least through weighting factors;

means for generating a first error signal for updating said weighting factors of said multi-level neural network, wherein said first error signal has an opposite polarity to that of a difference signal between an output unit signal of said multi-level neural network and said multi-level teacher signal on an output unit whereat a multi-level output unit signal derived from said output unit signal through multi-level threshold means coincides with said multi-level teacher signal, and an amplitude which decreases according to a distance from said multi-level teacher signal, when an absolute value of said difference signal is smaller than a threshold,

means for generating a second error signal for updating said weighting factors, wherein said second error signal has the same polarity as that of said difference signal on said output unit and an amplitude smaller than that of said difference signal, when said absolute value of said difference signal is larger than said threshold,

means for generating a third error signal for updating said weighting factors, wherein said third error signal has an amplitude equal to or smaller than that of said difference signal on an output unit whereat said multi-level output unit signal is different from said multi-level teacher signal, and

means for updating said weighting factors by using said first, second, and third error signals.

3. A multi-level neural network according to claim 2, wherein:

said weighting factors are updated by using said error signal whereof the amplitude of said error signal is controlled by detection of states trapped in local minima.

4. A multi-level neural network according to claim 3, wherein:

said amplitude of said error signal having said opposite polarity is temporarily increased, when the number of errors in said multi-level output unit signals caused by the difference from said multi-level teacher signal cannot be made to decrease in spite of increase of the number of training cycles.

5. A multi-level neural network according to claim 3, wherein:

said amplitude of said error signal is controlled with detection of states trapped in local minima by comparison of the minimum absolute value of said difference signals among erroneous multi-level output unit signals with a threshold.

6. A multi-level neural network according to claim 5, wherein:

said weighting factors are updated by said error signal whereof the amplitude of said error signal is controlled with detection of states trapped in local minima by comparison of the minimum difference between said output unit signals and a decision level of said multi-level threshold means among correct multi-level output unit signals with a threshold.

7. A multi-level neural network according to one of claims 2 to 6 wherein said neural network further comprising a main neural network and a sub neural network coupled parallel with said main neural network wherein:

said main neural network is trained first by using said multi-value teacher signal,

said sub neural network is trained next by using a compensatory multi-level teacher signal consisting of errors between a multi-level output unit signal of said main neural network derived through said multi-level threshold means and said multi-level teacher signal,

updating operation of said weighting factors for said main neural network is terminated, when the minimum difference between said output unit signals and said decision level of said multi-level threshold means among erroneous multi-level output unit signals of said main neural network, exceeds a predetermined threshold.

9. A multi-level neural network according to one of claims 2 to 6, wherein:

after said multi-level neural networks converged once for said training input signal such that all multi-level output unit signals coincide with said multi-level teacher signals, updating of weighting factors is continued until the minimum difference with said output unit signals and said multi-level threshold of said multi-level threshold means among correct multi-level output unit signals, exceeds a threshold.