Method of enhancing the performance of a neural network

• US 5,561,741 A
• Filed: 07/22/1994
• Issued: 10/01/1996
• Est. Priority Date: 07/22/1994
• Status: Expired due to Term

• Current Assignee: Unisys Corporation
• Original Assignee: Unisys Corporation
• Inventors: Wasserman, Philip D.
• Primary Examiner(s): Davis, George B.
• Application Number:
  US08/279,365
• Time in Patent Office: 802 Days
• Field of Search: 395/23, 395/24, 395/22
• US Class Current: 706/25
• CPC Class Codes: G06N 3/08 Learning methods

Abstract

A method for enhancing the performance of an artificially intelligent system employing a neural network by proving an optimized training set for training the neural network. The optimized training set is produced by identifying and removing inaccurate training pairs in the training set.

Claims

1. A method for optimizing a training set comprised of a large plurality of training pairs for use in training a neural network, said method comprising the steps of:

a) training a neural network using said training set;

b) measuring a generalization error of the trained neural network;

c) removing a training pair from the training set to produce a reduced training set;

d) training the neural network using the reduced training set;

e) measuring a generalization error of the trained reduced training set;

f) reinserting the removed training pair into the training set when the measured generalization error for the reduced set is not less than that obtained for a non-reduced training set;

g) permanently removing the removed training pair from the training set when the measured generalization error for the reduced set is less than that obtained for the non-reduced training set; and

h) performing steps c), d), e), f) and g) above for each non-reduced training pair in the training set.

2. The method of claim 1, wherein the step (a) employs a supervised training approach.

3. The method of claim 1, wherein a training pair comprises an input vector and a target vector, said input vector comprising a set of input characteristics and said target vector corresponding to a particular set of input characteristics.

4. The method of claim 3, wherein each target vector provides an output indicating the probability of correspondence to an input vector.

5. The method of claim 1, wherein training sets are produced based on human observations.

6. The method of claim 1, wherein a generalized regression neural network is used for the training steps.

7. The method of claim 1, wherein the steps of measuring a generalization error are accomplished using cross-validation based on an exclude one approach.

8. The method of claim 1, wherein measuring a generalization error is performed using the following steps:

(aa) removing a training pair from the training set whose generalization error is being measured, a training pair comprising an input vector I_k and a target vector T_k ;

(bb) training the neural network using the reduced training set obtained after removing training pair k;

(cc) applying the input vector I_k to the neural network for producing an output O_k corresponding thereto;

(dd) calculating an Error_k =∥

T_k -Out_k ∥

for the removed training pair k;

(ee) returning the training pair k to the training set;

(ff) performing steps (aa), (bb), (cc), (dd) and (ee) above for each of the other training pairs in the training set whose generalization error is being measured to obtain a corresponding Error_k value; and

(gg) summing the Error_k values to calculate the generalization error for the training set whose generalization error is being measured.

9. The method of claim 8, wherein a generalized regression neural network is used for the step of training the neural network.