Discriminant neural networks

• US 5,926,804 A
• Filed: 07/01/1994
• Issued: 07/20/1999
• Est. Priority Date: 07/01/1994
• Status: Expired due to Fees

• Current Assignee: Rhode Island Board of education (STATE OF RHODE ISLAND)
• Original Assignee: Rhode Island Board of education (STATE OF RHODE ISLAND)
• Inventors: Li, Qi, Tufts, Donald W.
• Primary Examiner(s): Downs, Robert W.
• Assistant Examiner(s): Shah, Sanjiv
• Application Number:
  US08/270,045
• Time in Patent Office: 1,845 Days
• Field of Search: 395/23, 395/21, 395/24, 706/25, 706/16
• US Class Current: 706/25
• CPC Class Codes: G06F 18/2132 based on discrimination criteria, e.g. discriminant analysis
• CPC Class Codes: G06F 18/24317 Piecewise classification, i.e. whereby each classification requires several discriminant rules
• CPC Class Codes: G06N 3/08 Learning methods

Abstract

A discriminant neural network and a method of training the network are disclosed. The network includes a set of hidden nodes having associated weights, and the number of hidden nodes is minimized by the training method of the invention. The training method includes the steps of 1) loading a training data set and assigning it to a residual data set, 2) computing a vector associated with a first hidden node using the residual data set, 3) projecting training data onto a hyperplane associated with said first hidden node, 4) determining the number and locations of hard-limiter thresholds associated with the first node, and 5) repeating the above for successive hidden nodes after removing satisfied subsets from the training data until all partitioned regions of the input data space are satisfied.

Claims

1. A method of training a discriminant neural network including a set of hidden nodes having associated weights, said method including the steps of:

loading a training data set and assigning it to a residual data set;

computing a vector associated with a first hidden node using the residual data set and a linear discriminant algorithm characterized by ##EQU7## where S_W is a linear combination of covariance matrices, X₁, X₂ are the data sets of two different classes and m₁, m₂ are sample means of the two different classes;

projecting training data onto a hyperplane associated with said first hidden node;

determining the number and locations of hard-limiter thresholds associated with the first node; and

repeating the above for successive hidden nodes after removing satisfied subsets from the training data until all partitioned regions of the input data space are satisfied, to train the discriminant neural network.

2. A method as claimed in claim 1, wherein said method further includes the steps of evaluating whether to add a hidden node, and adding a new hidden node under appropriate conditions.

3. A method as claimed in claim 1, wherein said method further includes the step of pruning unnecessary hidden nodes from the network.

4. A method as claimed in claim 1, wherein said method further includes the step of pruning hard-limiters from the network.

5. A method as claimed in claim 1, wherein said method further includes the step of training output nodes using boolean minimization techniques.

6. A discriminant neural network training system comprising:

means for loading a training data set and assigning it to a residual data set;

means for computing a vector associated with a first hidden node using the residual data set and a linear discriminant algorithm characterized by ##EQU8## where S_W is a linear combination of covariance matrices, X₁, X₂ are the data sets of two different classes and m₁, m₂ are sample means of the two different classes;

means for projecting training data onto a hyperplane associated with said first hidden node;

means for determining the number and locations of hard-limiter thresholds associated with the first node; and

means for repeating the above for successive hidden nodes after removing satisfied subsets from the training data until all partitioned regions of the input data space are satisfied, to train the discriminant neural network.