Method and system for improving pattern recognition system performance

• US 6,847,731 B1
• Filed: 08/07/2000
• Issued: 01/25/2005
• Est. Priority Date: 08/07/2000
• Status: Expired due to Fees

• Current Assignee: Northeast Photosciences, Inc.
• Original Assignee: Northeast Photosciences
• Inventors: Caulfield, H. John
• Primary Examiner(s): Dastouri, Mehrdad
• Assistant Examiner(s): KIBLER, VIRGINIA M
• Application Number:
  US09/633,330
• Time in Patent Office: 1,632 Days
• Field of Search: 382155-161, 382/170, 382/190, 382/210, 382/280, 382/224, 382/225, 382/228, 359/29, 359/559, 706/12, 706/14, 706/16, 706/18, 706/20, 706/25, 700/47, 700/48, 700/50, 700/51
• US Class Current: 382/159
• CPC Class Codes: G06F 18/214 Generating training patterns; Bootstrap methods, e.g. bagging or boosting
• CPC Class Codes: G06V 10/89 using frequency domain filters, e.g. Fourier masks implemented on spatial light modulators

Abstract

Stand-alone or assistive pattern recognition system and process enabling error free classification of all objects in a training set and application to unclassified objects. Parameters and/or features of the data objects in a training set are selected and measured, from which discriminants are computed. The measured data is plotted in discriminant space and decision boundaries or thresholds determined, preferably such that at least one object from one class is isolated from the remaining objects, removed from the training set, and the process repeated until an acceptable number of unclassified objects remain. The system can be applied sequentially to classify all the members of the training set belonging to one class and then applied to objects in other classes. Fuzzy quantifiable determinations of an object'"'"'s likelihood of class membership can be made. Objects'"'"' positions and classifications are obtainable in an optical system using Fourier techniques without limitation to linearly discriminable problems.

Claims

1. A process for training a pattern recognition system as comprising the steps of:

(a) providing a training set of objects each classified in two or more classes;

(b) taking data from each of the objects;

(c) selecting a first discriminant space and computing discriminant values from the data;

(d) plotting the values in the discriminant space;

(e) establishing a decision boundary associated with the discriminant space;

(f) setting the decision boundary and applying a decision rule wherein at least one object in the training set is separated and correctly classified;

(g) removing correctly classified objects from the training set, thereby creating a remaining set of objects;

(h) storing the first discriminant space, corresponding decision boundary and decision rule;

(i) selecting a number of objects in the remaining set of objects such that when that number or zero is reached the pattern recognition system is trained;

(j) repeating steps (a-g) with the remaining set of objects until the selected number or zero is reached;

(k) introducing a set of objects, the objects known to be members of the two or more classes but unclassified as to which class, to the trained pattern recognition system;

(l) taking data from the unclassified objects;

(m) retrieving the first stored discriminant space, decision boundary and decision rule;

(n) computing discriminant values from the data and plotting the values in the discriminant space;

(o) applying the decision boundary and decision rule to the plotted values;

(p) removing classified objects from the unclassified set;

(q) sequentially retrieving the next discriminant space, decision boundary and decision rule; and

(r) applying the next discriminant space, decision boundary and decision rule to the diminishing unclassified set of objects.

2. The process as defined in claim 1, further comprising the step of repeating the process as defined in claim 1 until the unclassified objects or the stored discriminant space, boundaries and decision rules are exhausted.

3. Process of classifying unclassified objects using a pattern recognizer as trained and defined in claim 1, comprising the steps of:

introducing unclassified objects to the trained pattern recognition system;

taking data from the unclassified objects;

retrieving the all of the stored discriminant spaces, decision boundaries and decision rules;

computing discriminant values from data and plotting discriminant values in the discriminant spaces;

applying corresponding decision boundaries and decision rules to all the plotted data in parallel, wherein each decision rule classifies objects of a particular class; and

combining objects in the same class.

4. The process as defined in claim 3, wherein the step of combining groups of all objects of at least one class.

5. The process as defined in claim 3, wherein each step of applying classifies a portion of one class from the unclassified objects.

6. A process of improving the performance of an existing pattern recognition system, wherein the existing system has defined discriminants comprising the steps of:

accumulating objects misclassified or unclassified by the existing system; and

using these misclassified objects as a training set, and applying the process as defined in claim 1 to said training set defined herein.

7. A process for training a pattern recognition system, comprising the steps of:

(a) providing a training set of objects each classified in two or more classes;

(b) taking data from each of the objects;

(c) selecting a first discriminant space and computing discriminant values from the data;

(d) plotting the values in the discriminant space;

(e) establishing a decision boundary associated with the discriminant space;

(f) setting the decision boundary and applying a decision rule wherein at least one object in the training set is separated and correctly classified;

(g) removing correctly classified objects from the training set, thereby creating a remaining set of objects;

(h) storing the first discriminant space, corresponding decision boundary and decision rule;

(i) selecting a number of objects in the remaining set of objects such that when that number or zero is reached the pattern recognition system is trained; and

(j) repeating steps (a) through (g) with the remaining set of objects until the number of objects remaining in the training set is the select number or zero,

(k) introducing unclassified objects to the trained pattern recognition system;

(l) taking data from the unclassified objects;

(m) retrieving the all of the stored discriminant spaces, decision boundaries and decision rules;

(n) computing discriminant values from the data and plotting discriminant values in the discriminant spaces;

(o) applying corresponding decision boundaries and decision rules to all the plotted data in parallel, wherein each decision rule classifies objects of a particular class; and

(p) combining objects in the same class into logical output groupings comprising fuzzy determinations of the likelihood of the object being within a class.

8. A process for training a Fourier filter pattern recognition system with a training set of patterns positioned in an x,y plane known to be in two or more classes, comprising the steps of:

(a) modulating a coherent light beam by each of the patterns;

(b) optical Fourier transforming the modulated coherent light beam;

(c) masking or filtering the Fourier transformed beam thereby providing a masked output;

(d) optical Fourier inverse transforming of the masked output, thereby forming an optical signal positioned in a plane that corresponds to the corresponding positions in the x,y plane;

(e) sensing the optical output of step (d);

(f) applying a threshold to the sensed optical outputs;

(g) determining the x,y positions of the sensed optical signals that exceed the threshold, and classifying those patterns located at the corresponding x,y positions;

(h) removing those classified patterns from the remainder of the training set, thereby creating a remaining set of patterns;

(i) storing the masks and the thresholds;

(j) selecting a number of patterns in the remaining set of patterns such that when that number or zero is reached the Fourier filter pattern recognition system is trained;

(k) repeating steps (a) through (i) with the remaining set of patterns until the selected number or zero is reached;

(l) introducing an unclassified set of patterns the patterns positioned in an x,y plane and known to be in two or more classes but unclassified as to which class, to the trained Fourier filter pattern recognition system;

(m) modulating a coherent light beam by each of the unclassified patterns;

(n) optical Fourier transforming the modulated coherent light beam;

(o) masking or filtering the Fourier transformed beam thereby providing a masked output;

(p) optical Fourier inverse transforming of the masked output, thereby forming an optical signal positioned in a plane that corresponds to the corresponding positions in the x,y plane;

(q) sensing the optical output of step (o);

(r) retrieving the first stored mask and threshold;

(s) applying the retrieved threshold to the sensed optical output of step (q);

(t) determining the x,y positions of the sensed optical signals that exceed the threshold, and classifying those patterns located at the corresponding x,y positions;

(u) removing classified objects from the unclassified set, thereby creating a remaining unclassified set of patterns; and

(v) sequentially retrieving the next stored mask and threshold and applying the retrieved stored mask and threshold the diminishing remaining unclassified set of patterns until the unclassified set of patterns is classified.

9. The process of classifying unclassified patterns using a Fourier filter recognizer system as trained and defined in claim 8, comprising the steps of:

introducing unclassified patterns to the trained pattern recognition system; and

retrieving the stored masks, and applying steps (a) through (h) repetitively.

10. The process of classifying unclassified patterns using a Fourier filter recognizer system as trained and defined in claim 9, comprising the steps of:

introducing unclassified patterns to the trained pattern recognition system;

modulating the light beams with the patterns; and

applying the each of the stored masks and corresponding thresholds to the unclassified objects in parallel, wherein each application provides an output classified grouping.

11. The process as defined in claim 10, wherein the application of each the stored masks and corresponding thresholds produces the logical determination that all of one class have been separated from the unclassified objects.

12. A pattern recognition system comprising:

(a) a training set of objects each classified in two or more classes;

(b) means for taking data from each of the objects;

(c) means for selecting at least one discriminant space and means for computing discriminant values from the data;

(d) means for plotting the values in the discriminant space;

(e) a decision boundary associated with the discriminant space;

(f) a decision rule, defined with respect to the decision boundary, wherein at least one object in the training set is separated and correctly classified by application of the decision rule;

(g) means for removing correctly classified objects from the training set, thereby creating a remaining set of objects thereby training said pattern recognition system;

(h) means for storing the at least one discriminant space, corresponding decision boundary and decision rule;

(i) means for introducing a set of objects known to be members of the two or more classes but unclassified as to which class, to the trained pattern recognition system;

(j) means for taking data from the unclassified objects;

(k) means for retrieving the at least one stored discriminant space, decision boundary and decision rule;

(l) means for computing discriminant values from the data and plotting in the discriminant space;

(m) means for applying the decision boundary and decision rule to the plotted values;

(n) means for removing classified objects from the unclassified set; and

(o) means for sequentially retrieving the next discriminant space, decision boundary and decision rule and applying them to the diminishing unclassified set of objects.

13. The system defined in claim 12, wherein the decision boundary, that is arranged to separate at least one object and correctly classify that object, is arranged to also maximize the relative distance from the boundary decision to the nearest object of a different class.