Automatic color and grain sorting of materials

• US 5,761,070 A
• Filed: 11/02/1995
• Issued: 06/02/1998
• Est. Priority Date: 11/02/1995
• Status: Expired due to Term

• Current Assignee: Virginia Tech Intellectual Properties, Inc.
• Original Assignee: Virginia Tech Intellectual Properties, Inc.
• Inventors: Lu, Qiang, Conners, Richard W.
• Primary Examiner(s): Trammell, James P.
• Assistant Examiner(s): Shah, Kamini S.
• Application Number:
  US08/556,815
• Time in Patent Office: 943 Days
• Field of Search: 364/478.11, 364/526, 358/518, 358/523, 358/527, 358/520, 358/534, 358/467, 345/199, 356/419, 356/406, 395/425, 395/131, 209/580-587, 382/165, 382/170
• US Class Current: 700/223
• CPC Class Codes: B07C 5/3422 using video scanning devices, e.g. TV-cameras

Abstract

A system for automatically color and grain sorting materials includes a pair of color cameras positioned to view the top and bottom faces, respectively, of each part. A computer analyzes the data from the top and bottom cameras, and also controls the voltage on the input line to each camera. Camera controllers accept analog data from the camera heads and digitize it so that the information can be passed to the computer for analysis. A white target inserted into the field of view of each camera allows the computer to collect the data needed to do "shading compensation" on the collected color image data. Three basic categories of algorithms are used in the sorting system in accordance with the present invention: (1) training algorithms used to teach the system to identify the color and grain classes that are to be used during the sorting process, (2) real-time operation algorithms which perform the color and grain sorts with parts moving through the system at the desired throughput rate, and (3) utility algorithms which allow operators to display images, perform system tests, etc. In the training and sorting algorithms, the computer produces a black/white histogram of a part face based on the data analysis, and applies a character mark algorithm to the black/white histogram to find a threshold value for eliminating the effects of character marks from the color and grain classification and sorting of the part.

Claims

1. Apparatus for the automatic color sorting of a part having top and bottom faces to be color and grain sorted, comprising:

a materials conveyor for moving the part;

top and bottom color cameras positioned to view the top and bottom faces, respectively, of the part, said top and bottom cameras defining fields of view through which the top and bottom faces pass and outputting color image data representing the colors of the top and bottom faces;

top and bottom light sources positioned to illuminate the top and bottom faces, respectively, of the part;

a power supply providing power to said top and bottom light sources; and

computer means for analyzing the data from said top and bottom camera, said computer means including color sorting means for performing color sorting in real time with parts moving on said conveyor past said top and bottom cameras, said color sorting means including means for computing a black/white histogram of a part face based on the data analysis, and means for applying a character mark algorithm to the black/white histogram to find a threshold value for eliminating the effects of character marks from a part face for use in the color sorting of the part faces.

2. The apparatus of claim 1, further comprising first and second cooling enclosures enclosing said top and bottom cameras, respectively.

3. The apparatus of claim 1, further comprising:

a single input line electrically connecting said switching power supply to said computer means, said input line controlling the amount of power supplied to each of said top and bottom light sources; and

voltage control means for permitting said computer means to control the voltage of said input line.

4. The apparatus of claim 3, wherein said voltage control means comprises a digital to analog converter.

5. The apparatus of claim 1, further comprising an air-conditioned dust free enclosure enclosing said computer means and power supply.

6. The apparatus of claim 1, further comprising top and bottom fiber optic light lines, light emitted by said top and bottom light sources being sent through said top and bottom fiber optic light lines, respectively, to illuminate the top and bottom faces.

7. The apparatus of claim 1, wherein each of said top and bottom light sources comprises at least two quartz tungsten halogen lamps.

8. The apparatus of claim 7, further comprising a plurality of fiber optic light lines optically connected to respective ones of said lamps, light emitted by said lamps being sent through said respective fiber optic light lines to illuminate the top and bottom faces, respectively.

9. The apparatus of claim 1, said apparatus further comprising first and second camera controllers interposed between said top and bottom cameras, respectively, and said computer means, said first and second camera controllers accepting analog data from said top and bottom cameras, respectively, and digitizing it for analysis by said computer means.

10. The apparatus of claim 9, wherein each of said first and second camera controllers each includes a single analog to digital converter, said apparatus further comprising a blue filter used on each of said top and bottom cameras.

11. The apparatus of claim 9, wherein each of said first and second camera controllers includes three separate analog to digital converters, each of said analog to digital converters having its own offset and gain.

12. The apparatus of claim 9, further comprising first and second targets selectively insertable into the fields of view of said top and bottom cameras, respectively, through which the part passes, said first and second targets being substantially identical to each other and having a reflectivity chosen based upon the reflectivity of the part;

and

wherein said first and second camera controllers include means for performing a rough analog shading correction prior to analog to digital conversion of the data, based on data collected from said top and bottom cameras while viewing said first and second targets, and for performing a fine analog shading correction following analog to digital conversion of the collected data.

13. The apparatus of claim 1, further comprising direct memory access computer interface means for directly transferring the data from said top and bottom cameras to said computer means.

14. The apparatus of claim 13, further comprising a first sensor for determining when a part is about to enter the field of view of said top and bottom cameras;

and

a second sensor for determining the number of pixels on each of said top and bottom cameras which must be read.

15. The apparatus of claim 14, wherein said first sensor is an optical sensor and said second sensor is an ultrasonic sensor.

16. The apparatus of claim 1, further comprising high speed processing board means for processing the data from said top and bottom cameras and direct memory access computer interface means for transferring the processed data from said interface means to said computer means.

17. The apparatus of claim 1, further comprising first and second targets insertable into said fields of view of said top and bottom cameras, respectively, outside which the part passes.

18. The apparatus of claim 1, wherein said top and bottom light sources each have a smooth spectral energy distribution from about 400 to about 700 nm and a substantially stable spectral energy distribution function across its lifetime, and are of a type that does not have spectral lines of the type present in the light output by most fluorescent bulbs.

19. The apparatus of claim 1, wherein said apparatus is also for the grain sorting of a part, and wherein said computer means further includes training means for teaching said computer means to identify color classes to be used during sorting, training means for teaching said computer means to identify grain classes to be used during sorting, and grain sorting means for performing sorting means for performing color and grain sorting in real time with parts moving on said conveyor past said top and bottom cameras, grain sorting in real time with parts moving on said conveyor past said top and bottom cameras, and utility means for allowing an operator to display images and perform system tests.

20. The apparatus of claim 19, wherein said color cameras have red, green, and blue channels;

and

wherein said color class identification training means includes;

means for applying a shading compensation algorithm to a color image of a training sample collected by one of said cameras to produce a shading compensated color image;

means for using the shading compensated color image to average the red, green, and blue channel values to form a black and white image;

means for applying a preselected threshold to the black and white image to find and remove background pixels in the black and white image from further consideration;

means for using the shading compensated color image to compute a three-dimensional color histogram for the training sample, ignoring any background pixels;

means for using the black and white image to compute the black/white histogram for the training sample, again ignoring any background pixels;

means for applying a character mark detection algorithm to the black/white histogram to find a threshold value for eliminating the effect of character marks;

means for removing character mark pixels from the three-dimensional color histogram using the threshold value for eliminating the effects of character marks from a training sample;

means for normalizing the three-dimensional color histogram to convert it to an estimated probability function of the training sample;

means for adding the estimated probability function of the training sample to a running sum of three-dimensional estimated probability functions for the color class;

means for also adding the estimated probability function of the training sample to a running sum for all color classes;

means for applying a color mapping algorithm to the running sum for all color classes to produce a color lookup table;

means for using the color lookup table to map the estimated probability function for each training sample into a first reduced measurement vector;

means for using the color lookup table to map each running sum of three-dimensional estimated probability functions for a color class into a second reduced measurement vector;

means for determining a single prototype for each color class based on the second reduced measurement vector; and

means for estimating a threshold for each color class by examining the first reduced measurement vector for all the color classes and selecting the threshold that gives the minimum probability of error.

21. The apparatus of claim 19, wherein said color cameras have red, green, and blue channels;

and

wherein said color class identification training means includes;

means for applying a shading compensation algorithm to a color image of a training sample collected by one of said cameras to produce a shading compensated color image;

means for using the shading compensated color image to average the red, green, and blue channel values to form a black and white image;

means for applying a preselected threshold to the black and white image to find and removing background pixels in the black and white image from further consideration;

means for using the shading compensated color image to compute a three-dimensional color histogram for the training sample, ignoring any background pixels;

means for using the black and white image to compute a black/white histogram for the training sample, again ignoring any background pixels;

means for applying a character mark detection algorithm to the black/white histogram to find a threshold value for eliminating the effect of character marks from a training sample;

means for removing character mark pixels from the three-dimensional color histogram using the threshold value for eliminating the effects of character marks from a training sample;

means for normalizing the three-dimensional color histogram to convert it to an estimated probability function of the training sample;

means for adding the estimated probability function of the training sample to a running sum of three-dimensional estimated probability functions for the color class;

means for also adding the estimated probability function of the training sample to a running sum for all color classes;

means for applying a color mapping algorithm to the running sum for all color classes to produce a color lookup table;

means for using the color lookup table to map the estimated probability function for each training sample into a reduced measurement vector;

means for using the color lookup table to map each estimated probability function into a reduced measurement vector; and

means for estimating the threshold for each color class by examining all of the training samples for all the color classes and selecting the threshold that gives the best results.

22. The apparatus of claim 19, wherein said color class identification training means further includes color mapping means for reducing the size of a measurement vector needed to represent one of the faces of one of the parts and for removing colors that might be character marks from the training samples.

23. The apparatus of claim 19, wherein said color class identification training means includes means for producing a color lookup table and means for determining a prototype for each color class using the lookup table;

and

wherein said color sorting means further includes;

means for shade compensating a color image of a training sample collected by one of said cameras to remove non-uniformities in light and in sensing element sensitivities across the filed of view;

means for averaging the red, green, and blue components of each color pixel in the shade compensated color image to create a black and white image;

means for applying a threshold to the black and white image to remove background pixels in the black and white image from consideration;

means for computing a reduced measurement vector of the part face using the color lookup table produced by said color class identification training means;

means for removing character mark pixels from the reduced measurement vector using the threshold value for eliminating the effects of character marks from a part face;

means for normalizing the reduced measurement vector to form a normalized reduced measurement vector of the part face;

means for removing character mark colors from each of the prototypes;

means for forming a new set of modified prototypes with the effects of character marks removed from consideration;

means for computing the distance of the normalized reduced measurement vector of the part face from each of the prototypes; and

means for assigning a color class to the part face based on said distance.

24. The apparatus of claim 19, wherein said color class identification training means includes means for producing a color lookup table and means for determining a prototype for each color class using the lookup table;

and

wherein said color sorting means further includes;

means for shade compensating a color image of a part face collected by one of said cameras to remove non-uniformities in light and in sensing element sensitivities across the filed of view;

means for averaging the red, green, and blue components of each color pixel in the shade compensated color image to create a black and white image;

means for applying a threshold to the black and white image to remove background pixels in the black and white image from consideration;

means for computing a reduced measurement vector of the part face using the color lookup table produced by said color class identification training means;

means for removing character mark pixels from the reduced measurement vector using the threshold value for eliminating the effects of character marks from a part face;

means for normalizing the reduced measurement vector to form a normalized reduced measurement vector of the part face;

means for removing character mark colors from each training sample to create a modified measurement vector for each training sample;

means for normalizing the modified measurement vector for each training sample;

means for computing the distance value of the normalized reduced measurement vector of the part face from the normalized reduced measurement vector for each training sample;

means for finding the k smallest of the distance values;

means for finding the color class label that occurs the most often in the training vectors with the k smallest distances; and

means for assigning the part face to a color class based on the smallest distance.

25. The apparatus of claim 19, wherein said color sorting means further includes color mapping means for reducing the size of a measurement vector needed to represent one of the faces of one of the parts and wherein said means for applying a character mark algorithm also functions to remove colors that might be character marks from the two reduced measurement vectors used to represent the color of each part face.

26. The apparatus of claim 19, wherein said color cameras have red, green, and blue channels;

and

wherein said grain class identification training means includes;

means for applying a shading compensation algorithm to a color image of a training sample collected by one of said cameras to remove non-uniformities in lighting and in camera sensing element sensitivities across the field of view to produce a shading compensated color image;

means for using the shading compensated color image to average the red, green, and blue channel values to form a black and white image;

means for applying a preselected threshold to the black and white image to find and remove background pixels in the black and white image from further consideration;

means for using the black and white image to compute a black/white histogram for the part face, ignoring any background pixels;

means for applying a character mark algorithm to the black/white histogram to find a threshold value for eliminating the effects of character marks from a training sample;

means for removing character mark pixels from the black/white histogram using the threshold value for eliminating the effects of character marks from a training sample;

means for removing character mark areas from further consideration by labeling them as background pixels in the black and white image;

means for computing a normalizing factor;

means for applying an equal probability quantizing algorithm to the black and white image of the part, the black/white histogram, the normalizing factor, and the number of gray levels that are to appear in a requantized version of the black and white image to obtain the requantized version of the black and white image;

means for applying edge detectors respectively most sensitive to the vertical, horizontal, right diagonal, and left diagonal edges to the requantized version of the black and white image to obtain gradient images which record the absolute values of, respectively, the vertical, horizontal, right diagonal, and left diagonal edges;

means for averaging the left and right diagonal gradient images to find a single gradient image that indicates the number of diagonal edges present in either the right or left diagonal dimensions;

means for creating an edge histogram and normalizing it;

means for finding a class prototype for each color class; and

means for estimating a threshold for each color class by examining the normalized edge histogram for all the training samples for all the color classes and selecting the threshold that gives the minimum probability of error.

27. The apparatus of claim 19, wherein said color cameras have red, green, and blue channels;

and

wherein said grain class identification training means includes;

means for applying a shading compensation algorithm to a color image of a training sample collected by one of said cameras to remove non-uniformities in lighting and in camera sensing element sensitivities across the field of view to produce a shading compensated color image;

means for using the shading compensated color image to average the red, green, and blue channel values to form a black and white image;

means for applying a preselected threshold to the black and white image to find and remove background pixels in the black and white image from further consideration;

means for using the black and white image to compute a black/white histogram for the part face, ignoring any background pixels;

means for applying a character mark algorithm to the black/white histogram to find a threshold value for eliminating the effects of character marks from a training sample;

means for removing character mark pixels from the black/white histogram using the threshold value for eliminating the effects of character marks from a training sample;

means for removing character mark areas from further consideration by labeling them as background pixels in the black and white image;

means for computing a normalizing factor;

means for applying an equal probability quantizing algorithm to the black and white image of the part, the black/white histogram, the normalizing factor, and the number of gray levels that are to appear in a requantized version of the black and white image to obtain the requantized version of the black and white image;

means for applying edge detectors respectively most sensitive to the vertical, horizontal, right diagonal, and left diagonal edges to the requantized version of the black and white image to obtain gradient images which record the absolute values of, respectively, the vertical, horizontal, right diagonal, and left diagonal edges;

means for averaging the left and right diagonal gradient images to find a single gradient image that indicates the number of diagonal edges present in either the right or left diagonal dimensions;

means for creating an edge histogram and normalizing it; and

means for estimating a threshold for each grain class by examining the normalized edge histogram for all the training samples for all the grain classes and selecting the threshold that gives the best results.

28. The apparatus of claim 19, wherein said grain class training identification means includes means for determining a prototype for each grain class;

and

wherein said grain sorting means includes;

means for removing character mark pixels from the black/white histogram of the best part face;

means for removing character mark areas from further consideration by labeling them as background pixels in the black and white image of the best part face;

means for computing a normalizing factor;

means for applying an equal probability quantizing algorithm to the black and white image of the best part face, the black/white histogram of the black and white image of the best part face, the normalizing factor, and the number of gray levels that are to appear in a requantized version of the black and white image to obtain the requantized version of the black and white image;

means for applying edge detectors respectively most sensitive to the vertical, horizontal, right diagonal, and left diagonal edges to the requantized version of the black and white image to obtain gradient images which record the absolute values of, respectively, the vertical, horizontal, right diagonal, and left diagonal edges;

means for averaging the left and right diagonal gradient images to find a single gradient image that indicates the number of diagonal edges present in either the right or left diagonal dimensions;

means for creating an edge histogram and normalizing it;

means for computing the distance of the normalized edge histogram from each of the prototypes; and

means for assigning a grain pattern class to the part face based on the distance.

29. The apparatus of claim 19, wherein said grain class training identification means includes means for determining a prototype for each grain class;

and

wherein said grain sorting means includes;

means for removing character mark pixels from the black/white histogram of the best part face;

means for removing character mark areas from further consideration by labeling them as background pixels in the black and white image of the best part face;

means for computing a normalizing factor;

means for applying an equal probability quantizing algorithm to the black and white image of the best part face, the black/white histogram of the black and white image of the best part face, the normalizing factor, and the number of gray levels that are to appear in a requantized version of the black and white image to obtain the requantized version of the black and white image;

means for applying edge detectors respectively most sensitive to the vertical, horizontal, right diagonal, and left diagonal edges to the requantized version of the black and white image to obtain gradient images which record the absolute values of, respectively, the vertical, horizontal, right diagonal, and left diagonal edges;

means for averaging the left and right diagonal gradient images to find a single gradient image that indicates the number of diagonal edges present in either the right or left diagonal dimensions;

creating an edge histogram and normalizing it;

means for computing the distance of the normalized edge histogram from each of the training samples;

means for finding the grain class label that occurs the most often in the training vectors with the k smallest distances; and

means for assigning the part face to a grain class based on the smallest distance.

30. The apparatus of claim 1, wherein said color cameras have red, green, and blue channels;

and

wherein said computer means includes normalizing means for executing a normalizing algorithm to normalize variations in sensitivity between said top and bottom cameras, said normalizing means including;

means for applying a shading correction algorithm to color images of color samples collected from said top and bottom cameras;

means for computing the values of matrices representing the relative red, green and blue responses of said top and bottom cameras based on the shading corrected color component images collected from said top and bottom cameras;

means for creating two-dimensional plots of the relative red, green, and blue responses of said top and bottom cameras to all of the color targets, with the horizontal axes of the plots representing the output gray level value for the red, green, and blue channels of one of said top and bottom cameras and the vertical axes of the plots representing the output gray level value for the red, green, and blue channels of the other of said top and bottom cameras;

means for terminating execution of the normalizing algorithm, if the function y=x is the function that best fits the points on all three of the red, green, and blue plots;

means for estimating the degree of polynomial function that appears to best fit the relative response data;

means for terminating execution of the normalizing algorithm, if the function y=x is not the function that best fits the points on all three of the red, green, and blue plots;

means for defining a function for each graph that fits the data, using a least squares fitting program; and

means for creating three lookup tables that map the output of one of said top and bottom cameras into the output of the other of said top and bottom cameras.

31. A method of automatically color sorting a part having opposite top and bottom faces to be color sorted, comprising the steps of:

providing top and bottom color cameras positioned to view the top and bottom faces, respectively, of a part, the top and bottom cameras defining fields of view through which the top and bottom faces pass;

illuminating the fields of view;

moving the part past the top and bottom cameras so that the top and bottom faces pass through the fields of view of the top and bottom cameras;

using the top and bottom cameras to output color image data representing the colors of the top and bottom faces;

computing black/white histograms of each of the top and bottom faces based on the color image data output by the cameras;

applying a character mark algorithm to the black/white histograms of each of the top and bottom faces to produce a threshold value for eliminating the effects of character marks from a part face for use in the color sorting of each of the top and bottom faces;

eliminating the effects of character mark from the data from the top and bottom cameras using the threshold value produced by the character mark algorithm for eliminating the effects of character marks from a part face;

determining the color class of the top and bottom faces based on the data from the top and bottom cameras; and

determining which of the top and bottom faces is the better looking based on the determination of the color class of the top and bottom faces.

32. The method of claim 31, further comprising cooling the top and bottom cameras to maintain them at a substantially constant temperature.

33. The method of claim 31, further comprising controlling the amount of power supplied to each of the light sources by controlling the amount of voltage on the input line from the power supply to the light sources.

34. The method of claim 33, wherein said step of controlling the amount of power comprises placing a target in the field of view of one of the top and bottom cameras, averaging the picture elements of the camera that cover an area of the target, comparing the average to a standard, and if necessary adjusting the voltage applied to the input line based on the comparison.

35. The method of claim 31, wherein said illuminating step comprises sending light emitted by top and bottom light sources through fiber optic light lines to illuminate the top and bottom faces.

36. The method of claim 31 wherein said illuminating step comprises using light sources having a smooth spectral energy distribution from about 400 to about 700 nm and a substantially stable spectral energy distribution function across its lifetime, and being of a type that do not have spectral lines of the type present in the light output by fluorescent bulbs.

37. The method of claim 31, further comprising the step of digitizing the data from the top and bottom cameras prior to said step of determining the color class of the top and bottom faces.

38. The method of claim 37, further comprising filtering light entering the top and bottom cameras using blue filters.

39. The method of claim 37, further comprising the steps of performing a rough analog shading correction prior to said digitizing step and performing a fine analog shading correction following said digitizing step.

40. The method of claim 31, further comprising the steps of determining when a part is about to enter the field of view of said top and bottom cameras;

and determining the number of pixels on each of said top and bottom cameras which must be read.

41. A method of automatically color sorting a part having opposite top and bottom faces to be color sorted, comprising the steps of:

providing a color sorting system including;

top and bottom color cameras positioned to view the top and bottom faces, respectively, of a part and for outputting color image data representing the colors of the top and bottom faces; and

processing means for processing the data from the top and bottom cameras;

training the system to identify a plurality of color classes that are to be used during color sorting;

collecting a color image of the part face using the cameras;

computing black/white histograms of each of the top and bottom faces based on the color image data output by the cameras;

applying a character mark algorithm to the black/white histograms of each of the top and bottom faces to produce a threshold value for eliminating the effects of character marks from a part face for use in the color sorting of each of the top and bottom faces;

eliminating the effects of character marks from the data from the top and bottom cameras using the threshold value for eliminating the effects of character marks from a part face; and

performing color sorting with parts moving through the system using the data from the top and bottom cameras, based on the color classes identified during said training step.

42. The method of claim 41, wherein said step of teaching the system to identify the color classes includes removing the effects of varying intensities among the color classes from the output of edge detectors, using an equal probability quantizing algorithm.

43. The method of claim 41, wherein the color cameras have red, green, and blue channels and wherein said color class training step includes:

collecting a color image of a training sample using the cameras;

applying a shading compensation algorithm to the color image of the training sample to produce a shading compensated color image of the training sample;

using the shading compensated color image to average the red, green, and blue channel values to form a black and white image;

applying a preselected threshold to the black and white image to find and remove background pixels in the black and white image from further consideration;

using the shading compensated color image to compute a three-dimensional color histogram for the training sample, ignoring any background pixels found in said step of applying a preselected threshold;

using the black and white image to compute a black/white histogram for the training sample, again ignoring any background pixels found in said step of applying a preselected threshold;

applying a character mark detection algorithm to the black/white histogram to find a threshold value for eliminating the effects of character marks from a training sample;

removing character mark pixels from the three-dimensional color histogram using the threshold value for eliminating the effects of character marks from a training sample;

normalizing the three-dimensional color histogram to convert it to an estimated probability function of the training sample;

adding the estimated probability function of the training sample to a running sum of three-dimensional estimated probability functions for the color class;

also adding the estimated probability function of the training sample to a running sum for all color classes;

repeating the above steps for additional training samples;

applying a color mapping algorithm to the running sum for all color classes to produce a color lookup table;

using the color lookup table to map the estimated probability function for each training sample into a first reduced measurement vector;

using the color lookup table to map each running sum of three-dimensional estimated probability functions for a color class into a second reduced measurement vector;

determining a single prototype for each color class based on the second reduced measurement vector; and

for each color class estimating a threshold by examining the first reduced measurement vector for all the color classes and selecting the threshold that gives the minimum probability of error.

44. The method of claim 43, wherein said step of applying a shading correction algorithm includes collecting a selected number of lines of color image data from one of the cameras while a lens cap is on the cameras, computing the average response for each pixel in each channel of data, removing the lens cap and scanning the selected number of lines of color image data off the white target, computing the average response for each pixel in each channel of data, and applying the steps of a standard shading correction algorithm to the color imagery as it is being collected.

45. The method of claim 41, wherein the color cameras have red, green, and blue channels and wherein said color class training step includes:

collecting a color image of a training sample using the cameras;

applying a shading compensation algorithm to the color image of the training sample to produce a shading compensated color image of the training sample;

using the shading compensated color image to average the red, green, and blue channel values to form a black and white image;

applying a preselected threshold to the black and white image to find and remove background pixels in the black and white image from further consideration;

using the shading compensated color image to compute a three-dimensional color histogram for the training sample, ignoring any background pixels found in said step of applying a preselected threshold;

using the black and white image to compute a black/white histogram for the training sample, again ignoring any background pixels found in said step of applying a preselected threshold;

applying a character mark detection algorithm to the black/white histogram to find a threshold value for eliminating the effects of character marks from a training sample;

removing character mark pixels from the three-dimensional color histogram using the threshold value for eliminating the effects of character marks from a training sample;

normalizing the three-dimensional color histogram to convert it to an estimated probability function of the training sample;

adding the estimated probability function of the training sample to a running sum of three-dimensional estimated probability functions for the color class;

also adding the estimated probability function of the training sample to a running sum for all color classes;

repeating the above steps for additional training samples;

applying a color mapping algorithm to the running sum for all color classes to produce a color lookup table;

using the color lookup table to map the estimated probability function for each training sample into a reduced measurement vector;

using the color lookup table to map each estimated probability function into a reduced measurement vector; and

for each color class, estimating the threshold by examining all of the training samples for all the color classes and selecting the threshold that gives the best results.

46. The method of claim 45, wherein said step of applying a shading correction algorithm includes collecting a selected number of lines of color image data from one of the cameras while a lens cap is on the cameras, computing the average response for each pixel in each channel of data, removing the lens cap and scanning the selected number of lines of color image data off the white target, computing the average response for each pixel in each channel of data, and applying the steps of a standard shading correction algorithm to the color imagery as it is being collected.

47. The method of claim 41, wherein said color class training identification step includes producing a color lookup table and determining a prototype for each color class;

and

wherein said color sorting step further includes;

after said step of collecting a color image of the part face using the cameras, shade compensating the color image of the part face to remove non-uniformities in light and in sensing element sensitivities across the field of view;

averaging the red, green, and blue components of each color pixel in the shade compensated color image to create a black and white image;

applying a threshold to the black and white image to remove background pixels in the black and white image from consideration;

computing a reduced measurement vector of the part face using the color lookup table computed in said color class identification training step;

performing said step of eliminating the effects of character marks from the data from the top and bottom cameras by removing character mark pixels from the reduced measurement vector of the part face using the threshold value for eliminating the effects of character marks from a part face;

normalized the reduced measurement vector to form the normalized reduced measurement vector of the part face;

removing character mark colors from each of the prototypes;

forming a new set of modified prototypes with the effects of character marks removed from consideration;

computing the distance of the normalized reduced measurement vector of the part face from each of the prototypes; and

assigning a color class to the part face based on the distance.

48. The method of claim 47, further comprising the steps of using information about the class label of the color class to which the face has been assigned, a distance measure, and the clear area of the part face to determine which of the two faces of a part is the best face.

49. The method of claim 41, wherein said color class training identification step includes producing a color lookup table;

and

wherein said color sorting step further includes;

shade compensating the color image of the part face to remove non-uniformities in light and in sensing element sensitivities across the filed of view;

averaging the red, green, and blue components of each color pixel in the shade compensated color image to create a black and white image;

applying a threshold to the black and white image to remove background pixels in the black and white image from consideration;

computing a reduced measurement vector of the part face using the color lookup table computed in said color class identification training step;

performing said step of eliminating the effects of character marks from the data from the top and bottom cameras by removing character mark pixels from the reduced measurement vector of the part face using the threshold value for eliminating the effects of character marks from a part face;

normalizing the reduced measurement vector of the part face to create a normalized reduced measurement vector of the part face;

removing character mark colors from each training sample to create a modified measurement vector for each training sample;

normalizing the modified measurement vector for each training sample to create a normalized measurement vector for each training sample;

computing the distance value of the normalized reduced measurement vector of the part face from the normalized reduced measurement vector for each training sample;

finding the k smallest of the distance values;

finding the color class label that occurs the most often in the training vectors with the k smallest distances; and

assigning the part face to a color class based on the smallest distance.

50. The method of claim 49, further comprising the steps of using information about the class label of the color class to which the face has been assigned, a distance measure, and the clear area of the part face to determine which of the two faces of a part is the best face.

51. The method of claim 41, wherein said method also includes grain sorting of the part and said method further includes:

training the system to identify the grain classes that are to be used during grain sorting; and

performing grain sorting on the parts which have been color sorted; and

wherein the color cameras have red, green, and blue channels and wherein said grain class training step includes;

collecting a color image of a training sample using the cameras;

applying a shading compensation algorithm to the color image of the training sample to remove non-uniformities in lighting and in camera sensing element sensitivities across the field of view to produce a shading compensated color image of the training sample;

using the shading compensated color image to average the red, green, and blue channel values to form a black and white image;

applying a preselected threshold to the black and white image, removing background pixels in the black and white image from further consideration;

using the black and white image to compute a black/white histogram for the part face, ignoring any background pixels;

applying a character mark algorithm to the black/white histogram to find a threshold value for eliminating the effects of character marks from a training sample;

removing character mark pixels from the black/white histogram using the threshold value for eliminating the effects of character marks from a training sample;

removing character mark areas from further consideration by labeling them as background pixels in the black and white image;

computing a normalizing factor;

applying an equal probability quantizing algorithm to the black and white image of the part, the black/white histogram, the normalizing factor, and the number of gray levels that are to appear in a requantized version of the black and white image to obtain the requantized version of the black and white image;

applying edge detectors respectively most sensitive to the vertical, horizontal, right diagonal, and left diagonal edges to the requantized version of the black and white image to obtain gradient images which record the absolute values of, respectively, the vertical, horizontal, right diagonal, and left diagonal edges;

averaging the left and right diagonal gradient images to find a single gradient image that indicates the number of diagonal edges present in either the right or left diagonal dimensions;

creating an edge histogram and normalizing it;

finding a class prototype for each color class; and

for each color class estimating a threshold by examining the normalized edge histogram for all the training samples for all the color classes and selecting the threshold that gives the minimum probability of error.

52. The method of claim 41, wherein said method also includes grain sorting of the part and said method further includes:

training the system to identify the grain classes that are to be used during grain sorting; and

performing grain sorting on the parts which have been color sorted; and

wherein the color cameras have red, green, and blue channels and wherein said grain class training step includes;

collecting a color image of a training sample using the cameras;

applying the shading compensation algorithm to the color image of the training sample to remove non-uniformities in lighting and in camera sensing element sensitivities across the field of view to produce a shading compensated color image of the training sample;

using the shading compensated color image to average the red, green, and blue channel values to form a black and white image;

applying a preselected threshold to the black and white image, to find and remove background pixels in the black and white image from further consideration;

using the black and white image to compute a black/white histogram for the part face, ignoring any background pixels;

applying a character mark algorithm to the black/white histogram to find a threshold value for eliminating the effects of character marks;

removing character mark pixels from the black/white histogram using the threshold value for eliminating the effects of character marks;

removing character mark areas from further consideration by labeling them as background pixels in the black and white image;

computing a normalizing factor;

applying an equal probability quantizing algorithm to the black and white image of the part, the black/white histogram, the normalizing factor, and the number of gray levels that are to appear in a requantized black and white image to obtain the requantized version of the black and white image;

applying edge detectors respectively most sensitive to the vertical, horizontal, right diagonal, and left diagonal edges to the requantized version of the black and white image to obtain gradient images which record the absolute values of, respectively, the vertical, horizontal, right diagonal, and left diagonal edges;

averaging the left and right diagonal gradient images to find a single gradient image that indicates the number of diagonal edges present in either the right or left diagonal dimensions;

creating an edge histogram and normalizing it; and

estimating a threshold for each grain class by examining the normalized edge histogram for all the training samples for all the grain classes and selecting the threshold that gives the best results.

53. The method of claim 41, wherein said method also includes grain sorting in said method further includes:

training the system to identify the grain classes that are to be used during grain sorting; and

performing grain sorting on the parts which have been color sorted; and

wherein said grain sorting step includes;

removing character mark pixels from the black/white histogram of the best part face;

removing character mark areas from further consideration by labeling them as background pixels in the black and white image of the best part face;

computing a normalizing factor;

applying an equal probability quantizing algorithm to the black and white image of the best part face, the black/white histogram of the black and white image of the best part face, the normalizing factor, and the number of gray levels that are to appear in a requantized black and white image to obtain the requantized version of the black and white image;

applying edge detectors respectively most sensitive to the vertical, horizontal, right diagonal, and left diagonal edges to the requantized version of the black and white image to obtain the gradient images which record the absolute values of, respectively, the vertical, horizontal, right diagonal, and left diagonal edges;

averaging the left and right diagonal gradient images to find a single gradient image that indicates the number of diagonal edges present in either the right or left diagonal dimensions;

creating an edge histogram and normalizing it;

computing the distance of the normalized edge histogram from each of the prototypes; and

assigning a grain pattern class to the part face based on the distance.

54. The method of claim 41, wherein said method also includes grain sorting of the part and said method further includes:

training the system to identify the grain classes that are to be used during grain sorting; and

performing grain sorting on the parts which have been color sorted; and

wherein said grain sorting step includes;

removing character mark pixels from the black/white histogram of the best part face;

removing character mark areas from further consideration by labeling them as background pixels in the black and white image of the best part face;

computing a normalizing factor;

applying an equal probability quantizing algorithm to the black and white image of the best part face, the black/white histogram of the black and white image of the best part face, the normalizing factor, and the number of gray levels that are to appear in a requantized black and white image to obtain the requantized version of the black and white image;

applying edge detectors respectively most sensitive to the vertical, horizontal, right diagonal, and left diagonal edges to the requantized version of the black and white image to obtain gradient images which record the absolute values of, respectively, the vertical, horizontal, right diagonal, and left diagonal edges;

averaging the left and right diagonal gradient images to find a single gradient image that indicates the number of diagonal edges present in either the right or left diagonal dimensions;

creating an edge histogram and normalizing it;

computing the distance of the normalized edge histogram from the training sample;

finding the k smallest of the distance values;

finding the grain class label that occurs the most often in the training vectors with the k smallest distances; and

assigning the part face to a grain class based on the smallest distance.

55. The method of claim 41, wherein said training step is carried out using a plurality of color samples, said method further including the step of normalizing variations in sensitivity between the top and bottom cameras, said normalizing step including:

for each color sample, scanning the color sample to collect a color image of the sample from both cameras;

applying a shading correction algorithm to the color images from both cameras to produce shading corrected color images, and computing the values of matrices representing the relative red, green and blue responses of the two cameras based on the shading corrected color images collected from both cameras;

creating a two-dimensional plot of the relative red responses of the two cameras to all of the color targets, with the horizontal axis of the two-dimensional plot representing the output gray level value for the red channel of the first camera and the vertical axis of the two-dimensional plot representing the output gray level value for the red channel of the second camera;

creating a two-dimensional plot of the relative green responses of the two cameras to all of the color targets, with the horizontal axis of the two-dimensional plot representing the output gray level value for the green channel of the first camera and the vertical axis of the two-dimensional plot representing the output gray level value for the green channel of the second camera;

creating a two-dimensional plot of the relative blue responses of the two cameras to all of the color targets, with the horizontal axis of the two-dimensional plot representing the output gray level value for the blue channel of the first camera and the vertical axis of the two-dimensional plot representing the output gray level value for the blue channel of the second camera;

determining if the function y=x is the function that best fits the points on all three of the red, green, and blue plots;

if y=x provides the best fit, terminating execution of the normalizing algorithm, as adjustment in either camera output is unneeded;

if y=x does not provide the best fit, estimating the degree of polynomial function that appears to best fit the relative response data;

defining a function for each graph that fits the data, using a least squares fitting program; and

creating three lookup tables that map the output of the first camera into the output of the second camera.