Product inspection method and apparatus

• US 5,335,293 A
• Filed: 06/16/1992
• Issued: 08/02/1994
• Est. Priority Date: 06/16/1992
• Status: Expired due to Fees

• Current Assignee: Key Technology, Inc. (Duravant LLC)
• Original Assignee: Key Technology, Inc. (Duravant LLC)
• Inventors: Vannelli, Anthony, Madsen, Thomas C.
• Primary Examiner(s): Moore, David K.
• Application Number:
  US07/899,450
• Time in Patent Office: 777 Days
• Field of Search: 382/17, 382/1, 382/8, 209/576, 209/577, 209/580, 209/581, 209/582, 356/407, 356/408, 356/237
• US Class Current: 382/110
• CPC Class Codes: G01N 2021/8887 based on image processing techniques
• CPC Class Codes: G01N 21/88 Investigating the presence of flaws or contamination
• CPC Class Codes: G06T 2207/10016 Video; Image sequence
• CPC Class Codes: G06T 2207/10024 Color image
• CPC Class Codes: G06T 2207/30128 Food products
• CPC Class Codes: G06T 7/001 using an image reference approach
• CPC Class Codes: G06V 10/56 relating to colour
• CPC Class Codes: G07C 3/14 Quality control systems

Abstract

Described herein is an automated quality inspection station for evaluating color component characteristics of a product. The inspection station includes a color video camera, for capturing video frames of product images, and a control system for analyzing those video frames. The control system is programmed to perform a reference calibration and then a sample calibration. During the reference calibration an operator identifies component type areas from a displayed reference frame of a typical product assortment. The control system calculates color value density curves from the identified areas. The density curves are then calibrated to each other by scaling each of the density curves by a scaling factor. The scaling factors can either be provided directly by the operator or default values can be calculated by the control system. Default scaling factor values are calculated by summing the product of the corresponding density curve and an overall histogram of a sample video frame over a range of color values. Individual pixels are classified as one of a plurality of component types according to the highest calibrated density curve at the pixels'"'"' color value.

Claims

1. An automated quality inspection device for evaluating optical component characteristics of a product, the inspection device comprising:

means for capturing video frames of product images, each video frame comprising an array of color values, each color value being specified by at least one variable;

component selection means for allowing an operator to identify portions of individual component types within a reference video frame;

control means for (1) deriving relative component type probability curves from the identified portions of the individual component types, and (2) classifying individual color values as single component types according to the highest relative component type probability for each individual color value;

wherein the control means includes processing means for deriving the relative component type probability curves by comparing color values of the identified portions of component types to the color values of an overall sample video frame.

2. The quality inspection device of claim 1, wherein each color value is specified by at least two variables.

3. The quality inspection device of claim 1, wherein the control means includes a look-up table for storing the component type classifications of individual color values.

4. The quality inspection device of claim 3, further comprising means for displaying segmented representations of the product images, wherein each image is segmented according to the component type classifications of its color values.

5. An automated quality inspection device for evaluating optical component characteristics of a product, the inspection device comprising:

means for capturing video frames of product images, each video frame comprising an array of color values, each color value being specified by at least one variable;

.

component selection means for allowing an operator to identify portions of individual component types within a reference video frame;

control means for (1) deriving relative component type probability curves from the identified portions of the individual component types, and (2) classifying individual color values as single component according to the highest relative component type probability for each individual color value;

wherein the control means includes processing means for;

compiling reference histograms of color values occurring within the identified portions of the reference video frame;

calculating color value density curves from the reference histograms; and

scaling the density curves by scaling factors to obtain the relative component type probability curves.

6. The quality inspection device of claim 5, wherein the control means includes a look-up table for storing the component type classifications of individual color values.

7. The quality inspection device of claim 5, further comprising means for obtaining the scaling factors from an operator.

8. The quality inspection device of claim 5, further comprising:

means for obtaining the scaling factors from an operator; and

means for displaying segmented representations of a sample video frame to the operator;

processing means for segmenting the sample video frame according to the component type classifications of its color values, the displayed segmented representations being updated in response to the operator providing new scaling factors.

9. The quality inspection device of claim 5, wherein the control means includes processing means for calculating the density curves by fitting a continuous function to each reference histogram.

10. The quality inspection device of claim 5, wherein the control means includes processing means for calculating the density curves by fitting a set of gaussian-weighted Hermite polynomials to each reference histogram.

11. An automated quality inspection device for evaluating optical component characteristics of a product, the inspection device comprising:

means for capturing video frames of product images, each video frame comprising an array of color values, each color value being specified by at least one variable;

component selection means for allowing an operator to identify portions of individual component types within a reference video frame;

control means for (1) deriving relative component type probability curves from the identified portions of the individual component types, and (2) classifying individual color values as single component types according to the highest relative component type probability for each individual color value;

wherein the control means has processing means for;

compiling reference histograms of color values occurring within the identified portions of the reference video frame;

calculating color value density curves from the reference histograms;

compiling an overall histogram of color values within a sample video frame;

comparing the density curves to the overall histogram to obtain a scaling factor corresponding to each density curve; and

scaling the density curves by the scaling factors to obtain the relative component type probability curves.

12. The quality inspection device of claim 11, further comprising:

means for obtaining probability multipliers corresponding to the relative probability curves from an operator;

the processing means being further for adjusting each density scaling factor by the corresponding probability multiplier before scaling the density curves.

13. The quality inspection device of claim 11, wherein the processing means includes a look-up table for storing the component type classifications of individual color values.

14. The quality inspection device of claim 11, wherein the processing means calculates the density curves by fitting a set of gaussian-weighted Hermite polynomials to each reference histogram.

15. The quality inspection device of claim 11, further comprising:

means for obtaining probability multipliers from an operator;

the processing means being further for adjusting each density scaling factor by the corresponding probability multiplier before scaling the density curves; and

means for displaying segmented representations of the sample video frame to the operator, wherein the sample frame is segmented according to the component type classifications of its color values, the segmented representations being updated in response to the operator providing new probability multipliers.

16. The quality inspection device of claim 11, wherein the control means obtains each scaling factor by summing the product of the corresponding density curve and the overall histogram over a range of color values.

17. The quality inspection device of claim 16, further comprising means for obtaining probability multipliers corresponding to the relative probability curves from an operator, the processing means being further for adjusting each density scaling factor by the corresponding probability multiplier before scaling the density curves.

18. An automated color component classification device for identifying component types in a sample product having a plurality of component types, the inspection device comprising:

a sample surface for supporting product which is to be inspected;

a camera positioned relative to the optical inspection surface to produce video flames of the supported sample product, each video frame containing an array of pixels, wherein each pixel has an associated color value which is specified by at least two variables;

a video display monitor which displays video frames to an operator;

component selection means for allowing an operator to identify portions of individual component types within a reference video frame;

a data processor operably connected to the camera, the video display monitor, and the component selection means, the data processor being programmed to derive relative component type probability curves from the identified portions of the individual component types, and to classify individual color values as single component types according to the highest relative component type probability for each color value;

wherein the data processor is programmed to derive the relative component type probability curves by comparing color values of the identified portions of component types to the color values of an overall sample video frame.

19. The component classification device of claim 18, further comprising a memory look-up table for storing the component type classifications of individual color values.

20. The component classification device of claim 19, further comprising means for displaying segmented sample video frames, wherein each sample video frame is segmented according to the component type classifications of its color values.

21. An automated color component classification device for identifying component types in a sample product having a plurality of component types, the inspection device comprising:

a sample surface for supporting product which is to be inspected;

a camera positioned relative to the optical inspection surface to produce video frames of the supported sample product, each video frame containing an array of pixels, wherein each pixel has an associated color value which is specified by at least two variables;

a video display monitor which displays video frames to an operator;

component selection means for allowing an operator to identify portions of individual component types within a reference video frame;

a data processor operably connected to the camera, the video display monitor, and the component selection means, the data processor being programmed to derive relative component type probability curves from the identified portions of the individual component types, and to classify individual color values as single component types according the highest relative component type probability for each color value;

wherein the data processor is further programmed to;

compile reference histograms of color values occurring within the identified portions of the reference video frame;

calculate color value density curves from the reference histograms; and

scale the density curves by scaling factors to obtain the relative component type probability curves.

22. The component classification device of claim 21, further comprising a memory look-up table for storing the component type classifications of individual color values.

23. The component classification device of claim 21, further comprising means for obtaining the scaling factors from an operator.

24. The component classification device of claim 21, further comprising:

means for obtaining the scaling factors from an operator; and

means for displaying segmented representations of sample video frames;

wherein the sample video frame is segmented according to the component type classifications of its color values, the displayed segmented representations being updated in response to the operator providing new scaling factors.

25. The component classification device of claim 21, wherein the data processor calculates the density curves by fitting a set of gaussian-weighted Hermite polynomials to each reference histogram.

26. An automated color component classification device for identifying Component types in a sample product having a plurality of component types, the inspection device comprising:

a sample surface for supporting product which is to be inspected;

a camera positioned relative to the optical inspection surface to produce video frames of the supported sample product, each video frame containing an array of pixels, wherein each pixel has an associated color value which is specified by at least two variables;

a video display monitor which displays video frames to an operator;

component selection means for allowing an operator to identify portions of individual component types within a reference video frame;

a data processor operably connected to the camera, the video display monitor, and the component selection means, the data processor being programmed to derive relative component type probability curves from the identified portions of the individual component types, and to classify individual color values as single component types according the highest relative component type probability for each color value;

wherein the data processor is further programmed to;

compile reference histograms of color values occurring within the identified portions of the reference video frame;

calculate color value density curves from the reference histograms;

compile an overall histogram of color values within a sample video frame;

compare the density curves to the overall histogram to obtain a scaling factor corresponding to each density curve; and

scale the density curves by the scaling factors to obtain the relative component type probability curves.

27. The component classification device of claim 26, further comprising:

means for obtaining probability multipliers corresponding to the relative probability curves from an operator;

the data processor being further programmed to adjust each density scaling factor by the corresponding probability multiplier before scaling the density curves.

28. The component classification device of claim 26, further comprising a memory look-up table for storing the component type classifications of individual color values.

29. The component classification device of claim 26, wherein the data processor calculates the density curves by fitting a set of gaussian-weighted Hermite polynomials to each reference histogram.

30. The component classification device of claim 26, further comprising:

means for obtaining probability multipliers from an operator;

the data processor being further programmed to adjust each density scaling factor by the corresponding probability multiplier before scaling the density curves; and

means for displaying segmented representations of the sample video frame to the operator, wherein the sample video frame is segmented according to the component type classifications of its color values, the segmented representations being updated in response to the operator providing new probability multipliers.

31. The component classification device of claim 26, wherein the data processor is programmed to obtain each scaling factor by summing the product of the corresponding density curve and the overall histogram over a range of color values.

32. The component classification device of claim 31, further comprising means for obtaining probability multipliers corresponding to the relative probability curves from an operator, the data processor being further programmed to adjust each density scaling factor by the corresponding probability multiplier before scaling the density curves.

33. An automated quality inspection device for evaluating color component characteristics of a product, the inspection device comprising:

an optical transducer which produces a video signal representative of color characteristics of the product;

a frame grabber which receives the video signal and captures video frames representing images of the product, each video frame containing an array of pixels, each pixel having an associated color value which is specified by at least two variables;

a video display upon which a reference video frame is displayed to an operator;

component selection means for allowing the operator to identify portions of component type areas from the displayed reference video frame;

a data processor operably connected to read pixel color values from the frame grabber and programmed to;

(a) compile reference histograms of color values within the identified portions of the reference frame component type areas;

(b) calculate color value density curves from the reference histograms;

(c) calibrate the density curves relative to each other;

(d) classify an individual pixel as the component type having the highest calibrated density curve value at the color value of the individual pixel;

wherein the data processor is programmed to calibrate the density curves by comparing them to an overall histogram of the color values occurring within an overall sample video frame.

34. The quality inspection device of claim 33, wherein the data processor includes a look-up table for storing component type classifications of individual color values.

35. An automated quality inspection device for evaluating color component characteristics of a product, the inspection device comprising:

an optical transducer which produces a video signal representative of color characteristics of the product;

a frame grabber which receives the video signal and captures video frames representing images of the product, each video frame containing an array of pixels, each pixel having an associated color value which is specified by at least two variables;

a video display upon which a reference video frame is displayed to an operator;

component selection means for allowing the operator to identify portions of component type areas from the displayed reference video frame;

a data processor operably connected to read pixel color values from the frame grabber and programmed to;

(a) compile reference histograms of color values within the identified portions of the reference frame component type areas;

(b) calculate color value density curves from the reference histograms;

(c) calibrate the density curves relative to each other;

(d) classify an individual pixel as the component type having the highest calibrated density curve value at the color value of the individual pixel;

wherein the data processor is programmed to calibrate the density curves by scaling each density curve by an operator-supplied scaling factor.

36. An automated quality inspection device for evaluating color component characteristics of a product, the inspection device comprising:

an optical transducer which produces a video signal representative of color characteristics of the product;

a frame grabber which receives the video signal and captures video frames representing images of the product, each video frame containing an array of pixels, each pixel having an associated color value which is specified by at least two variables;

a video display upon which a reference video frame is displayed to an operator;

component selection means for allowing the operator to identify portions of component type areas from the displayed reference video frame;

a data processor operably connected to read pixel color values from the frame grabber and programmed to;

(a) compile reference histograms of color values within the identified portions of the reference frame component type areas;

(b) calculate color value density curves from the reference histograms;

(c) calibrate the density curves relative to each other;

(d) classify an individual pixel as the component type having the highest calibrated density curve value at the color value of the individual pixel;

means for obtaining scaling factors from an operator;

wherein the data processor is programmed to calibrate the density curves by scaling the density curves by the scaling factors; and

wherein the data processor is programmed to display segmented representations of sample video frames on the video display, each sample video frame being segmented according to the component type classifications of its pixels, the data processor being programmed to update the displayed segmented representations in response to the operator providing new scaling factors.

37. An automated quality inspection device for evaluating color component characteristics of a product, the inspection device comprising:

an optical transducer which produces a video signal representative of color characteristics of the product;

a frame grabber which receives the video signal and captures video frames representing images of the product, each video frame containing an array of pixels, each pixel having an associated color value which is specified by at least two variables;

a video display upon which a reference video frame is displayed to an operator;

component selection means for allowing the operator to identify portions of component type areas from the displayed reference video frame;

a data processor operably connected to read pixel color values from the frame grabber and programmed to;

(a) compile reference histograms of color values within the identified portions of the reference frame component type areas;

(b) calculate color value density curves from the reference histograms;

(c) calibrate the density curves relative to each other;

(d) classify an individual pixel as the component type having the highest calibrated density curve value at the color value of the individual pixel;

wherein the data processor is programmed to calculate the density curves by fitting a set of gaussian-weighted Hermite polynomials to each reference histogram.

38. An automated quality inspection device for evaluating color component characteristics of a product, the inspection device comprising:

an optical transducer which produces a video signal representative of color characteristics of the product;

a frame grabber which receives the video signal and captures video frames representing images of the product, each video frame containing an array of pixels, each pixel having an associated color value which is specified by at least two variables;

a video display upon which a reference video frame is displayed to an operator;

component selection means for allowing the operator to identify portions of component type areas from the displayed reference video frame;

a data processor operably connected to read pixel color values from the frame grabber and programmed to;

(a) compile reference histograms of color values within the identified portions of the reference frame component type areas;

(b) calculate color value density curves from the reference histograms;

(c) calibrate the density curves relative to each other;

(d) classify an individual pixel as the component type having the highest calibrated density curve value at the color value of the individual pixel;

wherein the data processor is programmed to calibrate the density curves by;

compiling an overall histogram of color values within a sample video frame;

comparing the density curves to the overall histogram to obtain a density scaling factor corresponding to each density curve; and

scaling the density curves by their corresponding density scaling factors to obtain corresponding relative probability curves which represent the probability of any single color value occurring in any single component type relative to any other component types.

39. The quality inspection device of claim 38, further comprising:

means for obtaining probability multipliers corresponding to the relative probability curves from an operator;

the data processor being further programmed to adjust each density scaling factor by the corresponding probability multiplier before scaling the density curves.

40. The quality inspection device of claim 38, wherein the data processor includes a look-up table for storing component type classifications of individual color values.

41. The quality inspection device of claim 38, wherein the data processor is programmed to calculate the density curves by fitting a set of gaussian-weighted Hermite polynomials to each reference histogram.

42. The quality inspection device of claim 38, further comprising:

means for obtaining probability multipliers from an operator; and

means for displaying segmented representations of a sample video frame to the operator, wherein the sample video frame is segmented according to the component type classifications of its color values, the segmented representations being updated in response to the operator providing new probability multipliers.

43. The quality inspection device of claim 38, wherein the data processor is programmed to obtain each density scaling factor by summing the product of the corresponding density curve and the overall histogram over a range of color values.

44. The quality inspection device of claim 43, further comprising means for obtaining probability multipliers corresponding to the relative probability curves from an operator, the data processor being programmed to adjust each density scaling factor by the corresponding probability multiplier before scaling the density curves.

45. A method of classifying video pixels in a product inspection device as one of a plurality of component types, wherein each pixel has an associated color value, each color value being specified by at least one variable, the method comprising the following steps:

capturing a reference video frame of a product image, the reference video frame comprising an array of video pixels;

identifying portions of individual component types within the reference video frame;

deriving relative component type probability curves from the identified portions of the individual component types; and

classifying individual pixels as single component types according to the highest relative component type probability for each individual pixel'"'"'s color value;

wherein the step of deriving the relative component type. probability curves comprises comparing color values of the identified portions of component types to the color values of an overall sample video frame.

46. The method of claim 45, wherein each color value is specified by at least two variables.

47. The method of claim 45, further comprising the following additional steps:

classifying individual color values as individual component types according to their highest relative probability; and

storing the component type classifications of individual color values in a look-up table.

48. The method of claim 45, further comprising the following additional steps:

segmenting a video frame based on the classifications of its pixels; and

displaying the segmented video frame.

49. A method of classifying video pixels in a product inspection device as one of a plurality of component types, wherein each pixel has an associated color value, each color value being specified by at least one variable, the method comprising the following steps:

capturing a reference video frame of a product image, the reference video frame comprising an array of video pixels;

identifying portions of individual component types within the reference video frame;

deriving relative component type probability curves from the identified portions of the individual component types;

classifying individual pixels as single component types according to the highest relative component type probability for each individual pixel'"'"'s color value;

compiling reference histograms of color values occurring within the identified portions of the reference video frame;

calculating color value density curves from the reference histograms; and

scaling the density curves by scaling factors to obtain the relative component type probability curves.

50. The method of claim 49, further comprising the following additional steps:

classifying individual color values as individual component types according to their highest relative probability; and

storing the component type classifications of individual color values in a look-up table.

51. The method of claim 49, further comprising the additional step of obtaining the scaling factors from an operator.

52. The method of claim 49, further comprising the following additional steps:

obtaining the scaling factors from an operator;

segmenting the video frame based on the classifications of its pixels;

displaying the segmented video frame; and

updating the segmented video frame display in response to the operator providing new scaling factors.

53. The method of claim 49, wherein the step of calculating color value density curves from the reference histograms comprises fitting a set of gaussian-weighted Hermite polynomials to each reference histogram.

54. A method of classifying video pixels in a product inspection device as one of a plurality of component types, wherein each pixel has an associated color value, each color value being specified by at least one variable, the method comprising the following steps:

capturing a reference video frame of a product image, the reference video frame comprising an array of video pixels;

identifying portions of individual component types within the reference video frame;

deriving relative component type .probability curves from the identified portions of the individual component types;

classifying individual pixels as single component types according to the highest relative component type probability for each individual pixel'"'"'s color value;

compiling reference histograms of color values occurring within the identified portions of the reference video frame;

calculating color value density curves from the reference histograms;

compiling an overall histogram of color values within a sample video frame;

comparing the density curves to the overall histogram to obtain a scaling factor corresponding to each density curve; and

scaling the density curves by scaling factors to obtain the relative component type probability curves.

55. The method of claim 54, further comprising the following additional steps:

obtaining probability multipliers corresponding to the relative probability curves from an operator; and

adjusting each density scaling factor by the corresponding probability multiplier before scaling the density curves.

56. The method of claim 54, further comprising the following additional steps:

classifying individual color values as individual component types according to their highest relative probability; and

storing the component type classifications of individual color values in a look-up table.

57. The method of claim 54, wherein the step of calculating color value density curves from the reference histograms comprises fitting a set of gaussian-weighted Hermite polynomials to each reference histogram.

58. The method of claim 54, further comprising the following additional steps:

obtaining probability multipliers corresponding to the relative probability curves from an operator;

adjusting each density scaling factor by the corresponding probability multiplier before scaling the density curves;

segmenting the sample video frame based on the classifications of its pixels;

displaying the segmented video frame; and

updating the segmented video frame display in response to the operator providing new probability multipliers.

59. The method of claim 54, wherein the step of comparing the density curves to the overall histogram comprises summing the product of the corresponding density curve and the overall histogram over a range of color values.

60. The method of claim 59, further comprising the following additional steps:

obtaining probability multipliers corresponding to the relative probability curves from an operator;

adjusting each density scaling factor by the corresponding probability multiplier before scaling the density curves.