Methods and apparatus for optically determining the acceptability of products

• US 5,146,510 A
• Filed: 02/27/1991
• Issued: 09/08/1992
• Est. Priority Date: 02/09/1989
• Status: Expired due to Term

• Current Assignee: Philip Morris Incorporated (Altria Group, Inc.)
• Original Assignee: Philip Morris Incorporated (Altria Group, Inc.)
• Inventors: Dante, Henry M., Cox, Kenneth A., Maher, Robert J.
• Primary Examiner(s): COUSO, JOSE L
• Application Number:
  US07/661,809
• Time in Patent Office: 559 Days
• Field of Search: 382/8, 382/30, 382/34, 358/101, 358/106, 358/107, 356/71
• US Class Current: 382/143
• CPC Class Codes: B07C 5/10 measured by light-responsive means sorting according to colour B07C5/342
• CPC Class Codes: G01N 21/88 Investigating the presence of flaws or contamination
• CPC Class Codes: G01N 21/8851 Scan or image signal processing specially adapted therefor, e.g. for scan signal adjustment, for detecting different kinds of defects, for compensating for structures, markings, edges G01N21/8806 and G01N21/93 - G01N21/95692 take precedence; optical measurement of dimensions G01B11/00; optical scanning G02B26/10; image transformation G06T3/00; computerised image enhancement G06T5/00; image processing per se for flaw detection G06T7/0002
• CPC Class Codes: G06F 18/21 Design or setup of recognition systems or techniques; Extraction of features in feature space; Blind source separation
• CPC Class Codes: G06T 2207/30164 Workpiece; Machine component
• CPC Class Codes: G06T 7/001 using an image reference approach
• CPC Class Codes: G06V 10/70 using pattern recognition or machine learning optical pattern recognition or electronic computations therefor G06V10/88
• CPC Class Codes: G07C 3/14 Quality control systems

Abstract

The acceptability of the appearance of objects such as consumer products is determined by forming an initial discriminant function as a composite of a relatively small number of images which are known to be acceptable. This initial discriminant function is then used to gather statistical information about how a first relatively large number of images compares to the initial discriminant function. Thereafter, this statistical information is used to select or aid in the selection of acceptable images from a second relatively large number of images, and the images selected as acceptable are used to refine the discriminant function. The refined discriminant function is then used (e.g., during actual product inspection) to determine which objects have an acceptable appearance and which do not.

Claims

1. The method of determining the acceptability of the appearance of each of a plurality of objects comprising the steps of:

forming an image of each of a first subplurality of said objects;

combining the images of said first subplurality to produce an initial discriminant function;

forming an image of each of a second subplurality of said objects;

using said initial discriminant function to process each of the images of said second subplurality in order to produce a processed value for each of said images from said second subplurality which is indicative of how similar said image from said second subplurality is to said initial discriminant function;

selecting first processed value thresholds between which a first majority of said processed values fall;

selecting second processed value thresholds between which a second majority of said processes values fall, said second majority being larger than said first majority;

forming an image of each of a third subplurality of said objects;

using selected images of said third subplurality to progressively refine said discriminant function by (1) using said discriminant function to process each successive image from said third subplurality in order to produce a processed value for said image from said third subplurality, (2) determining whether said processed value is substantially between said second value thresholds, and (a) if not, discarding said image from said third subplurality and skipping step (3) below, or (b) if so, determining whether said processed value is substantially between first processed value thresholds, and (i) if so, continuing with step (3) below, or (ii) if not, discarding said image from said third subplurality and skipping step (3) below unless said image from said third subplurality appears acceptable, and (3) modifying said discriminant function in accordance with said image from said third subplurality;

forming an image of each of a fourth subplurality of said objects; and

using the refined discriminant function to determine whether or not each of the images of said fourth subplurality has an acceptable appearance.

2. The method defined in claim 1 wherein adaptive training is used in said step of modifying said discriminant function in accordance with said image from said third subplurality.

3. The method defined in claim 1 further comprising the step of:

separating the objects in said fourth subplurality whose images have been determined to have an acceptable appearance from the objects in said fourth subplurality whose images have been determined to not have an acceptable appearance.

4. The method defined in claim 1 wherein said step of using said refined discriminant function comprises the steps of:

using the refined discriminant function to process each of the images of said fourth subplurality in order to produce a processed value for said image from said fourth subplurality; and

comparing said processed value to said first processed value thresholds and identifying said image from said fourth subplurality as having an acceptable appearance only if said processed value is substantially between first processed value thresholds.

5. The method defined in claim 1 wherein at least one of said steps of forming an image comprises the steps of:

subdividing the image being formed in said at least one of said steps of forming an image into a plurality of pixels, each of which has an initial digital value indicative of the appearance of the associated portion of the associated object;

processing said initial digital values to produce edge detected digital values which emphasize edges in said image being formed; and

converting each edge detected digital value which is greater than a predetermined value to one of first and second numerical values, and converting each edge detected digital value which is not greater than said predetermined value to the other of said first and second numerical values.

6. The method defined in claim 5 wherein said step of forming an image of each of said first subplurality of objects is performed using the steps set forth in claim 5, and wherein said step of combining the images of said first subplurality of objects comprises the step of:

producing said initial discriminant function as the logical OR of all of said images of said first subplurality of said objects.

7. The method defined in claim 1 wherein said step of using said initial discriminant function comprises the step of:

calculating the dot product of said initial discriminant function and each of the images of said second subplurality of objects.

8. The method defined in claim 1 wherein said first majority of said processed values is a subset of said second majority of said processed values.

9. The method defined in claim 1 wherein substep (1) comprises the step of:

calculating the dot product of said discriminant function and each successive image of an object from said third subplurality.

12. The method defined in claim 1 wherein at least one of said steps of forming an image of an object comprises the step of positioning said object in the field of view of a video camera.

13. The method defined in claim 12 wherein said positioning step comprises the steps of:

conveying said objects one after another through the field of view of said video camera;

detecting when an object is at a predetermined location in said field of view of said video camera; and

utilizing the output signal of said video camera concurrent with detection that an object is at said predetermined location as said image of said object.

41. The method defined in claim 5 wherein said numerical values are binary 0 and 1.

10. The method of processing a plurality of images to produce an image discriminant function comprising the steps of:

subdividing a first of said images into a plurality of pixels, each of which has an initial digital value proportional to a predetermined image characteristic of the associated portion of the associated image;

assigning a first numerical value to each pixel having an initial digital value which is on one side of a predetermined threshold value and assigning a second numerical value to each pixel having an initial digital value which is on the other side of said threshold value to produce an initial discriminant function;

subdividing each remaining image in said plurality of images into a plurality of pixels, each of which has an initial digital value proportional to said predetermined image characteristic of the associated portion of the associated image;

assigning said first numerical value to each pixel in each remaining image having an initial digital value which is on one side of said threshold value and assigning said second numerical value to each pixel in each remaining image having an initial digital value which is on the other side of said threshold value; and

producing said image discriminant function by sequentially performing the logical OR operation between each of said remaining images after assigning said numerical values and the initial discriminant function, the result of each logical OR operation replacing said initial discriminant function for the next performance of said logical OR operation.

11. The method defined in claim 10 wherein said subdividing step comprises the step of:

edge detecting each of said images so that said initial digital values emphasize edges in said image.

42. The method defined in claim 10 wherein said numerical values are binary 0 and 1.

14. Apparatus for determining the acceptability of the appearance of each of a plurality of objects comprising:

means for forming an image of each of a first subplurality of said objects;

means for combining the images of said first subplurality to produce an initial discriminant function;

means for forming an image of each of a second subplurality of said objects;

means for using said initial discriminant function to process each of the images of said second subplurality in order to produce a processed value for each of said images from said second subplurality which is indicative of how similar said image from said second subplurality is to said initial discriminant function;

means for selecting first processed value thresholds between which a first majority of said processed values fall;

means for selecting second processed value thresholds between which a second majority of said processed values fall, said second majority being larger than said first majority;

means for forming an image of each of a third subplurality of objects;

means for using selected images of said third subplurality to progressively refine said discriminant function by (1) using the discriminant function to process each successive image from said third subplurality in order to produce a processed value for said image from said third subplurality, (2) determining whether said processed value is substantially between said second processed value thresholds, and (a) if not, discarding said image from said third subplurality and skipping (3) below, or (b) if so, determining whether said processed value is substantially between said first processed value thresholds, and (i) if so, continuing with (3) below, or (ii) if not, discarding said image from said third subplurality and skipping (3) below unless said image from said third subplurality appears acceptable, and (3) modifying said discriminant function in accordance with said image from said third subplurality;

means for forming an image of each of a fourth subplurality of said objects; and

means for using the refined discriminant function to determine whether or not each of the images of said fourth subplurality has an acceptable appearance.

15. The apparatus defined in claim 14 wherein said means for using selected images of said third plurality employs adaptive training in modifying said discriminant function in accordance with said image from said third subplurality.

16. The apparatus defined in claim 14 further comprising:

means for separating the objects in said fourth subplurality whose images have been determined to have an acceptable appearance from the objects in said fourth subplurality whose images have been determined to not have an acceptable appearance.

17. The apparatus defined in claim 14 wherein said means for using said refined discriminant function comprises:

means for using the refined discriminant function to process each of the images of said fourth subplurality in order to produce a processed value for said image from said fourth subplurality; and

means for comparing said processed value to said first processed value thresholds and identifying said image from said fourth subplurality as having an acceptable appearance only if said processed value is substantially between said first processed value thresholds.

18. The apparatus defined in claim 14 wherein at least one of said means for forming an image comprises:

means for subdividing the image being formed by said at least one of said means for forming an image into a plurality of pixels, each of which has an initial digital value indicative of the appearance of the associated object;

means for processing said initial digital values to produce edge detected digital values which emphasize edges in said image being formed; and

means for converting each edge detected digital value which is greater than a predetermined value to one of first and second numerical values, and converting each edge detected digital value which is not greater than said predetermined value to the other of said first and second numerical values.

19. The apparatus defined in claim 18 wherein said means for forming an image of each of said first subplurality of objects comprises the means set forth in claim 18, and wherein said means for combining the images of said first subplurality of objects comprises:

means for producing said initial discriminant function as the logical OR of all of said images of said first subplurality of said objects.

20. The apparatus defined in claim 14 wherein said means for using said initial discriminant function comprises:

means for calculating the dot product of said initial discriminant function and each of the images of said second subplurality of objects.

21. The apparatus defined in claim 14 wherein said first majority of said processed values is a subset of said second majority of said processed values.

22. The apparatus defined in claim 14 wherein said means for using selected images of said third subplurality employs means for calculating the dot product of said discriminant function and each successive image from said third subplurality in order to produce said processed value for said image from said third subplurality.

23. The apparatus defined in claim 14 wherein at least one of said means for forming an image comprises:

a video camera; and

means for positioning an object in the field of view of said video camera.

24. The apparatus defined in claim 23 further comprising:

means for illuminating said object in the field of view of said video camera.

25. The apparatus defined in claim 23 wherein said means for positioning comprises:

a conveyor for conveying said objects one after another through the field of view of said video camera.

26. The apparatus defined in claim 23 wherein said means for positioning comprises:

means for conveying said objects one after another through the field of view of said video camera;

means for detecting when an object being conveyed by said means for conveying is at a predetermined location in said field of view of said video camera; and

means responsive to said means for detecting for utilizing the output signal of said video camera concurrent with detection that an object is at said predetermined location as said image of said object.

43. The apparatus defined in claim 18 wherein said numerical values are binary 0 and 1.

44. The apparatus defined in claim 17 wherein said numerical values are binary 0 and 1.

27. Apparatus for processing a plurality of images to produce an image discriminant function comprising:

means for subdividing a first of said images into a plurality of pixels, each of which has an initial digital value proportional to a predetermined image characteristic of the associated portion of the associated image;

means for assigning a first numerical value to each pixel having an initial digital value which is on one side of a predetermined threshold value and assigning a second numerical value to each pixel having an initial digital value which is on the other side of said threshold value to produce an initial discriminant function;

means for subdividing each remaining image in said plurality of images into a plurality of pixels, each of which has an initial digital value proportional to said image characteristic of the associated portion of the associated image;

means for assigning said first numerical value to each pixel in each remaining image having an initial digital value which is on one side of said threshold value which is on one side of said threshold value and assigning said second numerical value to each pixel in each remaining image having an initial digital value which is on the other side of said threshold value; and

means for producing said image discriminant function by sequentially performing the logical OR operation between each of said remaining images after assigning said numerical values and the initial discriminant function, the result of each logical OR operation replacing said initial discriminant function for the next performance of said logical OR operation.

28. The apparatus defined in claim 27 wherein said means for subdividing comprises:

means for edge detecting each of said images so that said initial digital values emphasize edges in said image.

29. A method of determining the acceptability of a product, comprising the steps of:

generating a filter from a first set of acceptable images of said product;

comparing said filter with a member of a second set of images of said product to produce a processed value, said second set including at least some acceptable images;

repeating said comparing step to produce a distribution of processed values;

generating from said distribution of processed values a first range (A) of processed values, said first range comprising processed values associated with acceptable images of said product;

generating from said distribution of processed values a second range (B) of processed values, said second range comprising processed values associated with unacceptable images of said product;

comparing said filter with a further image of said product so as to produce a further processed value;

generating an indication of acceptability or unacceptability of said further image by comparing said further processed value to said first and second ranges;

if said further processed value is outside both said first and second ranges, selecting said further image only if said further image is acceptable; and

adaptively training said filter with said selected further image to produce a modified filter such that comparing said modified filter with said selected further image produces a modified processed value which is within said first range.

30. The method as claimed in claim 29, wherein said step of generating a filter includes combining said first set of acceptable images using a logical OR operation.

31. The method as claimed in claim 29, wherein said adaptively training step is analogous to a Widrow-Hoff training algorithm.

32. The method as claimed in claim 29, wherein said selecting step includes manually selecting according to an appearance of said product associated with said outside processed value.

33. The method as claimed in claim 29, further comprising the step of:

if said further processed value is in said first range, adaptively training said filter with said further image.

34. Apparatus for determining the acceptability of a product, comprising:

means for generating a filter from a first set of acceptable images of said product;

means for comparing said filter with a member of a second set of images of said product to produce a processed value, said second set including at least some acceptable images;

means for repeating said comparing function to produce a distribution of processed values;

means for generating from said distribution of processed values a first range (A) of processed values, said first range comprising processed values associated with acceptable images of said product;

means for generating from said distribution of processed values a second range (B) of processed values, said second range comprising processed values associated with unacceptable images of said product;

means for comparing said filter with a further image of said product so as to produce a further processed value;

means for generating an indication of acceptability or unacceptability of said further image by comparing said further processed value to said first and second ranges;

if said further processed value is outside first and second ranges, means for selecting said further image only if said further image is acceptable; and

means for adaptively training said filter with said selected further image to produce a modified filter such that comparing said modified filter with said selected further image produces a modified processed value which is within said first range.

35. The apparatus as claimed in claim 34, wherein said means for generating a filter includes means for combining said first set of acceptable images using a logical OR operation.

36. The apparatus as claimed in claim 34, wherein said means for adaptively training performs a function analogous to a Widrow-Hoff training algorithm.

37. The apparatus as claimed in claim 34, wherein said means for selecting includes means for allowing manual selection according to an appearance of said product associated with said outside processed value.

38. The apparatus as claimed in claim 34, further comprising:

if said further processed value is in said first range, means for adaptively training said filter with said further image.

39. The method of determining whether or not a sample image is substantially similar to a predetermined standard image comprising the steps of:

(a) forming a discriminant function which is representative of said standard image;

(b) comparing each of a first plurality of possible examples of said sample image to said discriminant function to produce a first processed value for each of said first plurality of examples of said sample image, each of said processed values being indicative of how similar the associated example is to said standard image;

(c) comparing a further possible example of said sample image to said discriminant function to produce a further processed value indicative of how similar said further example of said sample image is to said standard image;

(d) using said first processed values to at least partly determine whether said further possible example of said sample image is substantially similar to said standard image, and if so, modifying said discriminant function in accordance with said further example of said sample image;

(e) repeating steps (c) and (d) for each of a plurality of further possible examples of said sample image to progressively modify said discriminant function; and

(f) comparing said sample image to the modified discriminant function to produce a sample processed value indicative of how similar said sample image is to said standard image.

40. Apparatus for determining whether or not a sample image is substantially similar to a predetermined standard image comprising:

means for forming a discriminant function which is representative of said standard image;

means for comparing each of a first plurality of possible examples of said sample image to said discriminant function to produce a first processed value for each of said first plurality of examples of said sample image, each of said processed values being indicative of how similar the associated example is to said standard image;

means for comparing a further possible example of said sample image to said discriminant function to produce a further processed value indicative of how similar said further example of said sample image is to said standard image;

means for using said first processed values to at least partly determine whether said further possible example of said sample image is substantially similar to said standard image, and of so, modifying said discriminant function in accordance with said further example of said sample image;

means for repeating the two preceding functions for each of a plurality of further possible examples of said sample image to progressively modify said discriminant function; and

means for comparing said sample image to the modified discriminant function to produce a sample processed value indicative of how similar said sample image is to said standard image.