Methods and apparatus for optically determining the acceptability of products

• US 5,189,708 A
• Filed: 12/17/1990
• Issued: 02/23/1993
• Est. Priority Date: 02/09/1989
• Status: Expired due to Term

• Current Assignee: Philip Morris USA Incorporated (Altria Group, Inc.)
• Original Assignee: Philip Morris USA Incorporated (Altria Group, Inc.)
• Inventors: Cox, Kenneth A., Maher, Robert J.
• Primary Examiner(s): Razavi, Michael
• Application Number:
  US07/634,110
• Time in Patent Office: 799 Days
• Field of Search: 382/8, 382/14, 382/15, 382/39, 382/1, 358/101, 358/106, 358/107, 364/552
• US Class Current: 382/143
• CPC Class Codes: B07C 5/10 measured by light-responsive means sorting according to colour B07C5/342
• CPC Class Codes: G01N 21/88 Investigating the presence of flaws or contamination
• CPC Class Codes: G01N 21/8851 Scan or image signal processing specially adapted therefor, e.g. for scan signal adjustment, for detecting different kinds of defects, for compensating for structures, markings, edges G01N21/8806 and G01N21/93 - G01N21/95692 take precedence; optical measurement of dimensions G01B11/00; optical scanning G02B26/10; image transformation G06T3/00; computerised image enhancement G06T5/00; image processing per se for flaw detection G06T7/0002
• CPC Class Codes: G06F 18/21 Design or setup of recognition systems or techniques; Extraction of features in feature space; Blind source separation
• CPC Class Codes: G06T 2207/30164 Workpiece; Machine component
• CPC Class Codes: G06T 7/001 using an image reference approach
• CPC Class Codes: G06V 10/70 using pattern recognition or machine learning optical pattern recognition or electronic computations therefor G06V10/88
• CPC Class Codes: G07C 3/14 Quality control systems

Abstract

The appearance of a product is determined to be acceptable or unacceptable by computing a discriminant function or image from a plurality of two-dimensional images of products of the kind to be inspected. A two-dimensional image of the product is then formed and processed using the discriminant image to produce an output value which can be compared to a predetermined value used in computing the discriminant image. If the output value deviates from the predetermined value by more than a predetermined amount, the product has been found to have an unacceptable appearance.

Claims

1. A method of determining whether or not a product has an acceptable appearance comprising the steps of:

generating a plurality of reference images from reference products, the image generation for each reference product including the steps of acquiring a two-dimensional image of the product surface, subdividing the acquired image into a plurality of sub-areas, and assigning a value to each sub-area dependent on the optical characteristic of the sub-area;

deriving from the images acquired from the reference products a discriminant image;

setting a threshold level on the amount by which a dot product of the discriminant image and an actual product whose appearance is to be determined may deviate from a reference dot product of the discriminant image and an acceptable product image;

generating an image foe an actual product by repeating the steps for generating each reference image; and

comparing the dot product of the discriminant image and the acquired product image to the reference dot product;

whereby images of actual products whose dot product deviations from the reference dot product are greater than the threshold level are determined to have an unacceptable appearance.

2. A method according to claim 1, wherein the reference image generation further comprises comparing the assigned values of successive sub-areas to detect substantial changes from one sub-area to the next, and emphasizing the values of sub-areas at or near detected substantial changes.

3. A method according to claim 2, wherein sub-areas which are not emphasized are de-emphasized.

4. A method according to claim 1, 2, or 3 wherein the reference image generation further comprises comparing values of the sub-areas with a threshold value and each sub-area having a value greater than the threshold value is assigned a first binary value and the remaining sub-areas are assigned a second binary value, and wherein the discriminant image is formed from the assigned binary values.

5. A method according to claim 1, wherein said step of generating an image for an actual product comprises the step of positioning said actual product in the field of view of a video camera.

6. A method according to claim 5, wherein said positioning step comprises the steps of:

conveying actual products one after another through the field of view of said video camera;

detecting when an actual product is at a predetermined location in said field of view of said video camera; and

utilizing the output signal of said video camera concurrent with detection that an actual product is at said predetermined location as said image of said actual product.

7. A method according to claim 1 further characterized by the step of physically separating an actual product determined to have an unacceptable appearance from an actual product determined to have an acceptable appearance.

8. Apparatus for determining whether or not a product has an acceptable appearance, characterized by means for generating a plurality of reference images from reference products, the reference image generation means including means for acquiring a two-dimensional image of the reference product surface, and means for sub-dividing the acquired image into a plurality of sub-areas and for assigning to each sub-area a value dependent on the optical characteristic of the sub-area;

means for deriving from the plurality of reference images a discriminant image;

means for setting a threshold level on the amount by which a dot product of the discriminant image and an actual product whose appearance is to be determined may deviate from a reference dot product of the discriminant image and an acceptable product image;

means for generating an image for an actual product, the actual product image generating means being operable to generate product images comparable with images generated by the reference product image generating mean; and

means for comparing the dot product of the discriminant image and the acquired product image with the reference dot product;

whereby images of actual products whose dot product deviations from the reference dot product are greater than the threshold level are determined to have an unacceptable appearance.

9. Apparatus according to claim 8, wherein the reference image generation means comprises means for comparing the assigned values of the sub-areas to detect substantial changes from one sub-area to the next, and means for emphasizing the values of sub-areas at or near detected substantial changes.

10. Apparatus according to claim 9, comprising means for de-emphasizing those sub-areas not emphasized.

11. Apparatus according to claim 8, 9, or 10, wherein the reference image generation means further comprises comparison means for comparing the assigned values of the sub-areas to a threshold value and for assigning to each sub-area having a value greater than the threshold value a first binary value and to the remaining sub-areas a second binary value, wherein the means for deriving the discriminant image are arranged to derive the discriminant image from the assigned binary value of the sub-areas.

12. Apparatus according to claim 8, 9, or 10, wherein the means for generating the reference images and the means for generating the actual product images comprise a common camera and image processing system.

13. Apparatus according to claim 8 wherein said means for generating an image for an actual product comprises:

a video camera; and

means for positioning said actual product in the field of view of said video camera.

14. Apparatus according to claim 13 further comprising:

means for illuminating said actual product in the field of view of said video camera.

15. Apparatus according to claim 13 wherein said means for positioning comprises:

a conveyor for conveying actual products one after another though the field of view of said video camera.

16. Apparatus according to claim 13 wherein said means for positioning comprises:

means for conveying actual products one after another through the field of view of said video camera;

means for detecting when an actual product being conveyed by said means for conveying is at a predetermined location in said field of view of said video camera; and

means responsive to said means for detecting for utilizing the output signal of said video camera concurrent with detection that an actual product is at said predetermined location as said image of said actual product.

17. Apparatus according to claim 8 further characterized by:

means for physically separating an actual product determined to have an unacceptable appearance from an actual product determined to have an acceptable appearance.

48. Apparatus according to claim 11 wherein the means for generating the reference images and the means for generating the actual product images comprise a common camera and image processing system.

18. The method of determining whether or not a surface of an object has substantially the same appearance as the corresponding surface of a plurality of training objects, said method comprising the steps of:

for each training object, (a) forming a two-dimensional image of said surface, (b) subdividing said two-dimensional image into a plurality of pixels, (c) digitizing each of said pixels by associating with each pixel an output digital value proportional to an optical characteristic of the associated pixel, and (d) binarizing each of said output digital values by associating a first binary value with each of said output digital values which is on one side of a first predetermined threshold value and by associating a second binary value with each of said output digital values which is on the other side of said first predetermined threshold value;

associating a discriminant value with each of said pixels, each discriminant value being chosen so that, for each training object, the dot product of the discriminant values and the binary values is approximately a predetermined nonzero constant value;

repeating steps (a)-(d) for said object; and

comparing the dot product of said discriminant values and the binary values for said object to said predetermined nonzero constant value.

19. The method defined in claim 18 wherein said discriminant values collectively comprise a discriminant image F and wherein said method further comprises the steps of:

converting the binary values associated with said training objects to a plurality of eigenimages Φ

.sup.(j), all of said eigenimages being orthogonal to one another; and

forming said discriminant image from a truncated linear combination of said eigenimages.

20. The method defined in claim 19 wherein the binary values associated with each of said training objects i collectively comprise a training image I.sup.(i), and wherein said method further comprises the step of:

computing each of said eigenimages as in the equation;

##EQU10## where n is the total number of training images and the transformation matrix B is orthogonal.

21. The method defined in claim 20 wherein said step of forming said discriminant image from a truncated linear combination of said eigenimages comprises the step of:

computing said discriminant image as in the equation;

##EQU11## where s is an integer less than n.

22. The method defined in claim 21 further comprising the step of:

selecting the value of s for the equation in claim 21 which gives the minimum value for V(s), where V(s) is computed as in the equation;

##EQU12## where a_i is a nonzero constant for all good training images and zero for any bad training images.

23. The method defined in claim 22 further comprising the step of:

computing the values of C_j for the equation of claim 21 as in the equation;

##EQU13## where the eigenvalues γ

_j are the diagonal elements of a matrix G given by the equation AB=BG where A is the real, symmetric matrix given by the dot product of every possible training image pair, and the transformation matrix B is such that B·

B⁺ =B⁺ ·

B=1.

24. The method defined in claim 23 wherein said eigenimages are ordered such that γ

₁ >

γ

₂ >

. . . >

γ

_n.

25. The method defined in claim 18 wherein said discriminant values are additionally chosen so that the sum of the squares of the discriminant values is approximately minimized.

26. The method defined in claim 18 wherein said discriminant values are additionally chosen so that the sum of the discriminant values is approximately zero.

27. The method defined in claim 18 wherein said two-dimensional images of said training objects may not be in registration with one another, and wherein, prior to said step of associating a discriminant value with each of said pixels, said method further comprises the steps of:

for each training object, (e) computing the center of gravity of at least a portion of the associated two-dimensional image, and (f) shifting the associated binary values so that the centers of gravity associated with all of said training objects are approximately coincident.

28. The method defined in claim 27 wherein, after repeating steps (a)-(d) for said object, said method further comprises the steps of:

computing the center of gravity of at least a portion of the two-dimensional image associated with said object; and

shifting the binary values associated with said object so that the center of gravity associated with said object is approximately coincident with the approximately coincident centers of gravity associated with said training objects.

29. The method defined in claim 18 wherein a plurality of sample objects representative of the range of variation likely to occur among acceptable objects is available, and wherein, prior to said step of repeating steps (a)-(d) for said object, said method further comprises the steps of:

repeating steps (a)-(d) for each sample object;

computing a correlation value for each sample object by forming the dot product of said discriminant values and the binary values for said sample object;

computing the average of the correlation values for all of said sample objects;

computing the standard deviation of the correlation values for all of said sample objects; and

computing first and second threshold correlation values respectively equal to said average of the correlation values plus or minus predetermined numbers of said standard deviation.

30. The method defined in claim 29 wherein said step of comparing the dot product of said discriminant values and the binary values for said object to said predetermined nonzero constant value comprises the step of:

determining whether or not said dot product of said discriminant values and the binary values for said object is between said first and second threshold correlation values.

31. The method defined in claim 18 wherein at least one unacceptable object is available, and wherein said step of associating a discriminant value with each of said pixels comprises the steps of:

repeating steps (a)-(d) for said unacceptable object; and

additionally choosing said discriminant values so that the dot product of the discriminant values and the binary values associated with said unacceptable object is approximately zero.

32. The method defined in claim 18 wherein the number of pixels is substantially greater than the number of training objects.

33. Apparatus for determining whether or not a surface of an object has substantially the same appearance as the corresponding surface of a plurality of training objects comprising:

means for, for each training object, (a) forming a two-dimensional image of said surface, (b) subdividing said two-dimensional image into a plurality of pixels, (c) digitizing each of said pixels by associating with each pixel an output digital value proportional to an optical characteristic of the associated pixel, and (d) binarizing each of said output digital values by associating a first binary value with each of said output digital values which is on one side of a first predetermined threshold value;

means for associating a discriminant value with each of said pixels, each discriminant value being chosen so that, for each training object, the dot product of the discriminant values and the binary values is approximately a predetermined non-zero constant value;

means for repeating functions (a)-(d) with respect to said object; and

means for comparing the dot product of said discriminant values and the binary values for said object to said redetermined nonzero constant value.

34. The apparatus defined in claim 33 wherein said discriminant values collectively comprise a discriminant image F and wherein said apparatus further comprises:

means for converting the binary values associated with said training objects to a plurality of eigenimages Φ

.sup.(j), all of said eigenimages being orthogonal to one another; and

means for forming said discriminant image from a truncated linear combination of said eigenimages.

35. The apparatus defined in claim 34 wherein the binary values associated with each of said training objects i collectively comprise a training image I.sup.(i), and wherein said apparatus further comprises:

means for computing each of said eigenimages as in the equation;

##EQU14## where n is the total number of training images and the transformation matrix B is orthogonal.

36. The apparatus defined in claim 35 wherein said means for forming said discriminant image from a truncated linear combination of said eigenimages comprises:

means for computing said discriminant image as in the equation;

##EQU15## where s is an integer less than n.

37. The apparatus defined in claim 36 further comprising:

selecting the value of s for the equation of claim 36 which gives the minimum value for V(s), where V(s) is computed as in the equation;

##EQU16## where a_i is a nonzero constant for all good training images and zero for any bad training images.

38. The apparatus defined in claim 37 further comprising:

means for computing the values C_j for the equation of claim 36 as in the equation;

##EQU17## where the eigenvalues γ

_j are the diagonal elements of a matrix G given by the equation AB=BG, where A is the real, symmetric matrix, given by the dot product of every possible training image pair, and the transformation matrix B is such that B·

B⁺ =B⁺ ·

B=1.

39. The apparatus defined in claim 38 wherein said eigenimages are ordered such that γ

₁ >

γ

₂ >

. . . >

γ

_n.

40. The apparatus defined in claim 33 wherein said discriminant values are additionally chosen so that the sum of the squares of the discriminant values is approximately minimized.

41. The apparatus defined in claim 33 wherein said discriminant values are additionally chosen so that the sum of the discriminant values is approximately zero.

42. The apparatus defined in claim 33 wherein said two-dimensional images of said training objects may not be in registration with one another, and wherein, prior to operation of said means for associating a discriminant value with each of said pixels, said apparatus further comprises:

means for, for each training object, (e) computing the center of gravity of at least a portion of the associated two-dimensional image, and (f) shifting the associated binary values so that the centers of gravity associated with all of said training objects are approximately coincident.

43. The apparatus defined in claim 42 wherein said means for repeating functions (a)-(d) with respect to said object comprises:

means for computing the center of gravity of at least a portion of the two-dimensional image associated with said object; and

means for shifting the binary values associated with said object so that the center of gravity associated with said object is approximately coincident with the approximately coincident centers of gravity associated with said training objects.

44. The apparatus defined in claim 33 wherein a plurality of sample objects representative of the range of variation likely to occur among acceptable objects is available, and wherein, prior to operation of said means for repeating functions (a)-(d) with respect to said object, said apparatus further comprises the steps of:

means for repeating functions (a)-(d) with respect to each sample object;

means for computing a correlation value for each sample object by forming the dot product of said discriminant values and the binary values for said sample object;

means for computing the average of the correlation values for all of said sample objects;

means for computing the standard deviation of the correlation values for all of said sample objects; and

means for computing first and second threshold correlation values respectively equal to said average of the correlation values plus or minus predetermined numbers of said standard deviation.

45. The apparatus defined in claim 44 wherein said means for comparing the dot product of said discriminant values and the binary values for said object to said predetermined nonzero constant value comprises:

means for determining whether or not said dot product of said discriminant values and the binary values for said object is between said first and second threshold correlation values.

46. The apparatus defined in claim 33 wherein at least one unacceptable object is available, and wherein said means for associating a discriminant value with each of said pixels comprises:

means for repeating functions (a)-(d) with respect to said unacceptable object; and

means for additionally choosing said discriminant values so that the dot product of the discriminant values and the binary values associated with said unacceptable object is approximately zero.

47. The apparatus defined in claim 33 wherein the number of pixels is substantially greater than the number of training objects.