System and method for single image focus assessment

• US 7,668,388 B2
• Filed: 03/03/2005
• Issued: 02/23/2010
• Est. Priority Date: 03/03/2005
• Status: Active Grant

• Current Assignee: Mitutoyo Corporation
• Original Assignee: Mitutoyo Corporation
• Inventors: Bryll, Robert K.
• Primary Examiner(s): Mehta; Bhavesh M
• Assistant Examiner(s): LIEW, ALEX KOK SOON
• Application Number:
  US11/072,360
• Publication Number:
  US 20060204121A1
• Time in Patent Office: 1,818 Days
• Field of Search: None
• US Class Current: 382/255
• CPC Class Codes: G06F 18/21 Design or setup of recognition systems or techniques; Extraction of features in feature space; Blind source separation
• CPC Class Codes: G06F 18/25 Fusion techniques
• CPC Class Codes: G06T 2207/10056 Microscopic image
• CPC Class Codes: G06T 2207/30164 Workpiece; Machine component
• CPC Class Codes: G06T 2207/30168 Image quality inspection
• CPC Class Codes: G06T 7/0004 Industrial image inspection

Abstract

An image focus assessment method is provided that works reliably for images of a variety of relatively dissimilar workpieces or workpiece features. The focus assessment method is based on analysis of a single image (without the benefit of comparison to other images). The robustness of the focus assessment method is enhanced by the use of at least one classifier based on a plurality of focus classification features. In one application, a primary advantage of assessing focus from a single image is that an overall workpiece inspection time may be reduced by avoiding running an autofocus routine if an image is already in focus. In various embodiments, the focus assessment method may include an ensemble of classifiers. The ensemble of classifiers can be trained on different training data (sub)sets or different parameter (sub)sets, and their classification outcomes combined by a voting operation or the like, in order to enhance the overall accuracy and robustness of the focus assessment method.

Claims

1. A focus assessment method for determining a focus classification for a workpiece feature image, the focus classification usable to determine whether the image is sufficiently focused, such that a machine vision inspection system may perform useful machine vision inspection operations on the workpiece feature image, the method comprising:

using the machine vision inspection system to perform the steps comprising;

(a) acquiring a first image of a workpiece feature for which an image focus classification is to be determined;

(b) determining a plurality of focus classification features based on the first image, the focus classification features being abstract features included in a multi-parameter feature vector; and

(c) determining the focus classification based on processing the determined plurality of focus classification features using at least one classifier, wherein;

the at least one classifier comprises at least a first respective classifier that operates on a first respective plurality of classification features;

the focus assessment method does not require the acquisition of an additional image of the workpiece feature that is differently focused than the first image;

at least the first respective classifier is trained on at least a first respective set of training images;

the set of training images comprises respective images of a plurality of different workpieces, wherein the respective images include in-focus images and out-of-focus images; and

the respective images are assigned a respective desired focus classification prior to training the respective classifier.

2. The method of claim 1, wherein the at least one classifier comprises at least one classifier selected from a group of classifier types comprising a neural network classifier, a k-nearest neighbor rule classifier (k-NNR), a decision tree classifier, a Bayesian classifier, and a fuzzy logic classifier.

3. The method of claim 1, wherein the at least one classifier comprises a plurality of respective classifiers, the plurality of respective classifiers including the first respective classifier that operates on the first respective plurality of classification features.

4. The method of claim 3, wherein the plurality of respective classifiers includes at least two respective classifiers that operate on respective pluralities of classification features.

5. The method of claim 3, wherein:

each of the respective plurality of classifiers operates on at least one respective focus classification feature to determine a respective classifier outcome;

the respective classifier outcomes are combined to determine a multi-classifier outcome; and

the step of determining the focus classification comprises basing the focus classification on the multi-classifier outcome.

6. The method of claim 1, wherein the plurality of focus classification features comprises at least one edge strength type of focus classification feature.

7. The method of claim 6, wherein the at least one edge strength type of focus classification feature comprises an operation that suppresses the contribution of local edge strength values that do not reach a predetermined threshold value.

8. The method of claim 1, wherein the plurality of focus classification features comprises at least one focus classification feature that is insensitive to the spatial distribution of the pixel intensity variations in the image.

9. The method of claim 8, wherein the focus classification feature that is insensitive to the spatial distribution associated with the pixel intensity variations in the image is selected from a group of focus classification features that are image pixel intensity statistical distribution parameters.

10. The method of claim 9, wherein the group of focus classification features that are image pixel intensity statistical distribution parameters comprises a grayscale intensity variance, an intensity variance derived from converted color channel information, an average grayscale intensity, and an average intensity derived from converted color channel information.

11. The method of claim 8, the method further comprising determining the number of pixels included in at least some of the bins of an image pixel intensity histogram, wherein the focus classification feature that is insensitive to the spatial distribution associated with the pixel intensity variations in the image is determined based on the image pixel intensity histogram.

12. The method of claim 11, wherein the image pixel intensity histogram has at least 4 bins and at most 12 bins.

13. A machine vision inspection system comprising:

(a) a vision system components portion; and

(b) a control system portion comprising;

(i) an image focus assessment portion comprising an image focus classification portion and a classifier training portion;

(ii) a display device; and

(iii) a graphical user interface that can be displayed on the display device, the graphical user interface including at least one display area including control widgets operable to control operations of the classifier training portion, wherein the image focus assessment portion includes a routine operable for performing a single-image focus assessment method comprising;

determining a plurality of focus classification features based on a first image of a workpiece feature, the focus classification features being abstract features included in a multi-parameter feature vector, and

determining a focus classification for the first image based on processing the determined plurality of focus classification features using at least one classifier,

wherein the at least one classifier comprises at least a first respective classifier that operates on a first respective plurality of classification features and the focus assessment method does not require the acquisition of an additional image of the workpiece feature that is differently focused than the first image;

wherein the at least one display area including control widgets operable to control operations of the classifier training portion comprises a plurality of areas selected from a group comprising;

a display area for displaying training images, a display area operable for inputting a desired focus classification for a training image, a display area operable for activating training operations for a current classifier based on a current set of respective training images and their respective desired focus classifications, a display area operable for discontinuing training iterations for a current classifier, a display area for displaying a value indicative of a classification error rate of a current classifier during its training, the error rate based on the respective focus classifications of the respective training images by the current classifier in comparison to the respective desired focus classifications for the respective training images, and a display area for displaying a focus classification outcome value produced by a current classifier for a current training image; and

wherein the display area operable for inputting a desired focus classification for a training image is operable to input at least three levels of focus classification, the at least three levels of focus classification including a first classification level corresponding to the best in-focus classification that can be entered, a second classification level corresponding to the worst out-of-focus classification that can be entered, and at least one additional classification level between the best in-focus classification and the worst out-of-focus classification.

14. The machine vision inspection system of claim 13, wherein inputting a respective level of focus classification corresponds to inputting a respective numerical value indicative of the focus classification.

15. The machine vision inspection system of claim 14, wherein, in the routine operable for performing a single-image focus assessment method, the step of determining a focus classification for the first image based on processing the determined plurality of focus classification features using at least one classifier comprises:

determining a focus classification outcome value that can potentially vary over a continuous range of numerical values; and

comparing the determined focus classification outcome value to at least one numerical limit in order to determine the focus classification for the first image.

16. A recording medium that stores a control program, the control program executable on a computing device usable to receive data corresponding to images workpiece features for which an image focus classification is to be determined, the control program including instructions for performing a single-image focus assessment method, the single-image focus assessment method instructions comprising:

instructions for determining a plurality of focus classification features based on a first image of a workpiece feature for which an image focus classification is to be determined, the focus classification features being abstract features included in a multi-parameter feature vector; and

instructions for determining a focus classification based on processing the determined plurality of focus classification features using at least one classifier,

wherein;

the at least one classifier comprises at least a first respective classifier that operates on a first respective plurality of classification features;

the focus assessment method instructions do not require the acquisition of an additional image of the workpiece feature that is differently focused than the first image;

at least the first respective classifier is trained on at least a first respective set of training images;

the set of training images comprises respective images of a plurality of different workpieces, wherein the respective images include in-focus images and out-of-focus images; and

the respective images are assigned a respective desired focus classification prior to training the respective classifier.

17. A focus assessment method for determining a focus classification for a workpiece feature image, the focus classification usable to determine whether the image is sufficiently focused, such that a machine vision inspection system may perform useful machine vision inspection operations on the workpiece feature image, the method comprising:

using the machine vision inspection system to perform the steps comprising;

(a) acquiring a first image of a workpiece feature for which an image focus classification is to be determined;

(b) determining a plurality of focus classification features based on the first image, the focus classification features being abstract features included in a multi-parameter feature vector; and

(c) determining the focus classification based on processing the determined plurality of focus classification features using at least one classifier, wherein;

the at least one classifier comprises a plurality of respective classifiers, the plurality of respective classifiers including at least a first respective classifier that operates on a first respective plurality of classification features;

the focus assessment method does not require the acquisition of an additional image of the workpiece feature that is differently focused than the first image;

each of the respective plurality of classifiers operates on at least one respective focus classification feature to determine a respective classifier outcome;

the respective classifier outcomes are combined to determine a multi-classifier outcome; and

the step of determining the focus classification comprises basing the focus classification on the multi-classifier outcome.