Systems and methods for classification and alignment of highly similar or self-similar patterns

• US 9,639,781 B2
• Filed: 04/10/2015
• Issued: 05/02/2017
• Est. Priority Date: 04/10/2015
• Status: Active Grant

• Current Assignee: Cognex Corporation
• Original Assignee: Cognex Corporation
• Inventors: Barker, Simon
• Primary Examiner(s): MARIAM, DANIEL G
• Application Number:
  US14/683,948
• Publication Number:
  US 20160300125A1
• Time in Patent Office: 753 Days
• Field of Search: 382141, 382151, 382152, 382159, 382224, 382294
• CPC Class Codes: G06F 18/214 Generating training patterns; Bootstrap methods, e.g. bagging or boosting
• CPC Class Codes: G06F 18/24133 Distances to prototypes
• CPC Class Codes: G06T 2207/20081 Training; Learning
• CPC Class Codes: G06T 2207/30164 Workpiece; Machine component
• CPC Class Codes: G06T 7/0004 Industrial image inspection
• CPC Class Codes: G06T 7/001 using an image reference approach
• CPC Class Codes: G06V 10/454 Integrating the filters into a hierarchical structure, e.g. convolutional neural networks [CNN]
• CPC Class Codes: G06V 10/764 using classification, e.g. of video objects
• CPC Class Codes: G06V 10/774 Generating sets of training patterns; Bootstrap methods, e.g. bagging or boosting
• CPC Class Codes: G06V 20/00 Scenes; Scene-specific elements control of digital cameras H04N23/60

Abstract

Systems and methods for training a machine vision system create geometric models. The disclosed methods can extract one or more corresponding features and one or more differentiating features from different sets of training images. The one or more differentiating features can be used to differentiate between the different work pieces. The disclosed methods can generate an alignment model using the corresponding features and a classification model using the one or more differentiating features.

Claims

1. A method for training a machine vision system to create geometric models, the method comprising the steps of:

obtaining a first set of training images of a first work piece and a second set of training images of a second work piece;

selecting at least one training image from one of the first set of training images and the second set of training images as an at least one baseline image;

training at least one baseline alignment model from the least one the baseline image;

registering the training images in the first set of training images not selected as the at least one baseline image using the at least one baseline alignment model to obtain a first set of relatives poses to the at least one baseline image for each training image in the first set of training images;

registering the training images in the second set of training images not selected as the at least one baseline image using the at least one baseline alignment model to obtain a second set of relatives poses to the at least one baseline image for each training image in the second set of training images;

identifying first corresponding features from the first set of training images;

identifying second corresponding features from the second set of training images;

identifying at least one shared feature among the first corresponding features and the second corresponding features;

extracting one or more differentiating features from the first set of training images and the second set of training images based on the first corresponding features, the second corresponding features, and the at least one shared feature among the first corresponding features and the second corresponding features, wherein the one or more differentiating features can be used to differentiate between the first work piece and the second work piece;

generating an alignment model using at least one of the first corresponding features, the second corresponding features, and the at least one shared feature; and

generating a classification model using the one or more differentiating features.

2. The method of claim 1, wherein the first corresponding features are stable between the training images in the first set and wherein the second corresponding features are stable between the training images in the second set.

3. The method of claim 2, wherein the first corresponding features and the second corresponding features are stable when identified in a number of training images greater than a threshold value.

4. The method of claim 1, wherein the first work piece is a work piece in a first pose and the second work piece is the work piece in a second pose.

5. The method of claim 1, wherein the first work piece is a different work piece than the second work piece.

6. The method of claim 1, wherein the first corresponding features comprise common features between the baseline image and the training images of the first set.

7. The method of claim 6, wherein the step of identifying the first corresponding features comprises:

extracting features from the training images of the first set of training images;

mapping the extracted features using the first set of relative poses; and

applying to the mapped features a correspondence metric to identify the first corresponding features.

8. The method of claim 1, wherein the second corresponding features comprise common features between the baseline image and the training images of the second set.

9. The method of claim 8, wherein the step of identifying the second corresponding features comprises:

extracting features from the training images of the second set of training images;

mapping the extracted features using the second set of relative poses; and

applying to the mapped features a correspondence metric to identify the second corresponding features.

10. The method of claim 1, wherein the step of identifying at least one shared feature comprises:

mapping the first corresponding features and the second corresponding features; and

applying to the mapped features a correspondence metric to identify the at least one shared feature.

11. The method of claim 1, wherein the at least one baseline image comprises a first baseline image from the first set of training images and a second baseline image from the second set of training images.

12. The method of claim 11, wherein the step of training at least one baseline alignment model comprises:

training a first baseline alignment model from the first baseline image; and

training a second baseline alignment model from the second baseline image.

13. The method of claim 12, wherein the step of registering the training images in the first set of training images comprises registering the training images in the first set of training images not selected as the first baseline image using the first baseline alignment model to obtain a first set of relatives poses to the first baseline image for each training image in the first set of training images.

14. The method of claim 12, wherein the step of registering the training images in the second set of training images comprises registering the training images in the second set of training images not selected as the second baseline image using the second baseline alignment model to obtain a second set of relatives poses to the second baseline image for each training image in the second set of training images.

15. A method for training a machine vision system to create geometric models, the method comprising the steps of:

obtaining a set of training images depicting a pattern;

training a baseline alignment model from a first training image of the set of training images;

registering training images of the set of training images other than the first training image to obtain a first set of relative poses among the training images using the alignment model;

identifying common features from the training images by extracting features from each of the training images, mapping the extracted features using the first set of relative poses and applying to the mapped features a correspondence metric;

generating an alignment model for the pattern using a first subset of the common features derived from a first set of training images that fall in a first region of the pattern;

identifying a second region within the training images;

identifying a second subset of common features derived from a second set of training images that fall in the second region;

extracting one or more differentiating features from the first subset of common features and the second subset of common features, wherein the one or more differentiating features can be used to differentiate between a region containing a fiducial and the second region; and

generating a classification model from the one or more differentiating features.

16. The method of claim 15, wherein the step of identifying the second region comprises running the alignment model for the pattern over the training images and identifying a region where the alignment model for the pattern scores highly but the region does not contain an instance of the pattern.

17. A system for training a machine vision system to create geometric models comprising:

a camera configured to capture images; and

a processor in communication with the camera configured to;

obtain a first set of training images of a first work piece and a second set of training images of a second work piece;

select at least one training image from one of the first set of training images and the second set of training images as an at least one baseline image;

train at least one baseline alignment model from the least one the baseline image;

register the training images in the first set of training images not selected as the at least one baseline image using the at least one baseline alignment model to obtain a first set of relatives poses to the at least one baseline image for each training image in the first set of training images;

register the training images in the second set of training images not selected as the at least one baseline image using the at least one baseline alignment model to obtain a second set of relatives poses to the at least one baseline image for each training image in the second set of training images;

identify first corresponding features from the first set of training images;

identify second corresponding features from the second set of training images;

identify at least one shared feature among the first corresponding features and the second corresponding features;

extract one or more differentiating features from the first set of training images and the second set of training images based on the first corresponding features, the second corresponding features, and the at least one shared feature among the first corresponding features and the second corresponding features, wherein the one or more differentiating features can be used to differentiate between the first work piece and the second work piece;

generate an alignment model using at least one of the first corresponding features, the second corresponding features, and the at least one shared feature; and

generate a classification model using the one or more differentiating features.

18. A system for training a machine vision system to create geometric models comprising:

a camera configured to capture images; and

a processor in communication with the camera configured to;

obtain a set of training images depicting a pattern;

train a baseline alignment model from a first training image of the set of training images;

register training images of the set of training images other than the first training image to obtain a first set of relative poses among the training images using the alignment model;

identify common features from the training images by extracting features from each of the training images, mapping the extracted features using the first set of relative poses and applying to the mapped features a correspondence metric;

generate an alignment model for the pattern using a first subset of the common features derived from a first set of training images that fall in a first region of the pattern;

identify a second region within the training images;

identify a second subset of common features derived from a second set of training images that fall in the second region;

extract one or more differentiating features from the first subset of common features and the second subset of common features, wherein the one or more differentiating features can be used to differentiate between a region containing a fiducial and the second region; and

generate a classification model from the one or more differentiating features.