System and method for identifying and locating instances of a shape under large variations in linear degrees of freedom and/or stroke widths

• US 9,305,235 B1
• Filed: 12/21/2007
• Issued: 04/05/2016
• Est. Priority Date: 12/21/2007
• Status: Active Grant

• Current Assignee: Cognex Corporation
• Original Assignee: Cognex Corporation
• Inventors: Bogan, Nathaniel Rockwood, Motoyama, Toshimitsu
• Primary Examiner(s): Vu, Kim
• Assistant Examiner(s): Vanchy, Jr., Michael
• Application Number:
  US12/004,527
• Time in Patent Office: 3,028 Days
• Field of Search: 382/167, 382/141, 382/147, 382/151, 382/152, 382/159, 382/175, 382/190, 382/199, 382/209, 382/259
• US Class Current: 1/1
• CPC Class Codes: G06F 18/28 Determining representative reference patterns, e.g. by averaging or distorting; Generating dictionaries
• CPC Class Codes: G06T 2207/20081 Training; Learning
• CPC Class Codes: G06T 2207/30164 Workpiece; Machine component
• CPC Class Codes: G06T 7/73 using feature-based methods
• CPC Class Codes: G06V 10/34 Smoothing or thinning of the pattern; Morphological operations; Skeletonisation
• CPC Class Codes: G06V 10/44 Local feature extraction by analysis of parts of the pattern, e.g. by detecting edges, contours, loops, corners, strokes or intersections; Connectivity analysis, e.g. of connected components
• CPC Class Codes: G06V 10/752 Contour matching
• CPC Class Codes: G06V 10/772 Determining representative reference patterns, e.g. averaging or distorting patterns; Generating dictionaries
• CPC Class Codes: G06V 2201/06 Recognition of objects for industrial automation

Abstract

A system and method for identifying and locating instances of a shape under large variations in linear degrees of freedom is provided. The invention extracts boundary segments from an image and generates potential match candidates for boundary models based on the extracted boundary segments. The invention also extracts skeletal segments from the obtained image. Potential match candidates are then generated for skeletal models based on the extracted skeletal segments. Once the match candidates for the boundary models and skeletal models have been generated, the present invention then corresponds the various boundary segments and outputs a best correspondence. The output from the present invention may be utilized by a user to signify acceptance of the various correspondences located by the present invention. These correspondences may then be utilized to train a machine vision system.

Claims

1. A method for automatically training a machine vision system prior to run-time of an industrial automation system that utilizes the machine vision system, the machine vision system comprising a digital data processor, the method comprising the steps of:

a) receiving at the digital data processor an image of an object worked upon in the industrial automation system, the image comprising a plurality of edges;

b) extracting, using the digital data processor, a set of boundary line segments from the image that coincide with at least one of the plurality of edges;

c) generating, using the digital data processor, one or more boundary match candidates by selecting at least one boundary model shape from a library of predefined boundary model shapes and comparing the at least one boundary model shape to the set of extracted boundary segments;

d) extracting, using the digital data processor, a set of internal line segments from the image that are remote from the plurality of edges;

wherein a boundary of the object is skeletonized to create the set of internal line segments;

e) generating, using the digital data processor, one or more internal match candidates by selecting at least one internal line model shape from a library of predefined internal line model shapes and comparing the at least one internal line model shape to the set of extracted internal line segments;

f) corresponding, using the digital data processor, the set of boundary line segments to the one or more boundary match candidates and the set of internal line segments to the one or more internal match candidates to identify a set of best correspondences for the object; and

g) prior to run-time of the industrial automation system, automatically training the machine vision system for using one or more of the set of best correspondences for the object.

2. The method of claim 1 wherein step d) further comprises identifying a stroke width associated with the object within the image.

3. The method of claim 1, the method further comprising:

h) outputting the set of best correspondences.

4. The method of claim 1 wherein the object has the shape of a cross.

5. The method of claim 1 wherein the object has the shape of a T.

6. The method of claim 1 wherein the object has the shape of an L.

7. The method of claim 1 wherein the object has the shape of a rectangle.

8. The method of claim 1 wherein the library of predefined boundary model shapes and the library of predefined internal line model shapes comprise a single library of predefined shapes.

9. A system for automatically training a machine vision system prior to run-time of an industrial automation system that utilizes the machine vision system, the system comprising:

means for obtaining an image of an object worked upon in the industrial automation system, the image comprising a plurality of edges;

means for extracting a set of boundary line segments from the image that coincide with at least one of the plurality of edges;

means for generating one or more boundary match candidates by selecting at least one boundary model shape from a library of predefined boundary model shapes and comparing the at least one boundary model shape to the extracted set of boundary segments;

means for extracting a set of internal line segments from the image that are remote from the plurality of edges;

wherein a boundary of the object is skeletonized to create the set of internal line segments;

means for generating one or more internal match candidates by selecting at least one line model shape from a library of predefined line model shapes and comparing the at least one line model shape to the extracted set of line segments;

means for corresponding the set of boundary line segments to the one or more boundary match candidates and the set of internal line segments to the one or more internal match candidates to identify a set of best correspondences for the object;

and means for automatically training the machine vision system prior to run-time of the industrial automation system, the training for using one or more of the set of best correspondences for the object.

10. A method for automatically training a machine vision system prior to run-time of an industrial automation system that utilizes the machine vision system, the machine vision system comprising a digital data processor, the method comprising the steps of:

a) receiving at the digital data processor an image of an object worked upon in the industrial automation system, the image comprising a plurality of edges;

b) extracting, using the digital data processor, a set of boundary line segments and a set of internal line segments from the image, the set of boundary line segments coincide with at least one of the plurality of edges, the set of internal line segments are remote from the plurality of edges;

wherein a boundary of the object is skeletonized to create the set of internal line segments;

c) comparing, using the digital data processor, at least one of the set of boundary line segments to at least one predefined shape in a library of shapes;

d) comparing, using the digital data processor, at least one of the set of internal line segments to at least one predefined shape in the library of shapes;

e) generating, using the digital data processor, one or more match candidates based on the of steps c) and d);

f) corresponding, using the digital data processor, the set of boundary line segments and the set of internal line segments to the one or more match candidates to identify a set of best correspondences for the object; and

g) prior to run-time of the industrial automation system, automatically training the machine vision system for using one or more of the set of best correspondences for the object.

11. The method of claim 10 further including, during run-time of the industrial automation system, locating a shape on the object using the trained machine vision system.

12. The method of claim 11 wherein the step of locating the shape on the object includes locating the shape on the object while the object is moving on an assembly line in the industrial automation system.

13. The method of claim 11 wherein the step of obtaining the image of the object includes obtaining the image of the object while the object is not moving on an assembly line in the industrial automation system.

14. A method for locating a shape on an object after training a machine vision system to locate the shape, the machine vision system to be used in an industrial automation run-time application, the object to be worked upon in the industrial automation run-time application, the method comprising:

during the machine vision training time performing the steps of;

i) capturing an image of a model object comprising the shape, the image comprising a plurality of edges;

ii) transmitting the image to an image analysis system, the image analysis system performing the steps of;

a) extracting a set of boundary line segments and a set of internal line segments from the image, the set of boundary line segments coinciding with at least one of the plurality of edges, the set of internal line segments are remote from the plurality of edges;

wherein a boundary of the object is skeletonized to create the set of internal line segments;

b) comparing the set of boundary line segments to at least one predefined shape in a library of shapes;

c) comparing the set of internal line segments to at least one predefined shape in the library of shapes;

d) generating one or more match candidates based on the comparing of steps b) and c);

e) corresponding the set of boundary line segments or the set of internal line segments to identify a set of best correspondences that identify the shape on the model object; and

f) automatically training the machine vision system for using at least one of the set of best correspondences to locate the shape on the object; and

after training the machine vision system, performing the steps of;

iii) capturing an image of the object; and

iv) using the industrial automation run-time application to locate the shape on the object.

15. The method of claim 14 wherein the step of capturing the image of the object includes capturing the image of the object while the object is not moving on an assembly line in an industrial automation system.

16. The method of claim 14 wherein the step of using the industrial automation run-time application includes locating the shape on the object while the object is moving on an assembly line in an industrial automation system.