SYSTEMS AND METHODS FOR AUTOMATICALLY DETERMINING 3-DIMENSIONAL OBJECT INFORMATION AND FOR CONTROLLING A PROCESS BASED ON AUTOMATICALLY-DETERMINED 3-DIMENSIONAL OBJECT INFORMATION

• US 20080068379A1
• Filed: 09/17/2007
• Published: 03/20/2008
• Est. Priority Date: 09/19/2006
• Status: Active Grant

• Current Assignee: Wisconsin Alumni Research Foundation (University of Wisconsin System)
• Original Assignee: Wisconsin Alumni Research Foundation (University of Wisconsin System)
• Inventors: Rawlings, James B., Larsen, Paul A., Ferrier, Nicola J.
• Granted Patent: US 7,916,935 B2
• Time in Patent Office: Days
• US Class Current: 345/427
• CPC Class Codes: G01N 15/1433 using image recognition
• CPC Class Codes: G01N 2015/1497 Particle shape
• CPC Class Codes: G06T 2207/10016 Video; Image sequence
• CPC Class Codes: G06T 2207/10056 Microscopic image
• CPC Class Codes: G06T 7/74 involving reference images or patches
• CPC Class Codes: G06V 10/457 by analysing connectivity, e.g. edge linking, connected component analysis or slices
• CPC Class Codes: G06V 20/695 Preprocessing, e.g. image segmentation

Abstract

An image containing one or more types of objects to be located is analyzed to locate linear features within the image. The objects have edges having known spatial relationships. The linear features and identified virtual lines are analyzed to find groups of linear features and/or virtual lines that have one of the known spatial relationships. These relationships can include parallel edges, edges that meet at certain angles or angle ranges, the number of lines meeting a vertex and the like. The identified group is compared with projected 2-dimensional representation(s) of the object(s) to determine whether any additional lines appear in the image that are part of the located object. In various exemplary embodiments, two or more hypotheses for how the identified group of linear features maps to the 3-dimensional representation of the object can be generated. The best fitting hypothesis becomes the recognized 3-dimensional shape and orientation for that object.

Claims

1. A method for identifying an object appearing in an image, the image having a plurality of pixels, each pixel having an image value, and containing at least one object of at least one type of object to be identified, each type of object to be identified represented by a 3-dimensional model, each 3-deminsional model including edges and spatial relationships between the edges, the method comprising:

analyzing the plurality of pixels to determine a plurality of line-like features in the image;

grouping at least some of the line-like features into at least one viewpoint-invariant line group, each determined viewpoint-invariant line group corresponding to at least one spatial relationship between edges of at least one of the 3-dimensional models;

analyzing at least one determined viewpoint-invariant line group to identify, for each such determined viewpoint-invariant line group, a 2-dimensional projection of one of the at least one 3-dimensional models that corresponds to that determined viewpoint-invariant line group; and

identifying, for at least one determined viewpoint-invariant line group for which there is a 2-dimensional projection of one of the at least one 3-dimensional models, that viewpoint-invariant line group as corresponding to an object appearing in the image.

2. The method of claim 1, wherein there are a plurality of objects in the image, the objects having at least one parameter and being involved in a process having at least one controllable process parameter, the method further comprising:

repeating the at least one determined viewpoint-invariant line group analyzing step and the identifying step to determine a plurality of viewpoint-invariant line groups that each corresponds to one of the plurality of objects appearing in the image;

fitting, for each determined viewpoint-invariant line group that correspond to an object appearing in the image, the 3-dimensional model for that object to that viewpoint-invariant line group based on the 2-dimensional projection;

determining, for each object appearing in the image, at least one parameter for that object based on the fitted 3-dimensional model for that object; and

controlling at least one of the at least one controllable process parameter based on at least one of the at least one parameter determined from the determined objects.

3. The method of claim 1, wherein analyzing the plurality of pixels to determine a plurality of line-like features in the image comprises:

analyzing the plurality of pixels of the obtained image to locate linear features occurring within the image;

analyzing the located linear features at least to identify pairs of located linear features that are sufficiently co-linear;

generating virtual lines for at least some pairs of located linear features that are sufficiently co-linear;

wherein the line-like features include the located linear features and the generated virtual lines.

4. The method of claim 1, wherein analyzing at least one determined viewpoint-invariant line group to identify, for each such determined viewpoint-invariant line group, a 2-dimensional projection of one of the at least one 3-dimensional models that corresponds to that determined viewpoint-invariant line group comprises:

selecting one of at least one determined viewpoint-invariant line group;

determining, for the selected viewpoint-invariant line group and for each of at least one of the 3-dimensional models, at least one hypothesis for how the line-like features of the selected viewpoint-invariant line group correspond to edges of that 3-dimensional model; and

determining, for each determined hypothesis, a 2-dimensional projection of the 3-dimensional model of that hypothesis based on the corresponding selected viewpoint-invariant line group;

determining, for each determined hypothesis, if any other line-like features appearing in the image match visible edges of the 2-dimensional projection determined for that hypothesis;

associating, for each determined hypothesis, each such matching line-like feature with that hypothesis; and

determining a score for each determined hypothesis based on the selected viewpoint-invariant line group and any matching line-like features associated with that hypothesis.

5. The method of claim 4, wherein identifying, for at least one determined viewpoint-invariant line group that corresponds to a 2-dimensional projection of one of the at least one 3-dimensional models, that viewpoint-invariant line group as an object appearing in the image comprises:

selecting, for a selected viewpoint-invariant line group, a hypothesis associated with that viewpoint-invariant line group that has a highest score as the selected hypothesis for that selected viewpoint-invariant line group;

determining if the selected hypothesis has a score at least equal to a score threshold;

discarding, if the score is not at least equal to the score threshold, the selected viewpoint-invariant line group; and

identifying, if the score is at least equal to the score threshold for the selected hypothesis, the selected viewpoint-invariant line group and any matching other line-like features as an object appearing in the image.

6. The method of claim 5, further comprising:

determining, if the score is at least equal to the score threshold, if any other viewpoint-invariant line groups and/or any other line-like features sufficiently overlap the 2-dimensional projection of the 3-dimensional model of that hypothesis into the image;

removing from further consideration any determined other viewpoint-invariant line groups and any other line-like features that sufficiently overlap the 2-dimensional projection of the 3-dimensional model of that hypothesis into the image; and

repeating the viewpoint-invariant line group selecting step through the determined other viewpoint-invariant line groups and other line-like features removing step for at least one additional viewpoint-invariant line group.

7. The method of claim 5, further comprising:

repeating, if the score is not at least equal to the score threshold, the viewpoint-invariant line group selecting step through the selected hypothesis having a score at least equal to a score threshold determining step for at least one additional viewpoint-invariant line group.

8. The method of claim 1, further comprising:

repeating, after either the discarding step or the determined other viewpoint-invariant line groups and other line-like features removing step, the viewpoint-invariant line group selecting step through the selected hypothesis having a score at least equal to a score threshold determining step for at least one additional viewpoint-invariant line group.

9. The method of claim 1, further comprising obtaining an image comprising the plurality of pixels having image values.