Robust search for image features across image sequences exhibiting non-uniform changes in brightness

• US 6,005,978 A
• Filed: 02/07/1996
• Issued: 12/21/1999
• Est. Priority Date: 02/07/1996
• Status: Expired due to Term

• Current Assignee: Cognex Corporation
• Original Assignee: Cognex Corporation
• Inventors: Garakani, Arman
• Primary Examiner(s): Bella, Matthew C.
• Application Number:
  US08/597,653
• Time in Patent Office: 1,413 Days
• Field of Search: 382/103, 382/151, 382/190, 382/191, 382/192, 382/194, 382/195, 382/197, 382/199, 382/201, 382/203, 382/204, 382/205, 382/206, 382/260, 382/263, 382/266, 382/269, 382/273 , 382/274, 382/291, 382/294, 382/281, 382/282, 382/218, 348/169
• US Class Current: 382/218
• CPC Class Codes: G06V 10/443 by matching or filtering
• CPC Class Codes: G06V 2201/06 Recognition of objects for industrial automation
• CPC Class Codes: G06V 30/2504 Coarse or fine approaches, e.g. resolution of ambiguities or multiscale approaches

Abstract

An apparatus and method for two-dimensional search for a model image within atest image that exploits edge-based matching of edge maps, followed by area-based matching of at least coarse resolution edge maps of respective multi-resolution edge-maps, to efficiently and accurately provide the position of at least one instance of the model image. The invention can find a location of the model image within each test image of a sequence of test images, even when there are non-uniform brightness changes in the test image due to process steps or lighting changes. The postion of each verified instance of the model image within the test image can be provided to an alignment device, a measurement device, or any other device that can use such position information.

Claims

1. Apparatus for finding a model image within a test image, the apparatus comprising:

model image means for acquiring a model image;

means, connected to said model image means, for generating a multi-resolution representation of said model image, said multi-resolution representation of said model image including a coarsest single-resolution model image, and at least one single-resolution model image of finer resolution;

means, connected to said means for generating a multi-resolution representation of said model image, for performing edge detection upon at least said coarsest single-resolution model image to provide at least a coarsest model image edge map;

test image digitizer means for acquiring a test image;

means, connected to said test image digitizer means, for generating a multi-resolution representation of said test image, said multi-resolution representation of said test image including a coarsest single-resolution test image, and at least one single-resolution test image of finer resolution;

means, connected to said means for generating a multi-resolution representation of said test image, for performing edge detection upon at least said coarsest single-resolution test image to provide at least a coarsest test image edge map;

means, connected to said means for performing edge detection upon said coarsest single-resolution model image, and connected to said means for performing edge detection upon said coarsest single-resolution test image, for computing a match-metric value for each one of a plurality of relative displacements of the coarsest model image edge map, with respect to said coarsest test image edge map, to provide a match-metric surface in a match space;

means, connected to said means for computing a match-metric value for each one of a plurality of relative displacements, for determining characteristic information and height information for each local maximum of said match-metric surface;

means, connected to said means for determining characteristic information and height information, for selecting acceptable local maxima of said match-metric surface using both said characteristic information and said height information for each local maximum; and

output means, connected to said means for selecting acceptable local maxima of said match-metric surface, for providing position information of said acceptable local maxima to a manipulation device.

2. A method for finding a model image within a test image, the method comprising the steps of:

acquiring a model image;

generating a multi-resolution representation of said model image, said multi-resolution representation of said model image including a coarsest single-resolution model image, and at least one single-resolution model image of finer resolution;

performing edge detection upon at least said coarsest single-resolution model image to provide at least a coarsest model image edge map;

acquiring a test image;

generating a multi-resolution representation of said test image, said multi-resolution representation of said test image including a coarsest single-resolution test image, and at least one single-resolution test image of finer resolution;

performing edge detection upon at least said coarsest single-resolution test image to provide at least a coarsest test image edge map;

computing a match-metric value for a plurality of relative displacements of the coarsest model image edge map, with respect to said coarsest test image edge map, to provide a match-metric surface in a match space;

determining characteristic information and height information for each local maximum of said match-metric surface;

selecting an acceptable local maximum of said match-metric surface using said characteristic information and said height information for each local maximum; and

providing position information of said acceptable local maximum to indicate position of said model image.

3. The method of claim 2, wherein the step of generating a multi-resolution representation of said model image includes the step of:

generating a Gaussian pyramid of said model image.

4. The method of claim 2, wherein the step of performing edge detection upon at least said coarsest single resolution model image includes the step of:

convolving at least said coarsest single-resolution model image with an edge operator.

5. The method of claim 4, wherein said edge operator is an edge operator that provides both edge magnitude and edge angle information.

6. The method of claim 4, wherein said edge operator is a Sobel operator.

7. The method of claim 2, wherein the step of generating a multi-resolution representation of said test image includes the step of:

generating a Gaussian pyramid of said test image.

8. The method of claim 2, wherein the step of performing edge detection upon at least said coarsest single-resolution test image includes the step of:

convolving at least said coarsest single-resolution test image with an edge operator.

9. The method of claim 8, wherein said edge operator is an edge operator that provides both edge magnitude and edge angle information.

10. The method of claim 8, wherein said edge operator is a Sobel operator.

11. The method of claim 2, wherein the step of computing a match-metric value for a plurality of relative displacements includes the step of:

performing a Hough transform of said coarsest test image edge map using said coarsest model image edge map.

12. The method of claim 2, wherein the step of performing edge detection upon said single-resolution model image includes the step of:

discarding edge pixels of at least said coarsest model image edge map.

13. The method of claim 12, wherein the step of discarding edge pixels includes the step of:

discarding all edge pixels except edge pixels that correspond to an edge peak.

14. The method of claim 2, wherein the step of performing edge detection upon said coarsest single-resolution test image to provide a coarsest test image edge map includes the step of

discarding edge pixels from said test image edge map.

15. The method of claim 14, wherein the step of discarding edge pixels includes the step of:

discarding all edge pixels except edge pixels that correspond to an edge peak.

16. The method of claim 2, wherein said characteristic information is an area-based match quality score, and wherein the step of selecting an acceptable local maximum of said match-metric surface includes the steps of:

performing an area-based match step using a model edge map to provide an area-based match quality score; and

comparing said area-based match quality score of each local maximum to a threshold to determine an acceptable local maximum.

17. The method of claim 16, wherein the step of determining a best local maximum includes the step of:

including each local maxima in a list;

sorting said list in accordance with said area-based match score to provide an ordered list of local maxima; and

verifying each local maximum in said ordered list until an acceptable local maximum is found.

18. The method of claim 17, wherein the step of verifying includes the steps of:

projecting said local maximum to at least one evaluation resolution level;

searching for an image feature corresponding to said local maximum at each evaluation resolution level, and providing a match score for each evaluation resolution level; and

determining whether said match score satisfies a match threshold at each said evaluation resolution level.

19. The method of claim 18, further including the step of:

determining whether any other local maxima that satisfies said match threshold at each said evaluation resolution level is closer than a minimum acceptable distance in said match space.

20. The method of claim 18, wherein the step of projecting said local maximum to at least one evaluation resolution level includes the steps of:

defining a projection boundary to include image feature pixels at each evaluation resolution level that correspond to said local maximum, and to include additional freedom pixels that surround said image feature pixels.

21. The method of claim 18, wherein a user selects said at least one evaluation resolution level upon which the steps of searching and determining are executed.

22. The method of claim 2, wherein said coarsest model image edge map includes edge direction information, and wherein the step of computing a match-metric value for each one of a plurality of relative displacements includes the step of:

using said edge direction information from said model image edge map to compute said match-metric value.

23. The method of claim 2, wherein the step of performing edge detection on said coarsest single-resolution model image includes the step of:

performing hysterisis edge pixel selection.

24. The method of claim 2, wherein the step of performing edge detection on said coarsest single-resolution test image includes the step of:

performing hysterisis edge pixel selection.

25. A method for planar alignment usable over a plurality of surface-altering process steps, the method comprising the steps of:

repeatedly using the method for finding a model image within a test image of claim 2.

26. The method of claim 2, wherein the step of determining position information of said best local maximum includes the step of:

performing sub-pixel interpolation to determine position information of said best local maximum to sub-pixel accuracy.