Determining correspondence between image regions

• US 8,861,869 B2
• Filed: 10/17/2013
• Issued: 10/14/2014
• Est. Priority Date: 08/16/2010
• Status: Active Grant

• Current Assignee: Adobe Inc.
• Original Assignee: Adobe Systems Incorporated (Adobe Inc.)
• Inventors: Shechtman, Elya, Goldman, Daniel R., Barnes, Connelly, Finkelstein, Adam
• Primary Examiner(s): Carter, Aaron W
• Application Number:
  US14/056,696
• Publication Number:
  US 20140105499A1
• Time in Patent Office: 362 Days
• Field of Search: 382/209, 382217-222
• US Class Current: 382/218
• CPC Class Codes: G06T 7/337 involving reference images or patches
• CPC Class Codes: G06V 10/757 Matching configurations of points or features
• CPC Class Codes: G06V 10/759 Region-based matching

Abstract

Techniques for determining correspondence between image regions are described. A computing system stores images that are comparable to determine corresponding image patches of the images. An approximation algorithm is implemented, and for multiple image patches in a region in a first image, corresponding image patches are determined in a second image. The approximation algorithm performs iterations utilizing a nearby-pixel mapping evaluation and a random-perturbation mapping evaluation to determine and select the corresponding image patches in the second image.

Claims

1. A computing system, comprising:

a memory configured to store images that are comparable to determine corresponding image patches of the images;

a processing system to implement an approximation algorithm that is configured to;

for multiple image patches in a region in a first image, determine corresponding image patches in a second image;

perform iterations utilizing a nearby-pixel mapping evaluation and a random-perturbation mapping evaluation to determine and select the corresponding image patches in the second image; and

perform a geometric warping action on at least one of the corresponding image patches in at least one of the iterations, the geometric warping action performed as at least a translation and an additional action, or as the geometric warping action without the translation.

2. The computing system as recited in claim 1, wherein the approximation algorithm is configured to perform the iterations utilizing the nearby-pixel mapping evaluation and the random-perturbation mapping evaluation on a subset of the corresponding image patches that are determined in the second image.

3. The computing system as recited in claim 2, wherein the approximation algorithm is configured to randomly select the subset of the corresponding image patches.

4. The computing system as recited in claim 1, wherein the approximation algorithm is configured to:

utilize patch distances from the multiple image patches in the region of the first image to perform the iterations to determine the corresponding image patches in the second image;

determine whether one of the corresponding image patches determined from the iterations has a smaller patch distance from one of the image patches than one of k patches that has a greatest patch distance in a data structure of the k patches; and

if so, remove the image patch that has the greatest patch distance from the data structure, and add the corresponding image patch to the k patches in the data structure.

5. The computing system as recited in claim 4, wherein the approximation algorithm is configured to eliminate duplicate corresponding image patches from the k patches before determining the patch distances.

6. The computing system as recited in claim 4, wherein:

the random-perturbation mapping evaluation includes selecting n sample patches for each of the k patches in the data structure, for a total of nk sample patches; and

the approximation algorithm is configured to eliminate duplicate corresponding image patches from the nk sample patches before determining the patch distances.

7. The computing system as recited in claim 1, wherein the approximation algorithm is configured to randomly select k patches in the second image for use in an initial one of the iterations.

8. The computing system as recited in claim 1, wherein the approximation algorithm is configured to:

retain all of the corresponding image patches determined by the nearby-pixel mapping evaluation and the random-perturbation mapping evaluation; and

record a subset of the retained corresponding image patches having a smallest patch distance.

9. The computing system as recited in claim 1, wherein the approximation algorithm is configured to:

select an image descriptor for the first and second regions;

select a distance metric to evaluate mappings for the first patch; and

wherein the iterations use the selected image descriptor and the selected distance metric.

10. The computing system as recited in claim 1, wherein the geometric warping action includes at least one of a rotation, a scaling, and a homography of the at least one corresponding image patch.

11. A computing system, comprising:

a memory configured to store images that are comparable to determine corresponding image patches of the images;

a processing system to implement an approximation algorithm that is configured to;

for multiple image patches in a region in a first image, determine corresponding image patches in a second image based on a distance metric;

perform iterations utilizing a nearby-pixel mapping evaluation and a random-perturbation mapping evaluation to determine and select the corresponding image patches in the second image, the iterations based on the distance metric and patch distances from the multiple image patches in the region of the first image to the corresponding image patches in the second image;

determine whether one of the corresponding image patches determined from the iterations has a smaller patch distance from one of the image patches than one of k patches that has a greatest patch distance in a data structure of the k patches; and

if so, remove the image patch that has the greatest patch distance from the data structure, and add the corresponding image patch to the k patches in the data structure.

12. The computing system as recited in claim 11, wherein the approximation algorithm is configured to eliminate duplicate corresponding image patches from the k patches before determining the patch distances.

13. The computing system as recited in claim 11, wherein:

the random-perturbation mapping evaluation includes selecting n sample patches for each of the k patches in the data structure, for a total of nk sample patches; and

the approximation algorithm is configured to eliminate duplicate corresponding image patches from the nk sample patches before determining the patch distances.

14. The computing system as recited in claim 11, wherein the approximation algorithm is configured to randomly select k patches in the second image for use in an initial one of the iterations.

15. The computing system as recited in claim 11, wherein the approximation algorithm is configured to:

retain all of the corresponding image patches determined by the nearby-pixel mapping evaluation and the random-perturbation mapping evaluation; and

record a subset of the retained corresponding image patches having a smallest patch distance.

16. A computing system, comprising:

a memory configured to store images that are comparable to determine corresponding image patches of the images;

a processing system to implement an approximation algorithm that is configured to;

for multiple image patches in a region in a first image, determine corresponding image patches in a second image;

utilize heuristics to map the multiple image patches of the first image to the second image, the heuristics including enrichment to increase a collection of mapping candidates and at least one of a nearby-pixel mapping evaluation and a random-perturbation mapping evaluation, where the enrichment includes determining the corresponding image patches in the second image along a graph where every node is connected to its k offsets;

perform iterations using the selected heuristics to determine the corresponding image patches in the second image that correspond to the multiple image patches in the region of the first image.

17. The computing system as recited in claim 16, wherein the enrichment heuristic to increase the collection of the mapping candidates includes one of a forward-type enrichment heuristic or an inverse-type enrichment heuristic.

18. The computing system as recited in claim 16, wherein the enrichment heuristic to increase the collection of the mapping candidates utilizes a fixed number of hops in a patch-appearance-space to randomly sample the second image.

19. The computing system as recited in claim 16, wherein the approximation algorithm is configured to:

utilize patch distances from the multiple image patches in the region of the first image to perform the iterations to determine the corresponding image patches in the second image;

determine whether one of the corresponding image patches determined from the iterations has a smaller patch distance from one of the image patches than one of k patches that has a greatest patch distance in a data structure of the k patches; and

if so, remove the image patch that has the greatest patch distance from the data structure, and add the corresponding image patch to the k patches in the data structure.

20. The computing system as recited in claim 16, wherein the approximation algorithm is configured to:

retain all of the corresponding image patches determined using the selected heuristics; and

record a subset of the retained corresponding image patches having a smallest patch distance.