Methods, systems, and media for image processing using hierarchical expansion

• US 8,948,512 B2
• Filed: 03/15/2013
• Issued: 02/03/2015
• Est. Priority Date: 03/15/2013
• Status: Active Grant

• Current Assignee: Google LLC (Alphabet Inc.)
• Original Assignee: Google Inc. (Alphabet Inc.)
• Inventors: Zuczek, Pawel
• Primary Examiner(s): Do, Anh
• Application Number:
  US13/833,654
• Publication Number:
  US 20140270527A1
• Time in Patent Office: 690 Days
• Field of Search: 382/180, 382/278, 382/294, 382/298, 340/988, 340/990, 701/494, 707/722, 707/724, 715/273
• US Class Current: 382/180
• CPC Class Codes: G06T 2207/20016 Hierarchical, coarse-to-fine, multiscale or multiresolution image processing; Pyramid transform
• CPC Class Codes: G06T 7/11 Region-based segmentation
• CPC Class Codes: G06T 7/143 involving probabilistic approaches, e.g. Markov random field [MRF] modelling
• CPC Class Codes: G06T 7/174 involving the use of two or more images

Abstract

Methods, systems, and media for image processing using hierarchical expansion are provided. More particularly, the hierarchical expansion can include a merge expansion and/or a refine expansion. In some embodiments, a method for image processing, comprising: receiving a plurality of images, wherein each of the plurality of images has one of a plurality of resolutions; for a first image from the plurality of images that has a first resolution, determining a first set of labels corresponding to each pixel of the first image; and for a second image from the plurality of images that has a second resolution, generating a second set of labels corresponding to each pixel of the second image based on the first set of labels.

Claims

1. A method for image processing, the method comprising:

receiving, using a hardware processor, a plurality of images, wherein each of the plurality of images has one of a plurality of resolutions;

for a first image from the plurality of images that has a first resolution, determining, using the hardware processor, a first set of labels corresponding to each pixel of the first image; and

for a second image from the plurality of images that has a second resolution, generating, using the hardware processor, a second set of labels corresponding to each pixel of the second image based on the first set of labels by;

(a) scaling, using the hardware processor, the first set of labels to generate a set of scaled labels;

(b) determining, using the hardware processor, a set of proposed labels for each pixel of the second image based on the set of scaled labels;

(c) generating, using the hardware processor, a probability score for each proposed label in the set of proposed labels;

(d) selecting, using the hardware processor, a proposed label from the set of proposed labels based at least in part on the probability score; and

(e) determining, using the hardware processor, whether to assign the proposed label for each pixel to the second set of labels of the second image.

2. The method of claim 1, further comprising:

generating an incremented proposed label by incrementing the selected proposed label by a given value; and

selecting, for each pixel in the second image, an updated label between the proposed label and the incremented proposed label using a graph cut technique.

3. The method of claim 1, wherein the first set of labels corresponds to a first set of disparity values between the first image and a third image, and wherein the second set of labels corresponds to a second set of disparity values between the second image and a fourth image.

4. The method of claim 1, further comprising:

selecting at least two proposed labels from the set of proposed labels based at least in part on the probability score; and

selecting one proposed label from the at least two proposed labels using a graph cut technique.

5. The method of claim 1, further comprising determining whether to replace an initial label corresponding to a pixel in the first image with a predetermined label.

6. The method of claim 1, further comprising:

determining that the second resolution is not a desired resolution; and

repeating (a)-(e) until an image having the desired resolution is reached.

7. The method of claim 1, wherein the second resolution is greater than the first resolution.

8. The method of claim 1, wherein the first set of labels is scaled by a scale factor that is determined based on the first resolution and the second resolution.

9. The method of claim 1, wherein the set of proposed labels associated with a pixel in the second image includes at least one scaled label associated with the pixel and a plurality of scaled labels from neighboring pixels.

10. The method of claim 1, further comprising selecting the first image and the second image based on resolution.

11. A system for image processing, the system comprising:

a hardware processor that;

receives a plurality of images, wherein each of the plurality of images has one of a plurality of resolutions;

for a first image from the plurality of images that has a first resolution, determines a first set of labels corresponding to each pixel of the first image; and

for a second image from the plurality of images that has a second resolution, generates a second set of labels corresponding to each pixel of the second image based on the first set of labels by;

(a) scaling the first set of labels to generate a set of scaled labels;

(b) determining a set of proposed labels for each pixel of the second image based on the set of scaled labels;

(c) generating a probability score for each proposed label in the set of proposed labels;

(d) selecting a proposed label from the set of proposed labels based at least in part on the probability score; and

(e) determining whether to assign the proposed label for each pixel to the second set of labels of the second image.

12. The system of claim 11, wherein the hardware processor is further configured to:

generate an incremented proposed label by incrementing the selected proposed label by a given value; and

select, for each pixel in the second image, an updated label between the proposed label and the incremented proposed label using a graph cut technique.

13. The system of claim 11, wherein the first set of labels corresponds to a first set of disparity values between the first image and a third image, and wherein the second set of labels corresponds to a second set of disparity values between the second image and a fourth image.

14. The system of claim 11, wherein the hardware processor is further configured to:

select at least two proposed labels from the set of proposed labels based at least in part on the probability score; and

select one proposed label from the at least two proposed labels using a graph cut technique.

15. The system of claim 11, wherein the hardware processor is further configured to determine whether to replace an initial label corresponding to a pixel in the first image with a predetermined label.

16. The system of claim 11, wherein the hardware processor is further configured to:

determine that the second resolution is not a desired resolution; and

repeat (a)-(e) until an image having the desired resolution is reached.

17. The system of claim 11, wherein the second resolution is greater than the first resolution.

18. The system of claim 11, wherein the first set of labels is scaled by a scale factor that is determined based on the first resolution and the second resolution.

19. The system of claim 11, wherein the set of proposed labels associated with a pixel in the second image includes at least one scaled label associated with the pixel and a plurality of scaled labels from neighboring pixels.

20. The system of claim 11, wherein the hardware processor is further configured to select the first image and the second image based on resolution.

21. A computer-readable medium containing computer-executable instructions that, when executed by a processor, cause the process to perform a method for image processing, the method comprising:

receiving a plurality of images, wherein each of the plurality of images has one of a plurality of resolutions;

for a first image from the plurality of images that has a first resolution, determining a first set of labels corresponding to each pixel of the first image; and

for a second image from the plurality of images that has a second resolution, generating a second set of labels corresponding to each pixel of the second image based on the first set of labels by;

(a) scaling the first set of labels to generate a set of scaled labels;

(b) determining a set of proposed labels for each pixel of the second image based on the set of scaled labels;

(c) generating a probability score for each proposed label in the set of proposed labels;

(d) selecting a proposed label from the set of proposed labels based at least in part on the probability score; and

(e) determining whether to assign the proposed label for each pixel to the second set of labels of the second image.

22. The computer-readable medium of claim 21, wherein the method further comprises:

generating an incremented proposed label by incrementing the selected proposed label by a given value; and

selecting, for each pixel in the second image, an updated label between the proposed label and the incremented proposed label using a graph cut technique.

23. The computer-readable medium of claim 21, wherein the first set of labels corresponds to a first set of disparity values between the first image and a third image, and wherein the second set of labels corresponds to a second set of disparity values between the second image and a fourth image.

24. The computer-readable medium of claim 21, wherein the method further comprises:

selecting at least two proposed labels from the set of proposed labels based at least in part on the probability score; and

selecting one proposed label from the at least two proposed labels using a graph cut technique.

25. The computer-readable medium of claim 21, wherein the method further comprises determining whether to replace an initial label corresponding to a pixel in the first image with a predetermined label.

26. The computer-readable medium of claim 21, wherein the method further comprises:

determining that the second resolution is not a desired resolution; and

repeating (a)-(e) until an image having the desired resolution is reached.

27. The computer-readable medium of claim 21, wherein the second resolution is greater than the first resolution.

28. The computer-readable medium of claim 21, wherein the first set of labels is scaled by a scale factor that is determined based on the first resolution and the second resolution.

29. The computer-readable medium of claim 21, wherein the set of proposed labels associated with a pixel in the second image includes at least one scaled label associated with the pixel and a plurality of scaled labels from neighboring pixels.

30. The computer-readable medium of claim 21, wherein the method further comprises selecting the first image and the second image based on resolution.