Method and apparatus for disparity estimation

• US 9,704,252 B2
• Filed: 03/07/2015
• Issued: 07/11/2017
• Est. Priority Date: 03/07/2014
• Status: Expired due to Fees

• Current Assignee: Magnolia Licensing LLC (IP Edge LLC)
• Original Assignee: Thomson Licensing (Vantiva SA)
• Inventors: Jachalsky, Joern, Janssen, Uwe, Drazic, Valter
• Primary Examiner(s): Kassa, Yosef
• Application Number:
  US14/641,334
• Publication Number:
  US 20150254864A1
• Time in Patent Office: 857 Days
• Field of Search: 382190, 382209, 382218, 382219, 382278, 382282, 358537, 358538, 358453
• CPC Class Codes: G06F 18/24 Classification techniques
• CPC Class Codes: G06T 2207/20016 Hierarchical, coarse-to-fine, multiscale or multiresolution image processing; Pyramid transform
• CPC Class Codes: G06T 7/593 from stereo images
• CPC Class Codes: G06T 7/97 Determining parameters from multiple pictures depth or shape recovery from multiple images G06T7/55; stereo camera calibration G06T7/85

Abstract

A method for hierarchical disparity estimation on an image pair, wherein image pyramids are created by successively downscaling each image of the image pair, and an apparatus configured to perform the method are described. An initial disparity estimator applies a full search on a highest level of the image pyramids to determine initial disparity estimates, the highest level having the lowest resolution. A disparity propagator passes the initial disparity estimates to a next lower level of the image pyramids. An allocator then partitions the pixels of each remaining hierarchy level of the image pyramids into two or more groups of pixels, where each pixel in a group of pixels can be processed independently from remaining pixels of that group of pixels. A disparity estimator estimates disparity values for the pixels of a first group of pixels utilizing disparity estimates from a next higher level of the image pyramids. The disparity estimator further estimates disparity values for the pixels of remaining groups of pixels utilizing disparity estimates from a previously processed group of pixels in that hierarchy level.

Claims

1. A method for hierarchical disparity estimation on an image pair, wherein image pyramids are created by successively downscaling each image of the image pair, the method comprising:

applying a full search on a highest level of the image pyramids to determine initial disparity estimates;

passing the initial disparity estimates to a next lower level of the image pyramids, the highest level having the lowest resolution;

for each remaining hierarchy level of the image pyramids, partitioning the pixels of a hierarchy level into two or more groups of pixels, where each pixel in a group of pixels can be processed independently from remaining pixels of that group of pixels;

for a first group of pixels, estimating disparity values for the pixels utilizing disparity estimates from a next higher level of the image pyramids; and

for remaining groups of pixels, estimating disparity values for the pixels utilizing disparity estimates from a previously processed group of pixels in that hierarchy level.

2. The method according to claim 1, wherein the pixels of a hierarchy level are partitioned into four groups of pixels in accordance with row numbers and column numbers of the pixels.

3. The method according to claim 2, wherein the first group of pixels comprises only pixels with an even row number and an even column number, a second group of pixels comprises only pixels with an odd row number and an odd column number, a third group of pixels comprises only pixels with an even row number and an odd column number, and a fourth group of pixels comprises only pixels with an odd row number and an even column number.

4. The method according to claim 1, wherein a weighted median is applied on the disparity values estimated for the pixels of a hierarchy level.

5. The method according to claim 1, wherein each image of the image pair is downscaled by factor 2 in both dimensions between successive levels.

6. The method according to claim 1, wherein disparity estimates of a next higher level determine a search interval for the disparity estimation in a current level.

7. The method according to claim 6, wherein the search interval for the disparity estimation in the current level is defined by minimum and maximum disparity estimates of the next higher level.

8. The method according to claim 7, wherein an offset is added to the search interval for the disparity estimation in the current level.

9. The method according to claim 1, wherein a search window is scaled in correspondence with the levels of the image pyramid.

10. An apparatus configured to perform hierarchical disparity estimation on an image pair, wherein image pyramids are created by successively downscaling each image of the image pair, the apparatus comprising:

an initial disparity estimator configured to apply a full search on a highest level of the image pyramids to determine initial disparity estimates, the highest level having the lowest resolution;

a disparity propagator configured to pass the initial disparity estimates to a next lower level of the image pyramids;

an allocator configured to partition the pixels of each remaining hierarchy level of the image pyramids into two or more groups of pixels, where each pixel in a group of pixels can be processed independently from remaining pixels of that group of pixels; and

a disparity estimator configured to estimate disparity values for the pixels of a first group of pixels utilizing disparity estimates from a next higher level of the image pyramids and to estimate disparity values for the pixels of remaining groups of pixels utilizing disparity estimates from a previously processed group of pixels in that hierarchy level.

11. The apparatus according to claim 10, wherein the allocator partitions the pixels of a hierarchy level into four groups of pixels in accordance with row numbers and column numbers of the pixels.

12. The apparatus according to claim 10, comprising a weighting unit configured to apply a weighted median on the disparity values estimated for the pixels of a hierarchy level.

13. The apparatus according to claim 10, comprising a determining unit configured to determine a search interval for the disparity estimation in a current level based on disparity estimates of a next higher level.

14. The apparatus according to claim 13, comprising an adder to add an offset to the search interval for the disparity estimation in the current level.

15. A non-transitory computer readable storage medium having stored therein instructions enabling hierarchical disparity estimation on an image pair, wherein image pyramids are created by successively downscaling each image of the image pair, which when executed by a computer, cause the computer to:

apply a full search on a highest level of the image pyramids to determine initial disparity estimates, the highest level having the lowest resolution;

pass the initial disparity estimates to a next lower level of the image pyramids;

for each remaining hierarchy level of the image pyramids, partition the pixels of a hierarchy level into two or more groups of pixels, where each pixel in a group of pixels can be processed independently from remaining pixels of that group of pixels;

for a first group of pixels, estimate disparity values for the pixels utilizing disparity estimates from a next higher level of the image pyramids; and

for remaining groups of pixels, estimate disparity values for the pixels utilizing disparity estimates from a previously processed group of pixels in that hierarchy level.

16. The non-transitory computer readable storage medium according to claim 15, wherein the instructions cause the computer to partition the pixels of a hierarchy level into four groups of pixels in accordance with row numbers and column numbers of the pixels.

17. The non-transitory computer readable storage medium according to claim 16, wherein the first group of pixels comprises only pixels with an even row number and an even column number, a second group of pixels comprises only pixels with an odd row number and an odd column number, a third group of pixels comprises only pixels with an even row number and an odd column number, and a fourth group of pixels comprises only pixels with an odd row number and an even column number.

18. The non-transitory computer readable storage medium according to claim 15, wherein the instructions cause the computer to apply a weighted median on the disparity values estimated for the pixels of a hierarchy level.

19. The non-transitory computer readable storage medium according to claim 15, wherein the instructions cause the computer to downscale each image of the image pair by factor 2 in both dimensions between successive levels.

20. The non-transitory computer readable storage medium according to claim 15, wherein disparity estimates of a next higher level determine a search interval for the disparity estimation in a current level.

21. The non-transitory computer readable storage medium according to claim 20, wherein the search interval for the disparity estimation in the current level is defined by minimum and maximum disparity estimates of the next higher level.

22. The non-transitory computer readable storage medium according to claim 21, wherein the instructions cause the computer to add an offset to the search interval for the disparity estimation in the current level.

23. The non-transitory computer readable storage medium according to claim 15, wherein the instructions cause the computer to scale a search window in correspondence with the levels of the image pyramid.