Depth Map Stereo Correspondence Techniques

• US 20140153816A1
• Filed: 11/30/2012
• Published: 06/05/2014
• Est. Priority Date: 11/30/2012
• Status: Active Grant

• Current Assignee: Adobe Inc.
• Original Assignee: Adobe Systems Incorporated (Adobe Inc.)
• Inventors: Cohen, Scott D., Price, Brian L., Somanath, Gowri
• Granted Patent: US 9,135,710 B2
• Time in Patent Office: Days
• US Class Current: 382/154
• CPC Class Codes: G01S 17/86 Combinations of lidar systems with systems other than lidar, radar or sonar, e.g. with direction finders
• CPC Class Codes: G01S 17/894 3D imaging with simultaneous measurement of time-of-flight at a 2D array of receiver pixels, e.g. time-of-flight cameras or flash lidar
• CPC Class Codes: G06T 2207/20212 Image combination
• CPC Class Codes: G06T 7/593 from stereo images

Abstract

Depth map stereo correspondence techniques are described. In one or more implementations, a depth map generated through use of a depth sensor is leveraged as part of processing of stereo images to assist in identifying which parts of stereo images correspond to each other. For example, the depth map may be utilized to describe depth of an image scene which may be used as part of a stereo correspondence calculation. The depth map may also be utilized as part of a determination of a search range to be employed as part of the stereo correspondence calculation.

Claims

1. A method implemented by one or more computing devices, the method comprising:

determining a search range based at least in part on a depth map generated through use of a depth sensor of an image scene; and

calculating stereo correspondence of image data of stereoscopic images of the image scene, the calculating performed based at least in part using the determined search range.

2. A method as described in claim 1, wherein the determining is based at least in part on examining confidence of values generated by the depth sensor that are included in the depth map.

3. A method as described in claim 2, wherein the examining includes calculating a density map from the depth map, the density map describing density of values of depth described by the depth map, the density usable to define the confidence of the values.

4. A method as described in claim 3, wherein the determining further comprises calculating a search map from the density map, the search map defining a search area that is to be used based on the density described by the density map.

5. A method as described in claim 1, wherein the determining is based at least in part on generating a stereo search range based on uniqueness of pixels in relation to other pixels in a respective one of the stereoscopic images.

6. A method as described in claim 5, wherein the determining includes examining uniqueness of patches of pixels to other patches of pixels within a neighborhood defined in the stereoscopic images.

7. A method as described in claim 1, wherein the determining includes selecting between a search range calculated based at least in part on uniqueness of pixels in relation to each other and a search range calculated at least in part based on confidence of values of the depth sensor.

8. A method as described in claim 1, wherein the calculating includes calculating a data cost based on a set of known disparities around a neighborhood of a pixel, the set of known disparities indicated by the depth map.

9. A method as described in claim 1, wherein the calculating includes calculating a smoothness cost based at least in part on guided filtering performed using of a respective one of the stereoscopic images and the depth map.

10. A method as described in claim 1, further comprising up-sampling the depth map such that the depth map has a resolution that substantially corresponds to a resolution of the stereoscopic images.

11. A system comprising:

at least one module implemented at least partially in hardware, the at least one module configured to determine a search range based at least in part on a depth map generated through use of a depth sensor of an image scene; and

one or more modules implemented at least partially in hardware, the one or more modules configured to calculate stereo correspondence of image data of stereoscopic images of an image scene, the calculation configured to be performed using the search range and the depth map.

12. A system as described in claim 11, wherein the at least one module is configured to perform the determination by selecting between a search range calculated based at least in part based on uniqueness of pixels in relation to each other and a search range calculated at least in part based on confidence of values of the depth sensor.

13. A system as described in claim 11, wherein the one or more modules are configured to perform the calculation by calculating a data cost based on a set of known disparities around a fixed neighborhood of a pixel, the set of known disparities indicated by the depth map.

14. A system as described in claim 11, wherein the one or more modules are configured to perform the calculation by calculating a smoothness cost based at least in part on guided filtering based on image edges identified in a respective one of the stereoscopic images and the depth map.

15. One or more computer-readable storage media comprising instructions that are stored thereon that, responsive to execution by a computing device, causes the computing device to perform operations comprising:

receiving a depth map from one or more depth sensors of an image scene, the depth map describing depth of the image scene; and

calculating stereo correspondence of image data of stereoscopic images of the image scene, the calculating performed based at least in part using the depth map.

16. One or more computer-readable storage media as described in claim 15, wherein the calculating includes calculating a data cost based on a set of known disparities around a fixed neighborhood of a pixel, the set of known disparities indicated by the depth map.

17. One or more computer-readable storage media as described in claim 15, wherein the calculating includes calculating a smoothness cost based at least in part on guided filtering performed using image edges identified in a respective one of the stereoscopic images and the depth map.

18. One or more computer-readable storage media as described in claim 15, wherein the operations further comprise determining a search range based at least in part on the depth map, the search range used as part of the calculating of the stereo correspondence.

19. One or more computer-readable storage media as described in claim 18, wherein the determining of the search range is based on least in part on examining confidence of values that are included in the depth map.

20. One or more computer-readable storage media as described in claim 15, wherein the determining of the search range is based on least in part on generating a stereo search range based on uniqueness of pixels in relation to other pixels in a respective one of the stereoscopic images.