GENERATING A DEPTH MAP

• US 20130100256A1
• Filed: 10/21/2011
• Published: 04/25/2013
• Est. Priority Date: 10/21/2011
• Status: Active Grant

• Current Assignee: Microsoft Technology Licensing LLC (Microsoft Corporation)
• Original Assignee: Microsoft Corporation
• Inventors: Kirk, Adam G., Eshet, Yaron, Patiejunas, Kestutis, Kang, Sing Bing, Zitnick, Charles Lawrence III, Eraker, David, Winder, Simon
• Granted Patent: US 9,098,908 B2
• Time in Patent Office: Days
• US Class Current: 348/48
• CPC Class Codes: G06T 2207/10048 Infrared image
• CPC Class Codes: G06T 7/521 from laser ranging, e.g. using interferometry; from the projection of structured light
• CPC Class Codes: G06T 7/593 from stereo images

Abstract

Methods and systems for generating a depth map are provided. The method includes projecting an infrared (IR) dot pattern onto a scene. The method also includes capturing stereo images from each of two or more synchronized IR cameras, detecting a number of dots within the stereo images, computing a number of feature descriptors for the dots in the stereo images, and computing a disparity map between the stereo images. The method further includes generating a depth map for the scene using the disparity map.

Claims

1. A method for generating a depth map, comprising:

projecting an infrared (IR) dot pattern onto a scene;

capturing stereo images from each of two or more synchronized IR cameras;

detecting a plurality of dots within the stereo images;

computing a plurality of feature descriptors corresponding to the plurality of dots in the stereo images;

computing a disparity map between the stereo images; and

generating a depth map for the scene using the disparity map.

2. The method of claim 1, further comprising generating a depth map for each of two or more active IR stereo modules, wherein each active IR stereo module comprises an IR projector, two or more synchronized IR cameras, one or more synchronized RGB camera, or any combinations thereof.

3. The method of claim 2, comprising genlocking the two or more active IR stereo modules using a synchronization signal, wherein genlocking the two or more active IR stereo modules comprises genlocking all cameras within the two or more active IR stereo modules.

4. The method of claim 2, comprising combining the depth maps for each of the two or more active IR stereo modules to create a constructive view of the scene.

5. The method of claim 2, comprising projecting a plurality of IR dot patterns onto the scene from any number of the two or more active IR stereo modules.

6. The method of claim 5, comprising utilizing the plurality of IR dot patterns as one mutually-contributed IR dot pattern.

7. The method of claim 1, comprising capturing stereo images from each of two or more synchronized IR cameras, one or more synchronized RGB camera, or any combination thereof.

8. The method of claim 1, wherein computing the plurality of feature descriptors corresponding to the plurality of dots in the stereo mages comprises using a binning method to match similarities between corresponding dots within the stereo images.

9. The method of claim 1, comprising using differences between the plurality of feature descriptors for the plurality of dots in each of the stereo images to compute the disparity map between the stereo images.

10. A stereo module system for generating a depth map, comprising:

a processor configured to implement active IR stereo modules, wherein the active IR stereo modules comprise;

an infrared (IR) pattern projection module configured to project an IR dot pattern onto a scene;

a synchronized camera module configured to capture stereo images from each of two or more temporally-synchronized IR cameras;

a dot detection module configured to identify a plurality of dots within the stereo images;

a feature description module configured to compute a plurality of feature descriptors corresponding to the plurality of dots in the stereo images;

a disparity computation module configured to compute a disparity map between the stereo images; and

a depth map generation module configured to generate a depth map for the scene using the disparity map.

11. The stereo module system of claim 10, further comprising:

a multiview depth generation module configured to generate a depth map for each of two or more synchronized active IR stereo modules; and

a depth map combination module configured to combine the depth maps from each of the two or more active IR stereo modules to create a constructive view of the scene.

12. The multiview depth generation module of claim 11, wherein the depth map for each of the two or more synchronized active IR stereo modules is obtained using a combination of a plurality of IR dot patterns projected onto the scene.

13. The depth map combination module of claim 11, wherein the constructive view of the scene comprises a three-dimensional view of the scene.

14. The stereo module system of claim 10, wherein the synchronized camera module comprises a temporally-synchronized RGB camera.

15. The stereo module system of claim 10, wherein the disparity computation module is configured to compute the disparity map between the stereo images using differences between the plurality of feature descriptors corresponding to the plurality of dots in the stereo images.

16. One or more non-volatile computer-readable storage media for storing computer readable instructions, the computer-readable instructions providing a stereo module system for the generation of a depth map when executed by one or more processing modules, the computer-readable instructions comprising code configured to:

project an infrared (IR) dot pattern onto a scene;

capture stereo images from two or more synchronized IR cameras;

detect a plurality of dots within the stereo images;

compute a plurality of feature descriptors corresponding to the plurality of dots in the stereo images;

compute a disparity map between the stereo images; and

generate the depth map for the scene using the disparity map.

17. The non-volatile computer-readable storage media of claim 16, wherein the computer-readable instructions comprise code further configured to:

generate a depth map for each of two or more synchronized active IR stereo modules; and

combine the depth maps for each of the two or more active IR stereo modules to create a constructive view of the scene.

18. The non-volatile computer-readable storage media of claim 17, wherein generating a depth map for each of the two or more synchronized active IR stereo modules comprises using a combination of a plurality of IR dot patterns projected onto the scene.

19. The non-volatile computer-readable storage media of claim 17, wherein combining the depth maps for each of the two or more active IR stereo modules to create a constructive view of the scene comprises generating a constructive view of the scene in real-time.

20. The non-volatile computer-readable storage media of claim 16, wherein computing the plurality of feature descriptors corresponding to the plurality of dots in the stereo images comprises using the differences between the plurality of feature descriptors for the plurality of dots in each of the stereo images.