Depth sensing with depth-adaptive illumination

• US 8,896,594 B2
• Filed: 06/30/2012
• Issued: 11/25/2014
• Est. Priority Date: 06/30/2012
• Status: Active Grant

• Current Assignee: Microsoft Technology Licensing LLC (Microsoft Corporation)
• Original Assignee: Microsoft Corporation
• Inventors: Xiong, Zhiwei, Wu, Feng, Zhang, Yueyi, Cong, Pengyu
• Primary Examiner(s): McDowell, Jr., Maurice L
• Application Number:
  US13/539,362
• Publication Number:
  US 20140002445A1
• Time in Patent Office: 878 Days
• Field of Search: 345/419, 345/422, 356/603
• US Class Current: 345/419
• CPC Class Codes: G01B 11/2513 with several lines being projected in more than one direction, e.g. grids, patterns
• CPC Class Codes: H04N 13/271 wherein the generated image signals comprise depth maps or disparity maps

Abstract

An adaptive depth sensing system (ADSS) illuminates a scene with a pattern that is constructed based on an analysis of at least one prior-generated depth map. In one implementation, the pattern is a composite pattern that includes two or more component patterns associated with different depth regions in the depth map. The composite pattern may also include different illumination intensities associated with the different depth regions. By using this composite pattern, the ADSS can illuminate different objects in a scene with different component patterns and different illumination intensities, where those objects are located at different depths in the scene. This process, in turn, can reduce the occurrence of defocus blur, underexposure, and overexposure in the image information.

Claims

1. An adaptive depth sensing system, implemented by computing functionality, comprising:

a processor for implementing computing functionality for modules comprising;

a depth determination module configured to generate a depth map based on image information received from a camera device; and

pattern generation functionality configured to;

analyze the depth map to provide an analysis result;

generate a composite pattern based on the analysis result; and

instruct a projector device to project the composite pattern onto a scene,

the depth determination module and pattern generation functionality configured to repeatedly generate depth maps and composite patterns, respectively,

the depth determination module and pattern generation functionality being implemented by the computing functionality.

2. The adaptive depth sensing system of claim 1, wherein the depth determination module and the pattern generation functionality cooperatively interact to reduce occurrence of at least one of:

defocus blur in the image information;

underexposure in the image information; and

overexposure in the image information.

3. The adaptive depth sensing system of claim 1, wherein said pattern generating functionality includes a region analysis module that is configured to:

identify depth regions in the depth map, each depth region corresponding to a region of the scene with similar depths with respect to a reference point; and

identify a set of masks associated with the depth regions.

4. The adaptive depth sensing system of claim 3, wherein said pattern generating functionality also includes a pattern assignment module that is configured to assign component patterns to the respective depth regions, each component pattern including features having a particular property, and different component patterns including features having different respective properties.

5. The adaptive depth sensing system of claim 3, wherein said pattern generating functionality also includes an intensity assignment module that is configured to assign an illumination intensity β

for a depth region j in the depth map, as given by;

6. The adaptive depth sensing system of claim 1, wherein the composite pattern is given by:

7. A method, performed by computing functionality, for generating a depth map, comprising:

receiving image information from a camera device, the image information representing a scene that has been illuminated with a composite pattern;

generating a depth map based on the image information, the depth map having depth regions, each depth region corresponding to a region of the scene having similar depths with respect to a reference point;

generating a new composite pattern having parts that are selected based on the respective depth regions in the depth map;

instructing a projector device to project the new composite pattern onto the scene; and

repeating said receiving, generating a depth map, generating a new composite pattern, and instructing at least one time.

8. The method of claim 7, wherein said generating a new composite pattern includes:

identifying the depth regions in the depth map; and

identifying a set of masks associated with the respective depth regions.

9. The method of claim 7, wherein said generating a new composite pattern includes assigning component patterns to the respective depth regions, each component pattern including features having a particular property, and different component patterns including features having different respective properties.

10. The method of claim 9, wherein the particular property associated with each component pattern is feature size.

11. The method of claim 10, wherein the features associated with each component pattern correspond to speckle features of a particular size.

12. The method of claim 9, wherein the particular property associated with each component pattern is code-bearing design.

13. The method of claim 9, wherein said assigning involves assigning a first component pattern having a first feature size K₁ for a first depth region d₁ and a second component pattern having a second feature size K₂ for a second depth region d₂, where K₁>

K₂ if d₁<

d₂.

14. The method of claim 9, wherein the component patterns are produced using a simulation technique.

15. The method of claim 7, wherein said generating a new composite pattern includes assigning illumination intensities to the respective depth regions.

16. The method of claim 15, wherein an illumination intensity β

for a depth region j is given by;

17. The method of claim 7, wherein the new composite pattern is given by:

18. The method of claim 7, wherein the new composite pattern is formed based on a consideration of n previously generated depth maps, where n≧

2.

19. A computer readable storage device for storing computer readable instructions, the computer readable instructions providing an adaptive depth sensing system when executed by one or more processing devices, the computer readable instructions comprising:

logic configured to receive image information from a camera device, the image information representing a scene that has been illuminated with a composite pattern;

logic configured to generate a depth map based on the image information;

logic configured to identify depth regions in the depth map, each depth region corresponding to a region of the scene with similar depths with respect to a reference point;

logic configured to identify a set of masks associated with the depth regions;

logic configured to assign component patterns to the respective depth regions, each component pattern including features having a particular property, and different component patterns including features having different respective properties;

logic configured to assign illumination intensities to the respective depth regions;

logic configured to produce a new composite pattern based on the masks, component patterns, and illumination intensities; and

logic configured to instruct a projector device to project the new composite pattern onto the scene.

20. The computer readable storage device of claim 19, wherein the new composite pattern is given by: