Non-uniform spatial resource allocation for depth mapping

• US 8,982,182 B2
• Filed: 02/28/2011
• Issued: 03/17/2015
• Est. Priority Date: 03/01/2010
• Status: Active Grant

• Current Assignee: Apple Inc.
• Original Assignee: Apple Inc.
• Inventors: Shpunt, Alexander, Mor, Zafrir
• Primary Examiner(s): Perungavoor, Sath V
• Assistant Examiner(s): ABIMBOLA, KEHINDE
• Application Number:
  US13/036,023
• Publication Number:
  US 20110211044A1
• Time in Patent Office: 1,478 Days
• Field of Search: None
• US Class Current: 348/42
• CPC Class Codes: G06F 3/005 Input arrangements through a video camera
• CPC Class Codes: G06F 3/011 Arrangements for interaction with the human body, e.g. for user immersion in virtual reality blind teaching G09B21/00
• CPC Class Codes: G06T 2207/10016 Video; Image sequence
• CPC Class Codes: G06T 2207/30196 Human being; Person
• CPC Class Codes: G06T 7/521 from laser ranging, e.g. using interferometry; from the projection of structured light
• CPC Class Codes: H04N 13/254 in combination with electromagnetic radiation sources for illuminating objects
• CPC Class Codes: H04N 13/271 wherein the generated image signals comprise depth maps or disparity maps

Abstract

A method for depth mapping includes providing depth mapping resources including an illumination module, which is configured to project patterned optical radiation into a volume of interest containing the object, and an image capture module, which is configured to capture an image of the pattern reflected from the object. A depth map of the object is generated using the resources while applying at least one of the resources non-uniformly over the volume of interest.

Claims

1. A method for depth mapping, comprising:

providing depth mapping resources comprising;

an illumination module, which is configured to project patterned optical radiation into a volume of interest containing an object; and

an image capture module, which is configured to capture an image of the pattern reflected from the object,

wherein the volume of interest has a depth that varies with angle relative to the illumination and image capture modules; and

generating a depth map of the object using the resources while applying at least one of the resources with an optical resolution that varies non-uniformly over the volume of interest, so as to intentionally distort the image responsively to the varying depth as a function of the angle.

2. The method according to claim 1, wherein the volume of interest has a pentagonal profile, with a vertex at the image capture module.

3. The method according to claim 1, wherein varying the application of the at least one of the resources non-uniformly comprises angularly modulating an intensity distribution of the patterned optical radiation projected by the illumination module.

4. The method according to claim 1, wherein varying the application of the at least one of the resources non-uniformly comprises optically distorting the image captured by the image capture module.

5. The method according to claim 1, wherein the illumination module is configured to project a spatial pattern of the optical radiation onto the object, and wherein generating the depth map comprises deriving depth coordinates of the object based on transverse shifts of the spatial pattern in the image.

20. The method according to claim 1, wherein applying the at least one of the resources with an optical resolution that varies non-uniformly over the volume of interest compensates for smaller apparent sizes of objects appearing at greater distances from the image capture module.

21. The method according to claim 4, wherein the image capture module comprises an image sensor, and wherein optically distorting the image causes objects of a given physical size in different locations at outer limits of the volume of interest to be imaged onto equal areas of the image sensor.

6. A method for depth mapping, comprising:

providing depth mapping resources comprising;

an illumination module, which is configured to project patterned optical radiation into a volume of interest containing an object, wherein the illumination module generates the patterned optical radiation with a given pattern distortion; and

an image capture module, which is configured to capture an image of the pattern reflected from the object, wherein the image capture module comprises objective optics configured to form an image of the pattern on an image sensor while applying an optical distortion to the image that compensates at least partially for the pattern distortion; and

generating a depth map of the object using the resources.

7. The method according to claim 6, wherein the objective optics comprise an F-theta lens.

8. The method according to claim 6, wherein the illumination module comprises a diffractive optical element (DOE) and a radiation emitter configured to transilluminate the DOE so as to generate the patterned optical radiation.

9. Apparatus for depth mapping, comprising:

depth mapping resources comprising;

an illumination module, which is configured to project patterned optical radiation into a volume of interest containing an object; and

an image capture module, which is configured to capture an image of the pattern reflected from the object,

wherein the volume of interest has a depth that varies with angle relative to the illumination and image capture modules, and

wherein at least one of the resources is applied with an optical resolution that varies non-uniformly over the volume of interest so as to intentionally distort the image, responsively to the varying depth as a function of the angle; and

a processor, which is configured to process the image in order to generate a depth map of the object.

10. The apparatus according to claim 9, wherein the volume of interest has a pentagonal profile, with a vertex at the image capture module.

11. The apparatus according to claim 10, wherein an intensity distribution of the patterned optical radiation projected by the illumination module is angularly modulated.

12. The apparatus according to claim 11, wherein the image capture module comprises optics configured to optically distort the image captured by the image capture module.

13. The apparatus according to claim 9, wherein the illumination module is configured to project a spatial pattern of the optical radiation onto the object, and wherein the processor is configured to derive depth coordinates of the object based on transverse shifts of the spatial pattern in the image.

14. The apparatus according to claim 9, wherein the illumination module generates the patterned optical radiation with a given pattern distortion, and wherein the image capture module comprises objective optics configured to form an image of the pattern on an image sensor while applying an optical distortion to the image that compensates at least partially for the pattern distortion.

15. The apparatus according to claim 14, wherein the objective optics comprise an F-theta lens.

16. The apparatus according to claim 14, wherein the illumination module comprises a diffractive optical element (DOE) and a radiation emitter configured to transilluminate the DOE so as to generate the patterned optical radiation.

17. Apparatus for depth mapping, comprising:

an illumination module, which is configured to project patterned optical radiation, with a given pattern distortion, into a volume of interest containing an object; and

an image capture module, which comprises an image sensor and objective optics configured to form an image of the pattern on an image sensor while applying an optical distortion to the image that compensates at least partially for the pattern distortion; and

a processor, which is configured to process the image in order to generate a depth map of the object.

18. The apparatus according to claim 17, wherein the objective optics comprise an F-theta lens.

19. The apparatus according to claim 17, wherein the illumination module comprises a diffractive optical element (DOE) and a radiation emitter configured to transilluminate the DOE so as to generate the patterned optical radiation.