Wide angle depth detection

• US 9,696,427 B2
• Filed: 08/14/2012
• Issued: 07/04/2017
• Est. Priority Date: 08/14/2012
• Status: Active Grant

• Current Assignee: Microsoft Technology Licensing LLC (Microsoft Corporation)
• Original Assignee: Microsoft Technology Licensing LLC (Microsoft Corporation)
• Inventors: Benko, Hrvoje, Kapur, Jay, Hodges, Stephen Edward, Wilson, Andrew
• Primary Examiner(s): Vaughn, Jr., William C
• Assistant Examiner(s): Tekle, Daniel
• Application Number:
  US13/585,606
• Publication Number:
  US 20140049609A1
• Time in Patent Office: 1,785 Days
• Field of Search: 348 46
• CPC Class Codes: G01B 11/2513 with several lines being projected in more than one direction, e.g. grids, patterns
• CPC Class Codes: G01S 17/894 3D imaging with simultaneous measurement of time-of-flight at a 2D array of receiver pixels, e.g. time-of-flight cameras or flash lidar
• CPC Class Codes: H04N 13/254 in combination with electromagnetic radiation sources for illuminating objects
• CPC Class Codes: H04N 13/271 wherein the generated image signals comprise depth maps or disparity maps

Abstract

Embodiments for a depth sensing camera with a wide field of view are disclosed. In one example, a depth sensing camera comprises an illumination light projection subsystem, an image detection subsystem configured to acquire image data having a wide angle field of view, a logic subsystem configured to execute instructions, and a data-holding subsystem comprising stored instructions executable by the logic subsystem to control projection of illumination light and to determine depth values from image data acquired via the image sensor. The image detection subsystem comprises an image sensor and one or more lenses.

Claims

1. A depth sensing camera, comprising:

an illumination light projection subsystem;

an image detection subsystem configured to acquire image data including one or more images, each of the one or more images having a field of view of 100 degrees or greater, the image detection subsystem comprising an image sensor and one or more lenses, wherein the image sensor comprises a plurality of non-uniformly-sized pixels;

a logic subsystem configured to execute instructions; and

a data-holding subsystem comprising stored instructions executable by the logic subsystem to control projection of illumination light and to determine depth values from image data acquired via the image sensor.

2. The depth sensing camera of claim 1, wherein the illumination light projection subsystem comprises one or more coherent infrared light sources, and wherein the instructions are executable to determine depth values based on a round-trip time-of-flight of light emitted by the one or more coherent infrared light sources and reflected back to the image sensor.

3. The depth sensing camera of claim 2, wherein the instructions are further executable to correct for radial distortion.

4. The depth sensing camera of claim 2, wherein the instructions are executable to correct for radial distortion by performing one or more of upsampling and downsampling regions of an image detected by the image detection subsystem and converting image data to Cartesian coordinates.

5. The depth sensing camera of claim 1, wherein the illumination light projection subsystem comprises a projector configured to project a structured light pattern, and wherein the instructions are executable to determine depth values based on an image of the structured light pattern as reflected from one or more objects.

6. The depth sensing camera of claim 5, wherein the projector is configured to project a structured light pattern with more widely spaced pattern elements at a periphery of the structured light pattern than at a center of the structured light pattern.

7. The depth sensing camera of claim 5, wherein the instructions are executable to control a density of structured light features in the structured light pattern.

8. The depth sensing camera of claim 5, wherein the instructions are further executable to correct for radial distortion and convert image data to Cartesian coordinates.

9. The depth sensing camera of claim 1, wherein the image detecting subsystem comprises a mechanically scanning camera and the image sensor comprises a one-dimensional array of pixels.

10. The depth sensing camera of claim 1, wherein the illumination light projection subsystem is configured to control a field of illumination by dynamically directing illumination light to a region of interest.

11. The depth sensing camera of claim 1, wherein the plurality of non-uniformly-sized pixels provides lower resolution towards the periphery of the sensor surface as compared to the resolution at the center of the sensor surface.

12. The depth sensing camera of claim 1, wherein the depth sensing camera is configured to be mounted in a light socket.

13. The depth sensing camera of claim 12, further comprising a communication subsystem configured to communicate via a wireless network.

14. A time-of-flight depth camera, comprising:

an illumination light projection subsystem comprising one or more infrared light sources;

an image detection subsystem configured to acquire image data having a field of view of 170 degrees or greater, the image detection subsystem comprising an image sensor comprising a plurality of non-uniformly-sized pixels, and one or more lenses;

a logic subsystem configured to execute instructions; and

a data-holding subsystem comprising stored instructions executable by the logic sub system to;

control projection of the one or more infrared light sources;

acquire image data via the image detection subsystem;

correct for radial distortion in the image data to generate corrected image data; and

determine depth values from the corrected image data.

15. The time-of-flight depth camera of claim 14, wherein the instructions are further executable to correct for radial distortion by converting the image data to Cartesian coordinates.

16. The time-of-flight depth camera of claim 14, wherein the instructions are executable to correct for radial distortion by performing one or more of upsampling and downsampling regions of an image detected by the image detection subsystem.

17. The time-of-flight depth camera of claim 14, wherein the plurality of non-uniformly-sized pixels provides lower resolution towards the periphery of the sensor surface as compared to the resolution at the center of the sensor surface.

18. In a structured light depth camera including a projector, an image sensor comprising a plurality of non-uniformly-sized pixels, and one or more lenses, a method of acquiring image data comprising:

projecting a pre-distorted structured light pattern including a plurality of non-uniformly-spaced pattern elements having a lower resolution at a periphery of the structured light pattern than at a center of the structured light pattern;

acquiring image data via the plurality of non-uniformly-sized pixels of the image sensor, the image data capturing the structured light pattern as reflected from a depth imaging environment;

correcting for radial distortion in the image data to generate corrected image data; and

determining depth values from the corrected image data.

19. The method of claim 18, wherein the structured light pattern has a projected field of view of 100 degrees or greater, and wherein the plurality of non-uniformly-spaced pattern elements have a wider spacing at the periphery of the structured light pattern than at the center of the structured light pattern.

20. The method of claim 18, wherein correcting for radial distortion includes converting image data to Cartesian coordinates.