Use of wavefront coding to create a depth image

• US 8,558,873 B2
• Filed: 06/16/2010
• Issued: 10/15/2013
• Est. Priority Date: 06/16/2010
• Status: Active Grant

• Current Assignee: Microsoft Technology Licensing LLC (Microsoft Corporation)
• Original Assignee: Microsoft Corporation
• Inventors: McEldowney, Scott
• Primary Examiner(s): HOLDER, ANNER N
• Application Number:
  US12/816,985
• Publication Number:
  US 20110310226A1
• Time in Patent Office: 1,217 Days
• Field of Search: 348/46, 382/103, 382/128
• US Class Current: 348/46
• CPC Class Codes: G01B 11/2513 with several lines being projected in more than one direction, e.g. grids, patterns
• CPC Class Codes: G01S 17/46 Indirect determination of position data
• CPC Class Codes: G01S 17/89 for mapping or imaging
• CPC Class Codes: G06F 3/017 Gesture based interaction, e.g. based on a set of recognized hand gestures interaction based on gestures traced on a digitiser G06F3/04883
• CPC Class Codes: G06F 3/0325 using a plurality of light emitters or reflectors or a plurality of detectors forming a reference frame from which to derive the orientation of the object, e.g. by triangulation or on the basis of reference deformation in the picked up image

Abstract

A 3-D depth camera system, such as in a motion capture system, tracks an object such as a human in a field of view using an illuminator, where the field of view is illuminated using multiple diffracted beams. An image sensing component obtains an image of the object using a phase mask according to a double-helix point spread function, and determines a depth of each portion of the image based on a relative rotation of dots of light of the double-helix point spread function. In another aspect, dual image sensors are used to obtain a reference image and a phase-encoded image. A relative rotation of features in the images can be correlated with a depth. Depth information can be obtained using an optical transfer function of a point spread function of the reference image.

Claims

1. A 3-D depth camera system, comprising:

(a) an illuminator, the illuminator comprising;

(i) at least one collimated light source which provides a collimated light beam;

(ii) a diffractive optical element which receives the collimated light beam, and creates a plurality of diffracted light beams which illuminate a field of view including a human target;

(b) an image sensor which provides a detected image of the human target using reflections of the diffracted light beams from different portions of the human target, the image sensor includes a phase mask which adjusts the detected image so that a point spread function of each of the reflections of the diffracted light beams from the human target is imaged as a double helix; and

(c) at least one processor associated with the image sensor which determines depth information of the different portions of the human target, and in response to the depth information, distinguishes motion of the human target in the field of view.

2. The 3-D depth camera system of claim 1, wherein:

the at least one processor tracks movement of the human target over time based on the depth information, and provides a corresponding control input to an application, and the application updates a display based on the control input.

3. The 3-D depth camera system of claim 1, wherein:

the diffractive optical element creates the plurality of diffracted light beams as a plurality of diffractive orders; and

the at least one processor determines the depth information of the different portions of the human target based on a rotation of the double helix of each diffractive order of the detected image.

4. The 3-D depth camera system of claim 3, wherein:

the at least one processor determines the depth information of the different portions of the human target based on a rotation of two distinct spots of light which are offset from one another according to a rotation angle.

5. The 3-D depth camera system of claim 4, wherein:

the rotation angle varies by 180 degrees over a depth range of at least one meter.

6. The 3-D depth camera system of claim 1, wherein:

the at least one processor determines the depth information based on at least an intensity distribution of the detected image.

20. The 3-D depth camera system of claim 1, wherein:

the reflections of the diffracted light beams from the different portions of the human target comprise a continuum of rays which are reflected from the different portions of the human target.

7. A 3-D depth camera system for imaging an object in a field of view, comprising:

a first sensor which provides a reference image of the object without using a phase mask, the reference image comprises an image feature which does not appear redundantly in the reference image;

a second sensor which provides a coded image of the object using a phase mask, the coded image comprises redundant appearances of the image feature which are offset from one another according to a rotation angle, and the phase mask encodes light from the object according to a double helix point spread function; and

at least one processor which determines depth information of the object based on at least a light intensity distribution of the reference image (i_ref) and a light intensity distribution of the coded image (i_d).

8. The 3-D depth camera system of claim 7, wherein:

the reference image and the coded image are obtained at a same point in time.

9. The 3-D depth camera system of claim 7, wherein:

the at least one processor tracks movement of the object based on the depth information, and provides a corresponding control input to an application, and the application updates a display based on the control input.

10. The 3-D depth camera system of claim 9, wherein:

the object is a human target.

11. The 3-D depth camera system of claim 7, wherein:

the at least one processor, in response to the depth information, distinguishes motion of the object in the field of view.

12. The 3-D depth camera system of claim 7, wherein:

the reference image is uncoded.

13. The 3-D depth camera system of claim 7, wherein:

the at least one processor determines the depth information using an optical transfer function (H_ref) of a point spread function of the reference image.

14. The 3-D depth camera system of claim 13, wherein:

the at least one processor determines the depth information according to;

F<sup>−

1</sup> {F(i_dh)×

H_ref / F(i_ref)}, where F denotes a Fourier transform and F<sup>−

1</sup> denotes an inverse Fourier transform.

15. A 3-D depth camera system for imaging an object in a field of view, comprising:

a first sensor which provides a reference image of the object, the reference image comprises an image feature which does not appear redundantly in the reference image;

a second sensor which provides a coded image of the object using a phase mask, the phase mask encodes light from the object according to a double helix point spread function, the coded image comprises redundant appearances of the image feature which are offset from one another according to a rotation angle; and

at least one processor which determines the rotation angle based on the reference image and the coded image and determines depth information of the object based on the rotation angle.

16. The 3-D depth camera system of claim 15, wherein:

the first sensor detects light at a near-infrared wavelength to provide the reference image; and

the second sensor detects light at a near-infrared wavelength to provide the coded image.

17. The 3-D depth camera system of claim 15, wherein:

the 3-D depth camera system does not use an illuminator.

18. The 3-D depth camera system of claim 15, wherein:

the 3-D depth camera system does not illuminate the object with coded light.

19. The 3-D depth camera system of claim 15, wherein:

a depth range over which the redundant appearances of the image feature rotate 180 degrees is at least one meter.