Depth projector system with integrated VCSEL array

• US 8,320,621 B2
• Filed: 12/21/2009
• Issued: 11/27/2012
• Est. Priority Date: 12/21/2009
• Status: Active Grant

• Current Assignee: Microsoft Technology Licensing LLC (Microsoft Corporation)
• Original Assignee: Microsoft Corporation
• Inventors: McEldowney, Scott
• Primary Examiner(s): Azarian, Seyed
• Application Number:
  US12/643,114
• Publication Number:
  US 20120051588A1
• Time in Patent Office: 1,072 Days
• Field of Search: 382/100, 382/103, 382/107, 382/106, 382/123, 382/155, 382/162, 382/168, 382/180, 382/181, 382189-199, 382/219, 382/232, 382/254, 382/274, 382/276, 382/291, 382/312, 382/305, 382/154, 375/240.08, 356/5.01, 359/9
• US Class Current: 382/103
• CPC Class Codes: G03B 17/54 with projector
• CPC Class Codes: G03B 21/14 Details
• CPC Class Codes: G03B 21/2033 LED or laser light sources
• CPC Class Codes: G03B 35/00 Stereoscopic photography
• CPC Class Codes: G03B 35/08 by simultaneous recording
• CPC Class Codes: G06F 3/005 Input arrangements through a video camera
• CPC Class Codes: G06F 3/011 Arrangements for interaction with the human body, e.g. for user immersion in virtual reality blind teaching G09B21/00
• CPC Class Codes: G06F 3/017 Gesture based interaction, e.g. based on a set of recognized hand gestures interaction based on gestures traced on a digitiser G06F3/04883

Abstract

A projector is disclosed for use in a 3-D imaging device. The projector includes a light source formed of a vertical-cavity surface-emitting laser, or VCSEL array. The VCSEL array provides a light source for illuminating a capture area. Light from the VCSEL array is reflected off of objects in the capture area and received within a sensing device such as a 3-D camera. The projector may further include a collimating lens array for focusing the light emitted from each VCSEL in the array, as well as a DOE for patterning the light from the collimating lens array to enable the sensing device to generate a 3-D image of the objects in the capture area.

Claims

1. A depth capturing device for providing a depth image of a capture area, comprising:

a projector having a VCSEL array for providing a light source illuminating the capture area;

an imaging device for receiving light reflected back from one or more objects in the capture area;

a processor for processing light received by the imaging device into a 3-D image of the one or more objects in the capture area;

a lens array including lenses associated with VCSELs in the VCSEL array; and

a diffractive optical element for patterning the light received from the lens array.

2. The depth capturing device of claim 1, the diffractive optical element patterning the light received from the lens array into a pattern enabling 3-D imaging of the one or more objects by a structured light technique.

3. A system for recognizing, analyzing and tracking at least one user in a capture area, comprising:

a projector having a VCSEL array for providing a light source illuminating the capture area;

at least one depth camera providing a depth image of the capture area;

at least one RGB camera providing an RGB image of the capture area; and

at least one processor that receives the depth image and the RGB image and that processes the depth image and the RGB image to recognize the at least one user and to track movement of the at least one user over time.

4. The system of claim 3, the projector further comprising:

a lens array, one lens for each VCSEL in the VCSEL array; and

a diffractive optical element for patterning the light received from the lens array.

5. The system of claim 4, the at least one processor processing the depth image and the RGB image using one of a structured light process and a time-of-flight process.

6. The system of claim 3, the system further comprising a library for storing predefined gestures, the system capable of matching user movement recognized by the system to a predefined gesture stored in the library.

7. The system of claim 3, further comprising a housing that houses the at least one depth camera, the at least one RGB camera, the at least one microphone and the at least one processor.

8. The system of claim 3, wherein the at least one processor generates a skeletal model of the user based at least in part on the depth image.

9. The system of claim 8, wherein the at least one processor uses the skeletal model derived from the depth image to track movement of the user over time.

10. The system of claim 9, wherein when the at least one processor is unable to track movement of the user from the depth image, the processor uses the RGB image to supplement the depth image.

11. The system of claim 3, wherein the movement of the user is tracked over time based at least in part on known mechanics of the human muscular-skeletal system.

12. The system of claim 11, wherein the at least one processor generates a motion capture file of the movements of the user in real-time based on the tracked model.

13. The system of claim 12, wherein the at least one processor applies the motion capture file to an avatar.

14. The system of claim 3, further comprising at least one directional microphone, the at least one microphone comprises a directional microphone, wherein the sound information provided by the at least one microphone is used by the at least one processor to distinguish between a plurality of users in the capture area based on recognition of each user'"'"'s voice.

15. A method for tracking a human user in a capture area, comprising:

illuminating the capture area with light emitted from a VCSEL array and patterned by a diffractive optical element;

receiving from at least one depth camera a depth image of the capture area illuminated by the VCSEL array;

receiving from at least one RGB camera an RGB image of the capture area; and

recognizing and tracking the movement of the user in the capture area over time based on the depth image and the RGB image.

16. The method of claim 15, wherein the method is performed by at least one processor disposed within a housing together with the at least one VCSEL array, depth camera, the at least one microphone, and the at least one RGB camera.

17. The method of claim 15, further comprising the step of receiving from at least one microphone information about sound emanating from the capture area, and wherein said recognizing and tracking comprises recognizing and tracking the movement of the user in the capture area over time based on a combination of at least two of the depth image, the RGB image and the sound information.

18. The method of claim 15, further comprising the step of generating a skeletal model of the user from the depth image and using the skeletal model to track movement of the user over time based at least in part on known mechanics of the human muscular-skeletal system.

19. The method of claim 18, further comprising the step of recognizing a predefined gesture of the user based the tracked movement of the user and a library of stored, predefined gestures.