EXTERNAL IR ILLUMINATOR ENABLING IMPROVED HEAD TRACKING AND SURFACE RECONSTRUCTION FOR VIRTUAL REALITY

• US 20190304116A1
• Filed: 03/27/2018
• Published: 10/03/2019
• Est. Priority Date: 03/27/2018
• Status: Active Grant

• Current Assignee: Microsoft Technology Licensing LLC (Microsoft Corporation)
• Original Assignee: Microsoft Technology Licensing LLC (Microsoft Corporation)
• Inventors: Price, Raymond Kirk, Bleyer, Michael, Messer, Christopher Stephen
• Granted Patent: US 10,565,720 B2
• Time in Patent Office: Days
• CPC Class Codes: G01B 11/22 for measuring depth
• CPC Class Codes: G01B 11/2513 with several lines being projected in more than one direction, e.g. grids, patterns
• CPC Class Codes: G01S 17/02 Systems using the reflection of electromagnetic waves other than radio waves G01S17/66 takes precedence
• CPC Class Codes: G02B 2027/0138 comprising image capture systems, e.g. camera
• CPC Class Codes: G02B 27/017 Head mounted
• CPC Class Codes: G06F 3/011 Arrangements for interaction with the human body, e.g. for user immersion in virtual reality blind teaching G09B21/00
• CPC Class Codes: G06T 2207/10028 Range image; Depth image; 3D point clouds
• CPC Class Codes: G06T 7/521 from laser ranging, e.g. using interferometry; from the projection of structured light
• CPC Class Codes: G06T 7/593 from stereo images
• CPC Class Codes: H04N 13/254 in combination with electromagnetic radiation sources for illuminating objects
• CPC Class Codes: H04N 13/344 with head-mounted left-right displays

Abstract

Disclosed embodiments include methods and systems for utilizing a structured projection pattern to perform depth detection. In some instances, the structured projection pattern forms a dot pattern, which is projected by a projector, wherein the projector includes one or more infrared (IR) light dot pattern illuminators for projecting an IR light dot pattern to a surrounding environment. The IR dot pattern light that is at least partially reflected off one or more objects in the surrounding environment is detected by one or more cameras attached to a head-mounted display (HMD). The HMD, which is physically untethered from the projector, utilizes the captured IR dot pattern light reflections to track movement of the HMD and/or perform depth detection of one or more objects in the environment surrounding the HMD.

Claims

1. A system for performing active stereo depth computation of a head-mounted display (HMD) in a surrounding environment, the system comprising:

a projector comprising one or more dot pattern illuminators for projecting a dot pattern to the surrounding environment and that is at least partially reflected in the surrounding environment as reflected light; and

the HMD, wherein the HMD is physically untethered from the projector, the HMD comprising;

a first camera and a second camera having an overlapping field of view; and

one or more processors and one or more computer-readable media having stored thereon computer-executable instructions that are operable, when executed by the one or more processors, to cause the HMD to compute a depth of one or more objects within the overlapping field of view relative to the HMD and based at least in part on the reflected light from the dot pattern that is detected by both the first camera and the second camera.

2. The system of claim 1, wherein the system further includes one or more flood illuminators for projecting a flood of light to the surrounding environment and that is at least partially reflected from in the surrounding environment as reflected flood light detected by at least one of the first and second camera and that is used to track the movement of the HMD in the surrounding environment.

3. The system of claim 1, wherein the projector is an omnidirectional projector and wherein the one or more dot pattern illuminators project the dot pattern omnidirectionally to the surrounding environment.

4. The system of claim 1, wherein the projector is an omnidirectional projector that includes one or more flood illuminators that project a flood of light omnidirectionally to the surrounding environment.

5. The system of claim 1, wherein the one or more dot pattern illuminators are pulsed to synchronize with an exposure of at least one of the first camera or the second camera.

6. A system for performing tracking of a head-mounted display (HMD) in a surrounding environment, the system comprising:

a projector comprising one or more infrared light dot pattern illuminators for projecting an infrared (IR) light dot pattern to the surrounding environment and that is at least partially reflected in the surrounding environment as reflected IR light; and

the HMD, wherein the HMD is physically untethered from the projector, the HMD comprising;

a camera system having at least one camera sensor for detecting the reflected IR light; and

one or more processors and one or more computer-readable media having stored thereon computer-executable instructions that are operable, when executed by the one or more processors, to cause the HMD to track movement of the HMD based at least in part on the reflected IR light that is detected by the at least one camera sensor.

7. The system of claim 6, wherein the camera system comprises a first camera and a second camera having an overlapping field of view and wherein the first camera and the second camera obtain corresponding images of the reflected IR light to perform depth detection of at least one object in the surrounding environment.

8. The system of claim 6, wherein the projector is an omnidirectional projector and wherein the one or more IR light dot pattern illuminators project the IR dot pattern omnidirectionally to the surrounding environment.

9. The system of claim 6, wherein the projector is an omnidirectional projector and wherein the one or more IR flood illuminators project the flood of IR light omnidirectionally to the surrounding environment.

10. The system of claim 6, wherein the one or more infrared light dot pattern illuminators include at least a first illuminator and at least a second illuminator, the first illuminator projecting IR light with a different intensity than an intensity used by the second illuminator.

11. A method for performing at least one of depth detection and tracking with a head-mounted display (HMD) in an environment surrounding an omnidirectional projector that is untethered from the HMD, the method comprising:

projecting an infrared (IR) dot pattern on objects in the environment surrounding the omnidirectional projector;

detecting at least a portion of the IR dot pattern with one or more cameras attached to the HMD; and

utilizing the portion of the IR dot pattern detected by the one or more cameras to perform at least one of depth detection and tracking with the HMD.

12. The method of claim 11, wherein the method includes utilizing the portion of the IR dot pattern detected by the one or more cameras to perform the tracking with the HMD.

13. The method of claim 11 wherein the one or more cameras include a first camera and a second camera and wherein the method includes utilizing the portion of the IR dot pattern detected by the first and second cameras to perform the depth detection with the HMD.