TOTAL FIELD OF VIEW CLASSIFICATION FOR HEAD-MOUNTED DISPLAY

• US 20120327116A1
• Filed: 06/23/2011
• Published: 12/27/2012
• Est. Priority Date: 06/23/2011
• Status: Abandoned Application

• Current Assignee: Microsoft Technology Licensing LLC (Microsoft Corporation)
• Original Assignee: Microsoft Corporation
• Inventors: LIU, JAMES CHIA-MING, LATTA, STEPHEN GILCHRIST, ANDREWS, ANTON, VAUGHT, BENJAMIN ISSAC, NOVAK, CHRISTOPHER MICHAEL, SMALL, SHERIDAN LEIGH
• Application Number:
  US13/167,113
• Publication Number:
  US 20120327116A1
• Time in Patent Office: Days
• US Class Current: 345/633
• CPC Class Codes: G02B 2027/0187 slaved to motion of at least a part of the body of the user, e.g. head, eye
• CPC Class Codes: G02B 27/0093 with means for monitoring data relating to the user, e.g. head-tracking, eye-tracking
• CPC Class Codes: G02B 27/017 Head mounted
• CPC Class Codes: G02B 27/0172 characterised by optical features
• CPC Class Codes: G06F 3/011 Arrangements for interaction with the human body, e.g. for user immersion in virtual reality blind teaching G09B21/00
• CPC Class Codes: G06F 3/017 Gesture based interaction, e.g. based on a set of recognized hand gestures interaction based on gestures traced on a digitiser G06F3/04883
• CPC Class Codes: G09G 5/377 for mixing or overlaying two or more graphic patterns G09G5/02, G09G5/397 take precedence
• CPC Class Codes: G09G 5/38 with means for controlling the display position

Abstract

Virtual images are located for display in a head-mounted display (HMD) to provide an augment reality view to an HMD wearer. Sensor data may be collected from on-board sensors provided on an HMD. Additionally, other day may be collected from external sources. Based on the collected sensor data and other data, the position and rotation of the HMD wearer'"'"'s head relative to the HMD wearer'"'"'s body and surrounding environment may be determined. After resolving the HMD wearer'"'"'s head position, the HMD wearer'"'"'s total field of view (TFOV) may be classified into regions. Virtual images may then be located in the classified TFOV regions to locate the virtual images relative to the HMD wearer'"'"'s body and surrounding environment.

Claims

1. One or more computer storage media storing computer-useable instructions that, when used by one or more computing devices, cause the one or more computing device to perform a method, the method comprising:

receiving sensor data from one or more head-mounted display (HMD) on-board sensors;

using the sensor data to determine an HMD wearer'"'"'s head position and rotation relative to the HMD wearer'"'"'s body and an environment surrounding the HMD wearer;

classifying two or more regions within the HMD wearer'"'"'s total field of view (TFOV) based on one or more pre-determined rules and the HMD wearer'"'"'s head position and rotation relative to the HMD wearer'"'"'s body and an environment surrounding the HMD wearer; and

locating virtual images to be displayed by the HMD based on classifying the two or more regions within the HMD wearer'"'"'s TFOV.

2. The one or more computer storage media of claim 1, wherein the one or more HMD on-board sensors comprise one or more selected from the following:

a GPS sensor, an inertial measurement unit sensor, a depth sensor, a camera, an eye tracking sensor, a microphone, and a biometric sensor.

3. The one or more computer storage media of claim 1, wherein the method further comprises:

receiving other data from one or more sources external to the HMD; and

using the other data in conjunction with the sensor data to determine the HMD wearer'"'"'s head position and rotation relative to the HMD wearer'"'"'s body and the environment surrounding the HMD wearer.

4. The one or more computer storage media of claim 3, wherein the one or more sources external to the HMD comprise an external camera, an external depth sensor, another HMD, a mobile device, and historical sensor data stored remotely from the HMD.

5. The one or more computer storage media of claim 1, wherein the two or more regions are fixed within the HMD wearer'"'"'s TFOV.

6. The one or more computer storage media of claim 1, wherein the two or more regions dynamically change within the HMD wearer'"'"'s TFOV.

7. The one or more computer storage media of claim 1, wherein the two or more regions include a primary region and one or more non-primary regions.

8. The one or more computer storage media of claim 7, locating the virtual images comprises locating the virtual images to reduce occluding the primary region.

9. The one or more computer storage media of claim 1, wherein classifying the two or more regions within the HMD wearer'"'"'s TFOV comprises classifying at least one of the regions relative to the HMD wearer'"'"'s body.

10. The one or more computer storage media of claim 9, wherein locating the virtual reality images to be displayed by the HMD comprises locating at least one of the virtual reality images relative to HMD wearer'"'"'s body based on classifying the at least one of the regions relative to the HMD wearer'"'"'s body.

11. The one or more computer storage media of claim 1, wherein the method further comprises:

determining a field of view (FOV) for the HMD wearer;

comparing the FOV with the two or more regions of the HMD wearer'"'"'s TFOV;

determining a subset of virtual objects to display based on a location of the FOV relative to the two or more regions of the HMD wearer'"'"'s TFOV; and

displaying the subset of virtual objects via a display component of the HMD.

12. The one or more computer storage media of claim 1, wherein the virtual images comprise at least one static virtual image and at least one interactive virtual image.

13. The one or more computer storage media of claim 1, wherein the method further comprises identifying user interaction with an interactive virtual image located in a first region from the two or more regions, the first region being outside the HMD wearer'"'"'s current field of view, and performing an action in response to the user interaction with the interactive virtual image.

14. A head-mounted display (HMD) comprising:

one or more on-board sensors;

one or more processors configured to;

determine a position and rotation of an HMD wearer'"'"'s head relative to the HMD wearer'"'"'s body based on sensor data from the one or more on-board sensors, classify two or more regions of the HMD wearer'"'"'s TFOV based on the position and rotation of the HMD wearer'"'"'s head relative to the HMD wearer'"'"'s body, and locate one or more virtual objects in the two or more regions; and

one or more display components configured to display at least one of the one or more virtual objects to provide the HMD wearer an augmented view.

15. The HMD of claim 14, wherein the HMD further comprises a wireless communication component.

16. The HMD of claim 14, wherein the wireless communication component is configured to receive other data from one or more sources external to the HMD, and wherein the one or more processors are further configured to determine the position and rotation of an HMD wearer'"'"'s head relative to the HMD wearer'"'"'s body based on the other data from the one or more sources external to the HMD.

17. A method for classifying regions of an head-mounted display (HMD) wearer'"'"'s total field of view (TFOV), the method comprising:

continuously;

receiving sensor data from one or more sensors on board the HMD;

receiving other data from one or more sources external to the HMD; and

classifying regions of the HMD wearer'"'"'s TFOV relative to the HMD wearer'"'"'s body and surrounding environment based on the sensor data and other data.

18. The method of claim 17, wherein the regions include a primary region and one or more non-primary regions.

19. The method of claim 18, wherein the method further comprises locating virtual images in the regions based on classifying the regions.

20. The method of claim 19, wherein locating the virtual images comprises locating the virtual images in the regions to reduce occluding the primary region.