TRACKING HEAD MOVEMENT WHEN WEARING MOBILE DEVICE

• US 20140375680A1
• Filed: 06/24/2013
• Published: 12/25/2014
• Est. Priority Date: 06/24/2013
• Status: Active Grant

• Current Assignee: Microsoft Technology Licensing LLC (Microsoft Corporation)
• Original Assignee: Microsoft Technology Licensing LLC (Microsoft Corporation)
• Inventors: Ackerman, Nathan, Steedly, Drew, Kipman, Alex Aben-Athar, Hodge, Andy
• Granted Patent: US 9,256,987 B2
• Time in Patent Office: Days
• US Class Current: 345/633
• CPC Class Codes: G02B 2027/014 comprising information/image processing systems
• CPC Class Codes: G02B 2027/0178 Eyeglass type eyeglass details G02C
• CPC Class Codes: G02B 2027/0187 slaved to motion of at least a part of the body of the user, e.g. head, eye
• CPC Class Codes: G02B 27/0093 with means for monitoring data relating to the user, e.g. head-tracking, eye-tracking
• CPC Class Codes: G02B 27/017 Head mounted
• CPC Class Codes: G06F 2200/1637 Sensing arrangement for detection of housing movement or orientation, e.g. for controlling scrolling or cursor movement on the display of an handheld computer
• CPC Class Codes: G06F 3/012 Head tracking input arrangements
• CPC Class Codes: G06T 19/006 Mixed reality object pose determination, tracking or camera calibration for mixed reality G06T7/00

Abstract

Methods for tracking the head position of an end user of a head-mounted display device (HMD) relative to the HMD are described. In some embodiments, the HMD may determine an initial head tracking vector associated with an initial head position of the end user relative to the HMD, determine one or more head tracking vectors corresponding with one or more subsequent head positions of the end user relative to the HMD, track head movements of the end user over time based on the initial head tracking vector and the one or more head tracking vectors, and adjust positions of virtual objects displayed to the end user based on the head movements. In some embodiments, the resolution and/or number of virtual objects generated and displayed to the end user may be modified based on a degree of head movement of the end user relative to the HMD.

Claims

1. An electronic device for displaying an augmented reality environment, comprising:

one or more processors, the one or more processors determine an initial head position of an end user of the electronic device relative to the electronic device at a first point and time, the one or more processors determine a second head position of the end user relative to the electronic device at a second point in time subsequent to the first point in time, the second head position is different from the initial head position, the one or more processors render a second image associated with a virtual object based on the second head position; and

a see-through display in communication with the one or more processors, the see-through display displays the second image such that the virtual object is perceived to exist within an augmented reality environment displayed to the end user of the electronic device.

2. The electronic device of claim 1, wherein:

the electronic device comprises an HMD.

3. The electronic device of claim 2, wherein:

the one or more processors determine a degree of head movement relative to the HMD based on the initial head position and the second head position, the one or more processors determine whether the degree of head movement relative to the HMD exceeds a threshold.

4. The electronic device of claim 3, wherein:

the one or more processors reduce a resolution of the second image if the degree of head movement relative to the HMD exceeds the threshold.

5. The electronic device of claim 3, wherein:

the see-through display displays a warning to the end user if the degree of head movement exceeds the threshold.

6. The electronic device of claim 3, wherein:

the one or more processors determine the degree of head movement relative to the HMD by determining a maximum vector difference among a plurality of head tracking vectors, a first head tracking vector of the plurality of head tracking vectors is based on a difference between the second head position and the initial head position.

7. The electronic device of claim 1, wherein:

the one or more processors determine an absolute head position vector representing a head position of the end user relative to an environment in which the HMD exists.

8. A method for tracking a head position of an end user of an HMD relative to the HMD, comprising:

determining an initial head position of the end user relative to the HMD at a first point and time;

rendering and displaying a first image associated with a virtual object based on the initial head position of the end user relative to the HMD, the first image is displayed at a first position on the HMD;

determining a second head position of the end user relative to the HMD at a second point in time subsequent to the first point in time, the second head position is different from the initial head position; and

rendering and displaying a second image associated with the virtual object based on the second head position of the end user relative to the HMD, the second image is displayed at a second position on the HMD different from the first position.

9. The method of claim 8, further comprising:

determining a degree of head movement relative to the HMD based on the initial head position and the second head position; and

determining whether the degree of head movement relative to the HMD exceeds a threshold.

10. The method of claim 9, further comprising:

reducing a resolution of the second image if the degree of head movement relative to the HMD exceeds the threshold.

11. The method of claim 9, further comprising:

displaying a warning to the end user if the degree of head movement exceeds the threshold.

12. The method of claim 9, wherein:

the determining a degree of head movement relative to the HMD includes determining a maximum vector difference among a plurality of head tracking vectors, a first head tracking vector of the plurality of head tracking vectors is based on a difference between the second head position and the initial head position.

13. The method of claim 8, wherein:

the determining an initial head position includes capturing one or more images of the end user while the end user is wearing the HMD, the one or more images capture at least a portion of an eye of the end user, the determining an initial head position includes identifying one or more feature points associated with the eye of the end user by applying image processing techniques to the one or more images and determining a position and orientation associated with a 3D model of the end user'"'"'s eyes based on the identifying one or more feature points.

14. The method of claim 13, wherein:

the capturing one or more images is performed by an inward facing camera associated with the HMD.

15. The method of claim 8, further comprising:

determining an absolute head position vector representing a head position of the end user relative to an environment in which the HMD exists.

16. One or more storage devices containing processor readable code for programming one or more processors to perform a method for tracking a head position of an end user of an HMD relative to the HMD comprising the steps of:

capturing a first set of images of the end user at a first point in time, the first set of images capture at least a portion of an eye of the end user;

determining an initial head position of the end user relative to the HMD based on the first set of images;

rendering and displaying a first image associated with a virtual object based on the initial head position of the end user relative to the HMD;

capturing a second set of images of the end user at a second point in time subsequent to the first point in time;

determining a second head position of the end user relative to the HMD based on the second set of images, the second head position is different from the initial head position; and

rendering and displaying a second image associated with the virtual object based on the second head position of the end user relative to the HMD.

17. The one or more storage devices of claim 16, further comprising:

determining a degree of head movement relative to the HMD based on the initial head position and the second head position; and

determining whether the degree of head movement relative to the HMD exceeds a threshold.

18. The one or more storage devices of claim 17, further comprising:

reducing a resolution of the second image if the degree of head movement relative to the HMD exceeds the threshold.

19. The one or more storage devices of claim 17, further comprising:

displaying a warning to the end user if the degree of head movement exceeds the threshold.

20. The one or more storage devices of claim 16, further comprising:

determining an absolute head position vector representing a head position of the end user relative to an environment in which the HMD exists; and

transmitting the absolute head position vector to a server.