Tracking head movement when wearing mobile device

• US 9,256,987 B2
• Filed: 06/24/2013
• Issued: 02/09/2016
• Est. Priority Date: 06/24/2013
• Status: Active Grant

• Current Assignee: Microsoft Technology Licensing LLC (Microsoft Corporation)
• Original Assignee: Microsoft Technology Licensing LLC (Microsoft Corporation)
• Inventors: Ackerman, Nathan, Steedly, Drew, Hodge, Andy, Kipman, Alex Aben-Athar
• Primary Examiner(s): SIMPSON, LIXI CHOW
• Application Number:
  US13/925,314
• Publication Number:
  US 20140375680A1
• Time in Patent Office: 960 Days
• US Class Current: 1/1
• CPC Class Codes: G02B 2027/014 comprising information/image processing systems
• CPC Class Codes: G02B 2027/0178 Eyeglass type eyeglass details G02C
• CPC Class Codes: G02B 2027/0187 slaved to motion of at least a part of the body of the user, e.g. head, eye
• CPC Class Codes: G02B 27/0093 with means for monitoring data relating to the user, e.g. head-tracking, eye-tracking
• CPC Class Codes: G02B 27/017 Head mounted
• CPC Class Codes: G06F 2200/1637 Sensing arrangement for detection of housing movement or orientation, e.g. for controlling scrolling or cursor movement on the display of an handheld computer
• CPC Class Codes: G06F 3/012 Head tracking input arrangements
• CPC Class Codes: G06T 19/006 Mixed reality object pose determination, tracking or camera calibration for mixed reality G06T7/00

Abstract

Methods for tracking the head position of an end user of a head-mounted display device (HMD) relative to the HMD are described. In some embodiments, the HMD may determine an initial head tracking vector associated with an initial head position of the end user relative to the HMD, determine one or more head tracking vectors corresponding with one or more subsequent head positions of the end user relative to the HMD, track head movements of the end user over time based on the initial head tracking vector and the one or more head tracking vectors, and adjust positions of virtual objects displayed to the end user based on the head movements. In some embodiments, the resolution and/or number of virtual objects generated and displayed to the end user may be modified based on a degree of head movement of the end user relative to the HMD.

Claims

1. An electronic device for displaying an augmented reality environment, comprising:

one or more processors, the one or more processors determine an initial head position of an end user of the electronic device relative to the electronic device at a first point in time, the one or more processors render a first image associated with a virtual object based on the initial head position, the one or more processors determine a second head position of the end user relative to the electronic device at a second point in time subsequent to the first point in time, the second head position is different from the initial head position, the one or more processors detect that a degree of head movement relative to the electronic device has exceeded a threshold amount based on the initial head position and the second head position, the one or more processors render a second image associated with the virtual object based on the second head position; and

a see-through display in communication with the one or more processors, the see-through display displays the first image using a first resolution for the virtual object, the see-through display displays the second image using a second resolution for the virtual object less than the first resolution in response to detecting that the degree of head movement relative to the electronic device has exceeded the threshold amount.

2. The electronic device of claim 1, wherein:

the electronic device comprises an HMD.

3. The electronic device of claim 2, wherein:

the one or more processors reduce a number of virtual objects displayed on the HMD in response to detecting that the degree of head movement relative to the HMD has exceeded the threshold amount.

4. The electronic device of claim 3, wherein:

the second resolution is half of the first resolution; and

the number of virtual objects displayed on the HMD is cut in half in response to detecting that the degree of head movement relative to the HMD has exceeded the threshold amount.

5. The electronic device of claim 3, wherein:

the see-through display displays a warning to the end user if the degree of head movement exceeds the threshold amount.

6. The electronic device of claim 3, wherein:

the one or more processors detect that the degree of head movement relative to the HMD has exceeded the threshold amount by determining a maximum vector difference among a plurality of head tracking vectors, a first head tracking vector of the plurality of head tracking vectors is based on a difference between the second head position and the initial head position.

7. The electronic device of claim 2, wherein:

the one or more processors determine an absolute head position vector representing a head position of the end user relative to an environment in which the HMD exists.

8. A method for tracking a head position of an end user of an HMD relative to the HMD, comprising:

determining an initial head position of the end user relative to the HMD at a first point in time;

rendering and displaying a first image associated with a virtual object based on the initial head position of the end user relative to the HMD, the first image is displayed on the HMD using a first resolution for the virtual object;

determining a second head position of the end user relative to the HMD at a second point in time subsequent to the first point in time, the second head position is different from the initial head position;

detecting that a degree of head movement relative to the HMD has exceeded a threshold amount based on the initial head position and the second head position; and

rendering and displaying a second image associated with the virtual object based on the second head position of the end user relative to the HMD, the second image is displayed on the HMD using a second resolution for the virtual object less than the first resolution in response to detecting that the degree of head movement relative to the HMD has exceeded the threshold amount.

9. The method of claim 8, further comprising:

reducing a number of virtual objects displayed on the HMD in response to detecting that the degree of head movement relative to the HMD has exceeded the threshold amount.

10. The method of claim 9, wherein:

the second resolution is half of the first resolution; and

the number of virtual objects displayed on the HMD is cut in half in response to detecting that the degree of head movement relative to the HMD has exceeded the threshold amount.

11. The method of claim 9, further comprising:

displaying a warning to the end user if the degree of head movement exceeds the threshold amount.

12. The method of claim 9, wherein:

the detecting that a degree of head movement relative to the HMD has exceeded the threshold amount includes determining a maximum vector difference among a plurality of head tracking vectors, a first head tracking vector of the plurality of head tracking vectors is based on a difference between the second head position and the initial head position.

13. The method of claim 8, wherein:

the determining an initial head position includes capturing one or more images of the end user while the end user is wearing the HMD, the one or more images capture at least a portion of an eye of the end user, the determining an initial head position includes identifying one or more feature points associated with the eye of the end user by applying image processing techniques to the one or more images and determining a position and orientation associated with a 3D model of the end user'"'"'s eyes based on the identifying one or more feature points.

14. The method of claim 13, wherein:

the capturing one or more images is performed by an inward facing camera associated with the HMD.

15. The method of claim 8, further comprising:

determining an absolute head position vector representing a head position of the end user relative to an environment in which the HMD exists.

16. One or more computer readable memory devices containing processor readable code for programming one or more processors to perform a method for tracking a head position of an end user of an HMD relative to the HMD comprising the steps of:

capturing a first set of images of the end user at a first point in time, the first set of images capture at least a portion of an eye of the end user;

determining an initial head position of the end user relative to the HMD based on the first set of images;

rendering and displaying a first image associated with a virtual object based on the initial head position of the end user relative to the HMD, the first image is displayed on the HMD using a first resolution for the virtual object;

capturing a second set of images of the end user at a second point in time subsequent to the first point in time;

determining a second head position of the end user relative to the HMD based on the second set of images, the second head position is different from the initial head position;

detecting that a degree of head movement relative to the HMD has exceeded a threshold amount based on the initial head position and the second head position; and

rendering and displaying a second image associated with the virtual object based on the second head position of the end user relative to the HMD, the second image is displayed on the HMD using a second resolution for the virtual object less than the first resolution in response to detecting that the degree of head movement relative to the HMD has exceeded the threshold amount.

17. The one or more computer readable memory storage devices of claim 16, further comprising:

reducing a number of virtual objects displayed on the HMD in response to detecting that the degree of head movement relative to the HMD has exceeded the threshold amount.

18. The one or more computer readable memory storage devices of claim 17, further comprising:

the second resolution is half of the first resolution; and

the number of virtual objects displayed on the HMD is cut in half in response to detecting that the degree of head movement relative to the HMD has exceeded the threshold amount.

19. The one or more computer readable memory devices of claim 17, further comprising:

displaying a warning to the end user if the degree of head movement exceeds the threshold amount.

20. The one or more computer readable memory devices of claim 16, further comprising:

determining an absolute head position vector representing a head position of the end user relative to an environment in which the HMD exists; and

transmitting the absolute head position vector to a server.