PREDICTING HEAD MOVEMENT FOR RENDERING VIRTUAL OBJECTS IN AUGMENTED OR VIRTUAL REALITY SYSTEMS

• US 20150235430A1
• Filed: 05/03/2015
• Published: 08/20/2015
• Est. Priority Date: 03/15/2013
• Status: Active Grant

• Current Assignee: Magic Leap, Inc.
• Original Assignee: Magic Leap, Inc.
• Inventors: Schowengerdt, Brian T., Miller, Samuel A.
• Granted Patent: US 10,134,186 B2
• Time in Patent Office: Days
• US Class Current: 1/1
• CPC Class Codes: G02B 2027/014 comprising information/image processing systems
• CPC Class Codes: G02B 2027/0178 Eyeglass type eyeglass details G02C
• CPC Class Codes: G02B 2027/0187 slaved to motion of at least a part of the body of the user, e.g. head, eye
• CPC Class Codes: G02B 27/0093 with means for monitoring data relating to the user, e.g. head-tracking, eye-tracking
• CPC Class Codes: G02B 27/01 Head-up displays
• CPC Class Codes: G02B 27/017 Head mounted
• CPC Class Codes: G02B 27/0172 characterised by optical features
• CPC Class Codes: G06F 3/011 Arrangements for interaction with the human body, e.g. for user immersion in virtual reality blind teaching G09B21/00
• CPC Class Codes: G06F 3/012 Head tracking input arrangements
• CPC Class Codes: G06F 3/013 Eye tracking input arrangements G06F3/015 takes precedence
• CPC Class Codes: G06T 1/60 Memory management
• CPC Class Codes: G06T 19/00 Manipulating 3D models or images for computer graphics
• CPC Class Codes: G06T 19/006 Mixed reality object pose determination, tracking or camera calibration for mixed reality G06T7/00
• CPC Class Codes: G06T 2219/024 Multi-user, collaborative environment
• CPC Class Codes: G06T 7/70 Determining position or orientation of objects or cameras camera calibration G06T7/80
• CPC Class Codes: G09G 2320/0266 Reduction of sub-frame artefacts
• CPC Class Codes: G09G 3/007 Use of pixel shift techniques, e.g. by mechanical shift of the physical pixels or by optical shift of the perceived pixels

Abstract

One embodiment is directed to a user display device comprising a housing frame mountable on the head of the user, a lens mountable on the housing frame and a projection sub system coupled to the housing frame to determine a location of appearance of a display object in a field of view of the user based at least in part on at least one of a detection of a head movement of the user and a prediction of a head movement of the user, and to project the display object to the user based on the determined location of appearance of the display object.

Claims

1. A user display device, comprising:

a housing frame mountable on a head of a user;

a lens mountable on the housing frame; and

a projection subsystem coupled to the housing frame to determine a location of appearance of a display object in a field of view of the user based at least in part on at least one of a detection of a head movement of the user and a prediction of a head movement of the user, and to project the display object to the user based on the determined location of appearance of the display object.

2. The user display device of claim 1, wherein the location of appearance of the display object is moved in response to the at least one of the detection of the head movement of the user or prediction of the head movement of the user that exceeds or it predicted to exceed a nominal head movement value.

3. The user display device of claim 1, wherein the prediction of the head movement of the user is based on a prediction of a user'"'"'s shift in focus.

4. The user display device of claim 1, wherein the prediction of the head movement of the user is based on a set historical attributes of the user.

5. The user display device of claim 1, further comprising a first pair of cameras mountable on the housing frame to track a movement of the user'"'"'s eyes and estimate a depth of focus of the user'"'"'s eyes based on the tracked eye movements.

6. The user display device of claim 5, wherein the projection subsystem projects the display object based on the estimated depth of focus.

7. The user display device of claim 1, further comprising a second pair of cameras mountable on the housing frame to capture a field of view image as seen by the user'"'"'s eyes, wherein the field of view image contains at least one physical object.

8. The user display device of claim 7, wherein the projection sub system projects the display object in a manner such that the display object and the physical object captured through the second pair of cameras are inter-mixed and appear together in the same frame.

9. The user display device of claim 8, wherein the location of appearance is based at least in part on the physical object.

10. The user display device of claim 9, wherein the display object and the physical object have a predetermined relationship.

11. The user display device of claim 5, wherein the captured field of view image is used to gather information regarding movements of the head of the user, wherein the information regarding movements of the head of the user comprises a center of attention of the user, an orientation of the head of the user, a direction of the head of the user, a speed of movement of the head of the user, an acceleration of the head of the user and a distance of the head of the user in relation to a local environment of the user.

12. The user display device of claim 1, wherein the lens comprises at least one transparent surface to selectively allow a transmission light such that the user is able to view a local environment.

13. The user display device of claim 12, wherein the projection subsystem projects the display object in a manner such that the user views both the display object and the local environment as viewed through the transparent surface of the lens.

14. The user display device of claim 1, further comprising at least one intertial transducer to capture a set of intertial measurements indicative of movement of the head of the user, wherein the set of intertial measurements comprises a speed of movement of the head of the user, an acceleration of movement of the head of the user, a direction of movement of the head of the user, a position of the head of the user and an orientation of the head of the user.

15. The user display device of claim 1, further comprising at least one light source to illuminate at least one of the head of the user and a local environment of the user.

16. The user display device of claim 1, wherein the projection sub system adjusts at least one of a perceived size, an intensity and a resolution of a set of pixels associated with the display object to compensate for the at least one of the detected head movement and the predicted head movement.

17. The user display device of claim 1, wherein the display object is one of a virtual object and an augmented virtual object.