Using historical attributes of a user for virtual or augmented reality rendering

• US 9,429,752 B2
• Filed: 05/03/2015
• Issued: 08/30/2016
• Est. Priority Date: 03/15/2013
• Status: Active Grant

• Current Assignee: Magic Leap, Inc.
• Original Assignee: Magic Leap, Inc.
• Inventors: Schowengerdt, Brian T., Miller, Samuel A.
• Primary Examiner(s): Faulk, Devona
• Assistant Examiner(s): Sun, Hai Tao
• Application Number:
  US14/702,734
• Publication Number:
  US 20150234184A1
• Time in Patent Office: 485 Days
• Field of Search: 345/633
• US Class Current: 1/1
• CPC Class Codes: G02B 2027/014 comprising information/image processing systems
• CPC Class Codes: G02B 2027/0178 Eyeglass type eyeglass details G02C
• CPC Class Codes: G02B 2027/0187 slaved to motion of at least a part of the body of the user, e.g. head, eye
• CPC Class Codes: G02B 27/0093 with means for monitoring data relating to the user, e.g. head-tracking, eye-tracking
• CPC Class Codes: G02B 27/01 Head-up displays
• CPC Class Codes: G02B 27/017 Head mounted
• CPC Class Codes: G02B 27/0172 characterised by optical features
• CPC Class Codes: G06F 3/011 Arrangements for interaction with the human body, e.g. for user immersion in virtual reality blind teaching G09B21/00
• CPC Class Codes: G06F 3/012 Head tracking input arrangements
• CPC Class Codes: G06F 3/013 Eye tracking input arrangements G06F3/015 takes precedence
• CPC Class Codes: G06T 1/60 Memory management
• CPC Class Codes: G06T 19/00 Manipulating 3D models or images for computer graphics
• CPC Class Codes: G06T 19/006 Mixed reality object pose determination, tracking or camera calibration for mixed reality G06T7/00
• CPC Class Codes: G06T 2219/024 Multi-user, collaborative environment
• CPC Class Codes: G06T 7/70 Determining position or orientation of objects or cameras camera calibration G06T7/80
• CPC Class Codes: G09G 2320/0266 Reduction of sub-frame artefacts
• CPC Class Codes: G09G 3/007 Use of pixel shift techniques, e.g. by mechanical shift of the physical pixels or by optical shift of the perceived pixels

Abstract

One embodiment is directed to a user display device comprising a housing frame mountable on the head of the user, a lens mountable on the housing frame and a projection sub system coupled to the housing frame to determine a location of appearance of a display object in a field of view of the user based at least in part on at least one of a detection of a head movement of the user and a prediction of a head movement of the user, and to project the display object to the user based on the determined location of appearance of the display object.

Claims

1. A method of operation in an augmented reality system, the method comprising:

receiving information indicative of an identity of an end user;

retrieving at least one user specific historical attribute for the end user based at least in part on the received information indicative of the identity of the end user;

displaying frames to the end user based at least in part on the retrieved at least one user specific historical attribute for the end user;

predicting an end point of a head movement of the end user based on a location of a virtual object in a field of view of the end user;

displaying frames to the end user based at least in part on the retrieved at least one user specific historical attribute for the end user includes rendering at least one subsequent frame to at least one image buffer, the at least one subsequent frame shifted toward the predicted end point of the head movement;

rendering a plurality of subsequent frames that shift toward the predicted end point of the head movement in at least partial accommodation of at least one head movement attribute for the end user, the at least one head movement attribute indicative of at least one previous head movement of the end user; and

predicting an occurrence of a head movement of the end user based at least in part on a location of appearance of the virtual object in the field of view of the end user.

2. The method of claim 1 wherein the received information is image information indicative of an image of at least a portion of an eye of the end user.

3. The method of claim 2 wherein providing frames to the end user includes providing frames via at least one optical fiber and receiving image information indicative of an image of at least a portion of an eye of the end user includes receiving the image information via the at least an optical fiber which also provides the frames to the end user.

4. The method of claim 1 wherein the retrieved at least one user specific historical attribute for the end user is at least one attribute that provides an indication of the at least one head movement attribute for the end user.

5. The method of claim 1 wherein the retrieved at least one user specific historical attribute for the end user is at least one attribute that provides an indication of at least one previous head movement speed for at least one previous head movement for the end user.

6. The method of claim 1 wherein the retrieved at least one user specific historical attribute for the end user is at least one attribute that provides an indication of variation in a head movement speed across at least part of a range of at least one previous head movement by the end user.

7. The method of claim 1 wherein the retrieved at least one user specific historical attribute for the end user is at least one attribute that provides an indication of at least one previous head movement acceleration for at least one previous head movement by the end user.

8. The method of claim 1 wherein the retrieved at least one user specific historical attribute for the end user is at least one attribute that provides an indication of a relationship between at least one previous head movement and at least one previous eye movement by the end user.

9. The method of claim 1 wherein the retrieved at least one user specific historical attribute for the end user is at least one attribute that provides an indication of a ratio between at least one previous head movement and at least one previous eye movement by the end user.

10. The method of claim 1, wherein the at least one head movement attribute indicative of at least one previous head movement of the end user is a historical head movement speed for the end user.

11. The method of claim 1, wherein the at least one head movement attribute indicative of at least one previous head movement of the end user is historical head movement acceleration for the end user.

12. The method of claim 1, wherein the at least one head movement attribute indicative of at least one previous head movement of the end user is a historical ratio between head movement and eye movement for the end user.

13. The method of claim 1, further comprising determining the location of appearance of the virtual object when the virtual object is newly introduced in the field of view of the end user temporally relative to previous frames presented to the end user.

14. The method of claim 1, further comprising determining the location of appearance of the virtual object in a new position as presented to the end user relative to a position of the virtual object as previously presented to the end user.