Presenting virtual objects based on head movements in augmented or virtual reality systems

• US 10,553,028 B2
• Filed: 05/06/2015
• Issued: 02/04/2020
• Est. Priority Date: 03/15/2013
• Status: Active Grant

• Current Assignee: Magic Leap, Inc.
• Original Assignee: Magic Leap, Inc.
• Inventors: Schowengerdt, Brian T., Miller, Samuel A.
• Primary Examiner(s): Sun, Hai Tao
• Application Number:
  US14/704,975
• Publication Number:
  US 20150235451A1
• Time in Patent Office: 1,735 Days
• Field of Search: 345633
• CPC Class Codes: G02B 2027/014 comprising information/image processing systems
• CPC Class Codes: G02B 2027/0178 Eyeglass type eyeglass details G02C
• CPC Class Codes: G02B 2027/0187 slaved to motion of at least a part of the body of the user, e.g. head, eye
• CPC Class Codes: G02B 27/0093 with means for monitoring data relating to the user, e.g. head-tracking, eye-tracking
• CPC Class Codes: G02B 27/01 Head-up displays
• CPC Class Codes: G02B 27/017 Head mounted
• CPC Class Codes: G02B 27/0172 characterised by optical features
• CPC Class Codes: G06F 3/011 Arrangements for interaction with the human body, e.g. for user immersion in virtual reality blind teaching G09B21/00
• CPC Class Codes: G06F 3/012 Head tracking input arrangements
• CPC Class Codes: G06F 3/013 Eye tracking input arrangements G06F3/015 takes precedence
• CPC Class Codes: G06T 1/60 Memory management
• CPC Class Codes: G06T 19/00 Manipulating 3D models or images for computer graphics
• CPC Class Codes: G06T 19/006 Mixed reality object pose determination, tracking or camera calibration for mixed reality G06T7/00
• CPC Class Codes: G06T 2219/024 Multi-user, collaborative environment
• CPC Class Codes: G06T 7/70 Determining position or orientation of objects or cameras camera calibration G06T7/80
• CPC Class Codes: G09G 2320/0266 Reduction of sub-frame artefacts
• CPC Class Codes: G09G 3/007 Use of pixel shift techniques, e.g. by mechanical shift of the physical pixels or by optical shift of the perceived pixels

Abstract

One embodiment is directed to a user display device comprising a housing frame mountable on the head of the user, a lens mountable on the housing frame and a projection sub system coupled to the housing frame to determine a location of appearance of a display object in a field of view of the user based at least in part on at least one of a detection of a head movement of the user and a prediction of a head movement of the user, and to project the display object to the user based on the determined location of appearance of the display object.

Claims

1. A method of operation in an augmented reality image system, the method comprising:

detecting a head movement of an end user;

determining respective resolutions for each of at least two portions of an augmented reality image frame in response to the detected head movement of the end user;

adjusting a second slope of a second drive signal corresponding to a second lower resolution portion of the augmented reality image frame between a time of presenting a first higher resolution portion of the augmented reality image frame and presenting the second lower resolution portion of the augmented reality image frame to modify a second resolution of the second lower resolution portion of the augmented reality image frame such that it is lower than a first resolution of the first higher resolution portion of the augmented reality image frame, wherein the second slope of the second drive signal corresponding to the second lower resolution portion of the augmented reality image frame is different from a first slope of a first drive signal corresponding to the first higher resolution portion of the augmented reality image frame;

displaying a virtual object to the end user based on the first and second resolutions of the first higher resolution portion and the second lower resolution portion of the augmented reality image frame;

rendering the first higher resolution portion of the augmented reality image frame adjacent a center of the spiral scan pattern using a spiral scan pattern; and

rendering the second lower resolution portion of the augmented reality image frame using the spiral scan pattern, such that the second lower resolution portion surrounds the first higher resolution portion.

2. The method of claim 1, wherein each portion of the augmented reality image frame is at least one of a field of the augmented reality image frame, a line of the augmented reality image frame, and a pixel of the augmented reality image frame.

3. The method of claim 1, wherein modifying the second resolution of the second lower resolution portion of the augmented reality image frame further comprises adjusting an amplitude of the first drive signal.

4. The method of claim 1, further comprising:

assessing a point of attention in at least a first augmented reality image for the end user, based on at least one of a processed eye tracking data, a determined location of appearance of a virtual object in a field of view of the end user relative to an end user frame of reference, a determined location of appearance of the virtual object when newly introduced in the field of view of the end user, and a determined location of appearance of the virtual object in a new position in an augmented reality image relative to a position of the virtual object in at least one previous augmented reality image.

5. The method of claim 4, wherein the processing eye tracking data tracking data is indicative of at least one of an orientation of the end user'"'"'s eyes and an orientation of the end user'"'"'s head.

6. The method of claim 4, further comprising:

increasing a resolution in at least one subsequent augmented reality image in a portion of the at least one subsequent augmented reality image that is adjacent to the assessed point of attention.

7. The method of claim 4, further comprising:

decreasing the resolution in at least one subsequent augmented reality image in a portion of the at least one subsequent augmented reality image that is not adjacent to the assessed point of attention.