Adjusting pixels to compensate for spacing in augmented or virtual reality systems

• US 10,453,258 B2
• Filed: 05/03/2015
• Issued: 10/22/2019
• Est. Priority Date: 03/15/2013
• Status: Active Grant

• Current Assignee: Magic Leap, Inc.
• Original Assignee: Magic Leap, Inc.
• Inventors: Schowengerdt, Brian T., Miller, Samuel A.
• Primary Examiner(s): Sun, Hai Tao
• Application Number:
  US14/702,736
• Publication Number:
  US 20150235583A1
• Time in Patent Office: 1,633 Days
• Field of Search: 345633
• CPC Class Codes: G02B 2027/014 comprising information/image processing systems
• CPC Class Codes: G02B 2027/0178 Eyeglass type eyeglass details G02C
• CPC Class Codes: G02B 2027/0187 slaved to motion of at least a part of the body of the user, e.g. head, eye
• CPC Class Codes: G02B 27/0093 with means for monitoring data relating to the user, e.g. head-tracking, eye-tracking
• CPC Class Codes: G02B 27/01 Head-up displays
• CPC Class Codes: G02B 27/017 Head mounted
• CPC Class Codes: G02B 27/0172 characterised by optical features
• CPC Class Codes: G06F 3/011 Arrangements for interaction with the human body, e.g. for user immersion in virtual reality blind teaching G09B21/00
• CPC Class Codes: G06F 3/012 Head tracking input arrangements
• CPC Class Codes: G06F 3/013 Eye tracking input arrangements G06F3/015 takes precedence
• CPC Class Codes: G06T 1/60 Memory management
• CPC Class Codes: G06T 19/00 Manipulating 3D models or images for computer graphics
• CPC Class Codes: G06T 19/006 Mixed reality object pose determination, tracking or camera calibration for mixed reality G06T7/00
• CPC Class Codes: G06T 2219/024 Multi-user, collaborative environment
• CPC Class Codes: G06T 7/70 Determining position or orientation of objects or cameras camera calibration G06T7/80
• CPC Class Codes: G09G 2320/0266 Reduction of sub-frame artefacts
• CPC Class Codes: G09G 3/007 Use of pixel shift techniques, e.g. by mechanical shift of the physical pixels or by optical shift of the perceived pixels

Abstract

One embodiment is directed to a user display device comprising a housing frame mountable on the head of the user, a lens mountable on the housing frame and a projection sub system coupled to the housing frame to determine a location of appearance of a display object in a field of view of the user based at least in part on at least one of a detection of a head movement of the user and a prediction of a head movement of the user, and to project the display object to the user based on the determined location of appearance of the display object.

Claims

1. A method of operation in a virtual image presentation system, the method comprising:

detecting, at one or more sensors of a head-mount virtual or augmented image presentation system, an indication that indicates a first resolution in a first portion of a frame in a sequence of frames will differ from a second resolution in a second portion of the frame, wherein

the first resolution corresponds to a first spacing between first adjacent pixels in a first set of pixels in the first portion, and the second resolution corresponds to a second spacing between second adjacent pixels in the second set of pixels in the second portion;

selecting the first set of pixels of the frame for which the first spacing is to be modified based in part or in whole upon a directional characteristic pertaining to the indication, wherein

the first set of pixels of the frame is selected based in part or in whole on a direction of head movement of a user detected by one or more transducers, and

a direction of the first set of pixels is the same as the direction of the head movement;

adjusting, based at least in part on the indication, a resolution represented by the first set of pixels in the first portion to generate the first resolution represented by an adjusted first set of pixels in the first portion at least by modifying, at one or more modulation subsystems in the virtual image presentation system, respective radial characteristics of a drive signal that drives pixels in the first set of pixels into respective modified radial characteristics of a modified drive signal for a subsequent frame after the frame in the sequence of frames, without affecting a number of coils or loops of the scan pattern; and

presenting the subsequent frame including one or more virtual objects to a user using at least the scan pattern at least by starting a presentation of the frame, interrupting the presentation before completion of the presentation, increasing an intensity of the first set of pixels for the subsequent frame in response to the head movement, and replacing the first set of pixels that represents the first portion with the adjusted first set of pixels to at least render the first portion with the first resolution and the second portion in the frame with the second resolution.

2. The method of claim 1, further comprising:

adjusting a set of pixel characteristics that includes at least one of a perceived size or a perceived intensity;

adjusting one or more pixel characteristics that are perceptible to the user; and

detecting a head movement exceeding a nominal head movement value.

3. The method of claim 1, further comprising:

selecting the first set of pixels of the frame based on the direction of a detected head movement, wherein the direction of the first set of pixels is the same as the direction of the detected head movement; and

increasing a size of the first set of pixels of the subsequent frame.

4. The method of claim 1, further comprising adjusting a variable focus element of the first set of pixels or adjusting a variable size source of the first set of pixels.

5. The method of claim 1, the angular characteristics of the scan pattern comprising a number of coils or loops in the scan pattern.

6. The method of claim 1, further comprising adjusting a jitter of the first set of pixels.

7. The method of claim 1, further comprising:

selecting the first set of pixels based in part or in whole on the direction of head movement of the user detected by the one or more transducers, wherein a first direction of the first set of pixels is opposite to the direction of the head movement; and

decreasing a size of the first set of pixels of the subsequent frame in response to the head movement.

8. The method of claim 1, further comprising:

selecting the first set of pixels based in part or in whole on the direction of the head movement of the user detected by the one or more transducers, wherein a first direction of the first set of pixels is opposite to the direction of the head movement; and

decreasing an intensity of the first set of pixels of the at last one subsequent frame in response to the head movement.

9. The method of claim 1, wherein the indication is based in part or in whole on detecting that a head movement attribute of the user has exceeded a nominal value of the head movement attribute.

10. The method of claim 9, wherein the head movement attribute includes at least one of a speed of the head movement or an acceleration of the head movement, and the indication is based in part or in whole on a signal received through an inertial sensor.

11. The method of claim 1, wherein the indication is based in part or in whole on a signal received through an imager.

12. The method of claim 1, wherein the at least one subsequent frame is provided based in part or in whole on at least one of a raster scan type frame, a spiral scan type frame, or a Lissajous scan type frame.

13. A virtual or augmented image presentation system for presenting virtual contents, comprising:

one or more sensors configured to detect an indication that indicates a first resolution corresponding to a first spacing between first adjacent pixels in a first set of pixels in a first portion of a frame in a sequence of frames will differ from a second resolution corresponding to a different spacing between second adjacent pixels in a second set of pixels in a second portion of the frame, wherein

the first set of pixels of the frame is selected based in part or in whole on a direction of head movement of a user detected by one or more transducers, and

a direction of the first set of pixels is the same as the direction of the head movement;

the virtual or augmented image presentation system configured to select the first set of pixels of the frame based in part or in whole upon a directional characteristic pertaining to the indication;

one or more modulation subsystems comprising a projection subsystem and at least one processor in a virtual or augmented image presentation system operatively coupled to the one or more sensors and configured to adjust, based at least in part on the indication, a resolution represented by the first set of pixels in the first portion to generate the first resolution represented by an adjusted first set of pixels in the first portion at least by modifying respective radial characteristics of a drive signal that drives pixels in the first set of pixels into respective modified radial characteristics of a modified drive signal for a subsequent frame after the frame in the sequence of frames, without affecting a number of coils or loops of the scan pattern; and

a head-mount display including one or more projectors operatively coupled to the one or more modulation subsystems and configured to present the subsequent frame including one or more virtual objects to a user at least by starting a presentation of the frame, interrupting the presentation before completion of the presentation, increasing an intensity of the first set of pixels for the subsequent frame in response to the head movement, and replacing the first set of pixels that represents the first portion with the adjusted first set of pixels to at least render the first portion with the first resolution and the second portion in the frame with the second resolution.

14. The virtual or augmented image presentation system of claim 13, wherein the one or more modulation subsystems are further configured to adjust one or more pixel characteristics that are perceptible to the user.

15. The virtual or augmented image presentation system of claim 13, further comprising modifying an electric voltage or an electric current of the drive signal.

16. The virtual or augmented image presentation system of claim 13, further comprising modifying an amplitude of the drive signal.

17. The virtual or augmented image presentation system of claim 13, further comprising the one or more transducers configured to detect the head movement of the user, and the one or more modulation subsystems are further configured to adjust a set of pixel characteristics that include at least one of a perceived size or a perceived intensity.

18. The virtual or augmented image presentation system of claim 13, wherein the virtual or augmented image presentation system is further configured to select the first set of pixels of the frame based on the direction of the head movement, wherein a direction of the first set of pixels is the same as the direction of the head movement and to increase a size of the first set of pixels of the subsequent frame.

19. The virtual or augmented image presentation system of claim 13, wherein the virtual or augmented image presentation system is further configured to adjust a variable focus element for the first set of pixels.

20. The virtual or augmented image presentation system of claim 13, wherein the virtual or augmented image presentation system is further configured to adjust a variable size source for the first set of pixels.

21. The virtual or augmented image presentation system of claim 13, wherein the virtual or augmented image presentation system is further configured to adjust a jitter of the first set of pixels.

22. The virtual or augmented image presentation system of claim 13, further comprising the one or more transducers operatively coupled to a microprocessor that is configured to selecting the first set of pixels based in part or in whole on the direction of the head movement detected by the one or more transducers, wherein a direction of the first set of pixels is opposite to the direction of the head movement, and to decrease a size of the first set of pixels of the subsequent frame in response to the head movement detected by the one or more transducers.