Optical systems for head-worn computers

• US 10,591,728 B2
• Filed: 03/02/2016
• Issued: 03/17/2020
• Est. Priority Date: 03/02/2016
• Status: Active Grant

• Current Assignee: Mentor Acquisition One, LLC (Magic Leap, Inc.)
• Original Assignee: Mentor Acquisition One, LLC (Magic Leap, Inc.)
• Inventors: Shams, Nima L., Border, John N., Haddick, John D.
• Primary Examiner(s): Lhymn, Sarah
• Application Number:
  US15/058,383
• Publication Number:
  US 20170256029A1
• Time in Patent Office: 1,476 Days
• Field of Search: 345619, 345632, 345633
• CPC Class Codes: G02B 2027/0114 comprising dichroic elements
• CPC Class Codes: G02B 2027/0118 comprising devices for improving the contrast of the display / brillance control visibility
• CPC Class Codes: G02B 2027/013 comprising a combiner of particular shape, e.g. curvature
• CPC Class Codes: G02B 2027/0174 holographic
• CPC Class Codes: G02B 27/0093 with means for monitoring data relating to the user, e.g. head-tracking, eye-tracking
• CPC Class Codes: G02B 27/017 Head mounted
• CPC Class Codes: G06F 1/163 Wearable computers, e.g. on a belt
• CPC Class Codes: G06F 3/011 Arrangements for interaction with the human body, e.g. for user immersion in virtual reality blind teaching G09B21/00
• CPC Class Codes: G06F 3/012 Head tracking input arrangements
• CPC Class Codes: G06F 3/013 Eye tracking input arrangements G06F3/015 takes precedence
• CPC Class Codes: G06F 3/0346 with detection of the device orientation or free movement in a 3D space, e.g. 3D mice, 6-DOF [six degrees of freedom] pointers using gyroscopes, accelerometers or tilt-sensors
• CPC Class Codes: G06T 2207/30201 Face
• CPC Class Codes: G06T 3/02 Affine transformations for image registration G06T3/147; for image mosaicing G06T3/4038
• CPC Class Codes: G06T 3/4015 Image demosaicing, e.g. colour filter arrays [CFA] or Bayer patterns
• CPC Class Codes: G06T 7/73 using feature-based methods

Abstract

Aspects of the present disclosure relate to optical systems with ergonomic presentation of content for use in head-worn computing systems. A method for controlling a head-worn computer when viewing virtual images, including image content, that encourages an ergonomic head position to reduce neck pain, includes determining an angle of the head-worn computer relative to horizontal, determining an angle of a line of sight to the center of the virtual image as presented to a user'"'"'s eye, determining a deviation between the determined angle of the line of sight and a predetermined ergonomic angle, and shifting the image content of the virtual image vertically as displayed to the user'"'"'s eye so that a portion of the image content is not viewable, wherein the amount of shifting is in reverse correspondence to the magnitude of the determined deviation.

Claims

1. A method of presenting an image, the method comprising:

determining a base display image;

capturing, via a first one or more sensors of a wearable head device, an image of a user of the wearable head device;

determining, based on the captured image, a line of sight from the user to a center of a display of the wearable head device;

determining a viewing angle between the line of sight and a horizontal plane;

determining, based on a difference between the determined viewing angle and a desired viewing angle, that the base image should be adjusted;

in accordance with the determination that the base display image should be adjusted;

generating an edited display image, wherein generating the edited display image comprises;

cropping the base display image to reduce a vertical display size, and

shifting the base display image with respect to the horizontal plane, the shifting having a magnitude based on the difference between the determined viewing angle and the desired viewing angle and further having a vertical direction based on the difference between the determined viewing angle and the desired viewing angle; and

presenting the edited display image to the user via the display.

2. The method of claim 1, wherein the desired angle is 20 degrees from the horizontal plane.

3. The method of claim 1, wherein the position of the center of the display relative to the user is unaffected by lateral motion of the user.

4. The method of claim 1, wherein the line of sight is determined based on an image plane of the base display image.

5. The method of claim 1, wherein shifting the base display image comprises aligning the top of the base display image with the horizontal plane.

6. The method of claim 1, wherein presenting the edited display image to the user via the display comprises presenting the edited display image in a plane orthogonal to the line of sight angle.

7. The method of claim 1, wherein the captured image comprises an image of an eye of the user and the line of sight from the user to the center of the display comprises a line of sight from the eye to the center of the display.

8. The method of claim 1, further comprising determining an orientation of the wearable head device relative to the user, wherein the line of sight from the user to the center of the display is determined based on the orientation.

9. The method of claim 1, further comprising:

identifying, based on a second one or more sensors of the wearable head device, an activity of the user,

wherein the determination that the base display image should be adjusted is made based in part on the identified activity.

10. The method of claim 9, wherein the determination that the base display image should be adjusted comprises a determination that the activity comprises viewing video content.

11. The method of claim 9, wherein the determination that the base display image should be adjusted comprises a determination that the activity does not comprise traveling in a moving vehicle.

12. The method of claim 9, wherein the determination that the base display image should be adjusted comprises a determination that the activity does not comprise walking or running.

13. The method of claim 9, wherein the determination that the base display image should be adjusted comprises a determination that the activity does not comprise laying down.

14. The method of claim 9, wherein the second one or more sensors include one or more of an IMU, a GPS unit, an altimeter, and a magnetometer.

15. A wearable device comprising:

a display;

a first one or more sensors; and

one or more processors configured to perform;

determining a base display image;

capturing, via the first one or more sensors, an image of a user of the wearable device;

determining, based on the captured image, a line of sight from the user to a center of the display;

determining a viewing angle between the line of sight and a horizontal plane;

determining, based on a difference between the determined viewing angle and a desired viewing angle, that the base image should be adjusted;

in accordance with the determination that the base display image should be adjusted;

generating an edited display image, wherein generating the edited display image comprises;

cropping the base display image to reduce a vertical display size, and

shifting the base display image with respect to the horizontal plane, the shifting having a magnitude based on the difference between the determined viewing angle and the desired viewing angle and further having a vertical direction based on the difference between the determined viewing angle and the desired viewing angle; and

presenting the edited display image to the user via the display.

16. The wearable device of claim 15, wherein the captured image comprises an image of an eye of the user and the line of sight from the user to the center of the display comprises a line of sight from the eye to the center of the display.

17. The wearable device of claim 15, further comprising a second one or more sensors, wherein:

the one or more processors are further configured to perform identifying, based on the second one or more sensors, an activity of the user, and

the determination that the base display image should be adjusted is made based on the identified activity.

18. A non-transitory computer readable storage medium storing instructions that, when executed, cause one or more processors to perform a method comprising:

determining a base display image;

capturing, via a first one or more sensors of a wearable head device, an image of a user of the wearable head device;

determining, based on the captured image, a line of sight from the user to a center of a display of the wearable head device;

determining a viewing angle between the line of sight and a horizontal plane;

determining, based on a difference between the determined viewing angle and a desired viewing angle, that the base image should be adjusted;

in accordance with the determination that the base display image should be adjusted;

generating an edited display image, wherein generating the edited display image comprises;

cropping the base display image to reduce a vertical display size, and

shifting the base display image with respect to the horizontal plane, the shifting having a magnitude based on the difference between the determined viewing angle and the desired viewing angle and further having a vertical direction based on the difference between the determined viewing angle and the desired viewing angle; and

presenting the edited display image to the user via the display.

19. The non-transitory computer readable storage medium of claim 18, wherein the captured image comprises an image of an eye of the user and the line of sight from the user to the center of the display comprises a line of sight from the eye to the center of the display.

20. The non-transitory computer readable storage medium of claim 18, wherein the method further comprises identifying, based on a second one or more sensors of the wearable head device, an activity of the user, and wherein the determination that the base display image should be adjusted is made based on the identified activity.