AUTOMATIC FOCUS IMPROVEMENT FOR AUGMENTED REALITY DISPLAYS

• US 20120127062A1
• Filed: 11/18/2010
• Published: 05/24/2012
• Est. Priority Date: 11/18/2010
• Status: Active Grant

• Current Assignee: Microsoft Technology Licensing LLC (Microsoft Corporation)
• Original Assignee: Microsoft Technology Licensing LLC (Microsoft Corporation)
• Inventors: Bar-Zeev, Avi, Lewis, John
• Granted Patent: US 9,304,319 B2
• Time in Patent Office: Days
• US Class Current: 345/6
• CPC Class Codes: G02B 2027/0127 comprising devices increasing the depth of field
• CPC Class Codes: G02B 2027/014 comprising information/image processing systems
• CPC Class Codes: G02B 2027/0178 Eyeglass type eyeglass details G02C
• CPC Class Codes: G02B 2027/0187 slaved to motion of at least a part of the body of the user, e.g. head, eye
• CPC Class Codes: G02B 27/017 Head mounted
• CPC Class Codes: G02B 27/0172 characterised by optical features
• CPC Class Codes: G02B 3/14 of variable focal length
• CPC Class Codes: G06F 3/011 Arrangements for interaction with the human body, e.g. for user immersion in virtual reality blind teaching G09B21/00
• CPC Class Codes: G06F 3/013 Eye tracking input arrangements G06F3/015 takes precedence
• CPC Class Codes: G06T 19/006 Mixed reality object pose determination, tracking or camera calibration for mixed reality G06T7/00

Abstract

An augmented reality system provides improved focus of real and virtual objects. A see-through display device includes a variable focus lens a user looks through. A focal region adjustment unit automatically focuses the variable focus lens in a current user focal region. A microdisplay assembly attached to the see-through display device generates a virtual object for display in the user'"'"'s current focal region by adjusting its focal region. The variable focus lens may also be adjusted to provide one or more zoom features. Visual enhancement of an object may also be provided to improve a user'"'"'s perception of an object.

Claims

1. An augmented reality system providing improved focus of objects comprising:

a see-through display device including a variable focus lens positioned to be seen through by a user;

a microdisplay assembly attached to the see-through display device for generating a virtual image for display for at least one eye;

a memory for storing software and data;

a processor for determining a current user focal region under the control of software, the processor having access to the memory and being communicatively coupled to the microdisplay assembly; and

a focal region adjustment unit for focusing the variable focus lens in the current user focal region under the control of the processor.

2. The system of claim 1, further comprising:

the processor determines a current field of view of the user based on a three dimensional model of a space in which the user is located.

3. The system of claim 2, further comprising:

the processor, under the control of software, causes the microdisplay to display one or more virtual objects at their real world focal distances in the user field of view based on the three dimensional model.

4. The system of claim 2, wherein:

the processor determines the current focal region of the user based upon eye tracking data.

5. The system of claim 1, further comprising:

the see-through display device being a head mounted display device including an orientation sensor; and

the processor determines a current field of view of the user based on a three dimensional model of a space in which the user is located and position and orientation data from the orientation sensor.

6. The system of claim 1, wherein the variable focus lens is a liquid lens.

7. The system of claim 6, wherein the focal region adjustment unit focuses the variable focus lens by adjusting the radius of curvature of the liquid lens to adjust the focal length of the lens to be in the current user focal region.

8. The system of claim 1 further comprising:

the processor determines a focal distance to a focal point in the focal region;

the processor selects an adjustment value for a variable focus lens for a focal length which approximates the focal distance; and

the focal region adjustment unit adjusts the variable focus lens based on the selected adjustment value.

9. An augmented reality system providing improved focus of objects comprising:

a see-through display device including a variable focus lens positioned to be seen through by a user;

a microdisplay assembly attached to the see-through display device for generating a virtual image for display for at least one eye;

a memory for storing software and data;

a processor for determining a focal distance under the control of software based on a zoom feature and a current user focal region, the processor having access to the memory and being communicatively coupled to the microdisplay assembly; and

a focal region adjustment unit for focusing the variable focus lens to about the determined focal distance.

10. The system of claim 9, wherein:

the processor for determining a focal distance under the control of software based on a zoom feature and a current user focal region further comprises;

the processor for receiving user input requesting the zoom feature;

the processor for determining a focal distance based on the requested zoom feature and the current focal region;

the processor for selecting an adjustment value for the variable focus lens for a focal length about the focal distance; and

the focal region adjustment unit for adjusting the variable focus lens based on the selected adjustment value.

11. The system of claim 9, wherein the variable focus lens is a liquid lens.

12. The system of claim 11, wherein:

the focal region adjustment unit focuses the variable focus lens by adjusting the radius of curvature of the liquid lens to adjust the focal length of the lens to be about the focal distance determined for the zoom feature.

13. In an augmented reality system, a method for providing improved focus of objects comprising:

determining a field of view of a user in a three dimensional model of a space including a real object;

determining a current user focal region;

automatically adjusting a variable focus lens for at least one eye to focus in the current user focal region, the variable focus lens being positioned to be seen through by the at least one eye;

displaying a virtual object at its real world focal distance in the user field of view; and

visually enhancing an image of one of the objects in the user field of view.

14. The method of claim 13, wherein:

the automatically adjusting the variable focus lens for at least one eye to focus in the current user focal region further comprises;

adjusting the radius of curvature of the variable focus lens to adjust the focal length of the lens to be in the current user focal region.

15. The method of claim 13, wherein:

displaying a virtual object at its real world focal distance in the user field of view further comprises

identifying each virtual object which is in the current user focal region based on its location in the model;

placing each virtual object identified as being in the current user focal region by changing the focal region of the microdisplay assembly; and

displaying an image including each virtual object in the current user focal region.

16. The method of claim 13, wherein:

determining the current user focal region further comprises

determining the current user focal region based upon eye tracking data.

17. The method of claim 13, wherein:

the visually enhancing an image of one of the objects in the user field of view further comprises

visually enhancing the real object in the user field of view.

18. The method of claim 16, wherein:

visually enhancing the real object in the user field of view further comprises;

identifying the real object in the user field of view which satisfies enhancement criteria;

selecting an enhancement technique based on the enhancement criteria;

selecting real object image data used by the selected enhancement technique;

generating a virtual image implementing the selected enhancement technique; and

tracking the display of the virtual image to the real object based on the three dimensional model.

19. The method of claim 18, wherein:

the real object image data is edge data and the enhancement technique is one of the group consisting of;

edge enhancement of the at least one real object;

highlighting of the at least one real object; and

color enhancement of the at least one real object.

20. The method of claim 19, wherein:

the enhancement criteria is motion criteria for the real object.