OPTIMIZED FOCAL AREA FOR AUGMENTED REALITY DISPLAYS

• US 20120154277A1
• Filed: 12/17/2010
• Published: 06/21/2012
• Est. Priority Date: 12/17/2010
• Status: Active Grant

• Current Assignee: Microsoft Technology Licensing LLC (Microsoft Corporation)
• Original Assignee: Microsoft Technology Licensing LLC (Microsoft Corporation)
• Inventors: Bar-Zeev, Avi, Lewis, John R., Klein, Georg
• Granted Patent: US 9,690,099 B2
• Time in Patent Office: Days
• US Class Current: 345/158
• CPC Class Codes: G02B 2027/014 comprising information/image processing systems
• CPC Class Codes: G02B 2027/0178 Eyeglass type eyeglass details G02C
• CPC Class Codes: G02B 2027/0187 slaved to motion of at least a part of the body of the user, e.g. head, eye
• CPC Class Codes: G02B 27/0093 with means for monitoring data relating to the user, e.g. head-tracking, eye-tracking
• CPC Class Codes: G02B 27/017 Head mounted
• CPC Class Codes: G06F 3/013 Eye tracking input arrangements G06F3/015 takes precedence

Abstract

A method and system that enhances a user'"'"'s experience when using a near eye display device, such as a see-through display device or a head mounted display device is provided. An optimized image for display relative to the a field of view of a user in a scene is created. The user'"'"'s head and eye position and movement are tracked to determine a focal region for the user. A portion of the optimized image is coupled to the user'"'"'s focal region in the current position of the eyes, a next position of the head and eyes predicted, and a portion of the optimized image coupled to the user'"'"'s focal region in the next position.

Claims

1. A method for presenting an optimized image to a user, the method comprising:

creating an optimized image based on a field of view of a user in a physical environment;

tracking eye position and movement relative to the field of view to determine a focal region for the user;

optically coupling a portion of the optimized image to the user in the focal region of the user;

predicting a next eye movement for the user; and

.

optically coupling a portion of the optimized image to the user in the focal region of the user at the next eye position.

2. The method of claim 1 a wherein the step of creating an optimized image includes determining a field of view for the user and creating an optimized image based on a user position in a scene.

3. The method of claim 1 wherein the step of presenting a portion of the optimized image includes determining a portion of the optimized image relative to the field of view and coupling the portion to a fovea of a user'"'"'s eye.

4. The method of claim 1 wherein the step of optically coupling comprises positioning the portion of the image in the focal region at the eye position on a head mounted display.

5. The method of claim 4 wherein the step of optically coupling comprises positioning the portion of the image using a mechanically controlled mirror reflecting a projection of the image.

6. The method of claim 4 wherein the step of optically coupling comprises positioning the portion of the image using an emissive display positioned in front of each eye of a user.

7. The method of claim 1 wherein the step of predicting includes:

determining at least one next position of a user eye based on data retrieved in the tracking step using at least one predictive algorithm.

8. The method of claim 1 wherein the step of optically coupling includes presenting a portion of the optimized image over a real-world object in the environment.

9. The method of claim 1 wherein the step of optically coupling includes highlighting a real world object in the environment.

10. The method of claim 1 wherein the method further includes displaying a portion of the optimized image at a position other than a current or next position to attract user attention when performing one of said optically coupling steps.

11. A head mounted display device, comprising:

a display coupling at least a portion of an optimized image to a user'"'"'s focal region;

inertial sensors that sense orientation information for the head mounted display device,

eye tracking sensors detecting user eye position; and

at least one processing unit, in communication with the display, inertial sensors and eye tracking sensors, to automatically

display an optimized portion of the optimized image at a current position of the user'"'"'s eye relative to the display so that the portion of the image is coupled to the user'"'"'s focal region,

determine a next position of the user'"'"'s eye and

display another optimized portion of the optimized image at the next position of the user'"'"'s eye relative to the display so that the portion of the image is coupled to the user'"'"'s focal region at the next position.

12. The head mounted display device of claim 11, wherein the display comprises:

an image projector;

a lightguide optical element aligned with the display to allow the user to view objects in a scene in which the user is positioned; and

wherein the display further comprises a mirror element coupled to mechanical control elements, the control elements responsive to the at least one processing unit to position the optimized portion of the image.

13. The head mounted display device of claim 11, wherein:

the at least one processing unit communicates with a hub computing device to receive a three dimensional model of a space that identifies one or more objects; and

the at least one processing unit determines a field of view of a user, determines if a first object is in the field of view, determines a position of the first object in the display, adjusting a size and orientation of the optimized image based on size and orientation of the user in the scene.

14. One or more processor readable storage devices having processor readable code embodied on said one or more processor readable storage devices, the processor readable code for programming one or more processors to perform a method comprising:

creating a three dimensional model of a user viewing a scene using a display device, the scene comprising one or more objects;

determining a field of view of the user based on the three dimensional model and determining a focal region of the user within the field of view;

detecting eye movement patterns of the user in the focal region of the user;

generating an optimized image, the optimized image visually enhances the appearance of one or more of the objects in the scene;

coupling a portion of the optimized image to the user in the focal region of the user;

predicting a next eye movement for the user; and

coupling a portion of the optimized image to the user in the focal region of the user at the next eye position.

15. The processor readable storage device of claim 14 a wherein the step of generating an optimized image includes determining a field of view for the user and creating an optimized image based on a user position in a scene.

16. The processor readable storage device of claim 15 wherein the step of presenting a portion of the optimized image includes determining a portion of the optimized image relative to the field of view and coupling the portion to a fovea of a user'"'"'s eye.

17. The processor readable storage device of claim 16 wherein the step of optically coupling comprises positioning the portion of the image using a mechanically controlled mirror reflecting a projection of the image.

18. The processor readable storage device of claim 16 wherein the step of predicting includes:

determining at least one next position of a user eye based on data retrieved in the tracking step using at least one predictive algorithm.

19. The processor readable storage device of claim 16 wherein the step of coupling includes presenting a portion of the optimized image over a real-world object in the scene.

20. The processor readable storage device of claim 19 wherein the method further includes displaying a portion of the optimized image at a position other than a current or next position when performing one of said coupling steps.