Augmenting Reality For A User

• US 20090128449A1
• Filed: 11/15/2007
• Published: 05/21/2009
• Est. Priority Date: 11/15/2007
• Status: Active Grant

• Current Assignee: Activision Publishing Incorporated (Microsoft Corporation)
• Original Assignee: International Business Machines Corporation
• Inventors: Wolfe, Martin A., Muirhead, Richard W., Reddington, Francis X., Brown, William A.
• Granted Patent: US 8,102,334 B2
• Time in Patent Office: Days
• US Class Current: 345/8
• CPC Class Codes: G06F 3/012 Head tracking input arrangements

Abstract

Augmenting reality for a user including creating a model of a region of the user'"'"'s environment in a direction of the user'"'"'s field of vision; determining, in dependence upon the model of the region of the user'"'"'s environment, that an object located within the user'"'"'s environment is invisible to the user; determining, in dependence upon the direction of the user'"'"'s field of vision, the object'"'"'s location, and the model of the region of the user'"'"'s environment, a location to display a representation of the object on a transparent display in front of the user'"'"'s field of vision; and displaying the representation of the object at the determined location on the transparent display in front of the user'"'"'s field of vision.

Claims

1. A method of augmenting reality for a user, the method comprising:

creating a model of a region of the user'"'"'s environment in a direction of the user'"'"'s field of vision;

determining, in dependence upon the model of the region of the user'"'"'s environment, that an object located within the user'"'"'s environment is invisible to the user;

determining, in dependence upon the direction of the user'"'"'s field of vision, the object'"'"'s location, and the model of the region of the user'"'"'s environment, a location to display a representation of the object on a transparent display in front of the user'"'"'s field of vision; and

displaying the representation of the object at the determined location on the transparent display in front of the user'"'"'s field of vision.

2. The method of claim 1 further comprising:

displaying, with the representation of the object, contextual information describing the object.

3. The method of claim 1 wherein determining, in dependence upon the model of the region of the user'"'"'s environment, whether an object located within the user'"'"'s environment is invisible to the user;

identifying the object'"'"'s location within the user'"'"'s environment;

identifying the user'"'"'s location within the user'"'"'s environment; and

determining, from the model of the region of the user'"'"'s environment, that there is an obstruction located between the user and the object.

4. The method of claim 1 wherein determining, in dependence upon the model of the region of the user'"'"'s environment, that an object located within the user'"'"'s environment is invisible to the user;

identifying the object'"'"'s location within the user'"'"'s environment; and

determining, from the model of user'"'"'s environment, whether the object is located in an area of low light.

5. The method of claim 1 wherein determining, in dependence upon the direction of the user'"'"'s field of vision, the location of the object, and the model of the region of the user'"'"'s environment, a location to display a representation of the object on a transparent display in front of the user'"'"'s field of vision further comprises:

identifying a grid location of the object on the model of the region of the user'"'"'s environment; and

mapping the grid location of the object on the model of user'"'"'s environment to a grid location on the transparent display.

6. The method of claim 1 wherein displaying the representation of the object at the determined location on the transparent display in front of the user'"'"'s field of vision further comprises projecting the representation of the object on the transparent display.

7. An apparatus for augmenting reality for a user, the apparatus comprising a computer processor, a computer memory operatively coupled to the computer processor, the computer memory having disposed within it computer program instructions capable of:

creating a model of a region of the user'"'"'s environment in a direction of the user'"'"'s field of vision;

determining, in dependence upon the model of the region of the user'"'"'s environment, that an object located within the user'"'"'s environment is invisible to the user;

determining, in dependence upon the direction of the user'"'"'s field of vision, the object'"'"'s location, and the model of the region of the user'"'"'s environment, a location to display a representation of the object on a transparent display in front of the user'"'"'s field of vision; and

displaying the representation of the object at the determined location on the transparent display in front of the user'"'"'s field of vision.

8. The apparatus of claim 7 further comprising computer program instructions capable of:

displaying, with the representation of the object, contextual information describing the object.

9. The apparatus of claim 7 wherein determining, in dependence upon the model of the region of the user'"'"'s environment, whether an object located within the user'"'"'s environment is invisible to the user;

identifying the object'"'"'s location within the user'"'"'s environment;

identifying the user'"'"'s location within the user'"'"'s environment; and

determining, from the model of the region of the user'"'"'s environment, that there is an obstruction located between the user and the object.

10. The apparatus of claim 7 wherein determining, in dependence upon the model of the region of the user'"'"'s environment, that an object located within the user'"'"'s environment is invisible to the user;

identifying the object'"'"'s location within the user'"'"'s environment; and

determining, from the model of user'"'"'s environment, whether the object is located in an area of low light.

11. The apparatus of claim 7 wherein determining, in dependence upon the direction of the user'"'"'s field of vision, the location of the object, and the model of the region of the user'"'"'s environment, a location to display a representation of the object on a transparent display in front of the user'"'"'s field of vision further comprises:

identifying a grid location of the object on the model of the region of the user'"'"'s environment; and

mapping the grid location of the object on the model of user'"'"'s environment to a grid location on the transparent display.

12. The apparatus of claim 7 wherein displaying the representation of the object at the determined location on the transparent display in front of the user'"'"'s field of vision further comprises projecting the representation of the object on the transparent display.

13. A computer program product for augmenting reality for a user, the computer program product disposed in a computer readable, signal bearing medium, the computer program product comprising computer program instructions capable of:

creating a model of a region of the user'"'"'s environment in a direction of the user'"'"'s field of vision;

determining, in dependence upon the model of the region of the user'"'"'s environment, that an object located within the user'"'"'s environment is invisible to the user;

determining, in dependence upon the direction of the user'"'"'s field of vision, the object'"'"'s location, and the model of the region of the user'"'"'s environment, a location to display a representation of the object on a transparent display in front of the user'"'"'s field of vision; and

displaying the representation of the object at the determined location on the transparent display in front of the user'"'"'s field of vision.

14. The computer program product of claim 13 further comprising computer program instructions capable of:

displaying, with the representation of the object, contextual information describing the object.

15. The computer program product of claim 13 wherein determining, in dependence upon the model of the region of the user'"'"'s environment, whether an object located within the user'"'"'s environment is invisible to the user;

identifying the object'"'"'s location within the user'"'"'s environment;

identifying the user'"'"'s location within the user'"'"'s environment; and

determining, from the model of the region of the user'"'"'s environment, that there is an obstruction located between the user and the object.

16. The computer program product of claim 13 wherein determining, in dependence upon the model of the region of the user'"'"'s environment, that an object located within the user'"'"'s environment is invisible to the user;

identifying the object'"'"'s location within the user'"'"'s environment; and

determining, from the model of user'"'"'s environment, whether the object is located in an area of low light.

17. The computer program product of claim 13 wherein determining, in dependence upon the direction of the user'"'"'s field of vision, the location of the object, and the model of the region of the user'"'"'s environment, a location to display a representation of the object on a transparent display in front of the user'"'"'s field of vision further comprises:

identifying a grid location of the object on the model of the region of the user'"'"'s environment; and

mapping the grid location of the object on the model of user'"'"'s environment to a grid location on the transparent display.

18. The computer program product of claim 13 wherein displaying the representation of the object at the determined location on the transparent display in front of the user'"'"'s field of vision further comprises projecting the representation of the object on the transparent display.

19. The computer program product of claim 13 wherein the signal bearing medium comprises a recordable medium.

20. The computer program product of claim 13 wherein the signal bearing medium comprises a transmission medium.