Augmenting reality for a user

• US 8,102,334 B2
• Filed: 11/15/2007
• Issued: 01/24/2012
• Est. Priority Date: 11/15/2007
• Status: Active Grant

• Current Assignee: Activision Publishing Incorporated (Microsoft Corporation)
• Original Assignee: International Business Machines Corporation
• Inventors: Brown, William A., Muirhead, Richard W., Reddington, Francis X., Wolfe, Martin A.
• Primary Examiner(s): Liang, Regina
• Application Number:
  US11/940,850
• Publication Number:
  US 20090128449A1
• Time in Patent Office: 1,531 Days
• Field of Search: 345 7- 9, 345156-158, 463 30- 34
• US Class Current: 345/8
• CPC Class Codes: G06F 3/012 Head tracking input arrangements

Abstract

Augmenting reality for a user including creating a model of a region of the user'"'"'s environment in a direction of the user'"'"'s field of vision; determining, in dependence upon the model of the region of the user'"'"'s environment, that an object located within the user'"'"'s environment is invisible to the user; determining, in dependence upon the direction of the user'"'"'s field of vision, the object'"'"'s location, and the model of the region of the user'"'"'s environment, a location to display a representation of the object on a transparent display in front of the user'"'"'s field of vision; and displaying the representation of the object at the determined location on the transparent display in front of the user'"'"'s field of vision.

Claims

1. A method of augmenting reality for a user, the method comprising:

a computer creating a model of a region of an environment of the user in a direction of a field of vision of the user;

the computer determining, in dependence upon the model of the region of the environment, that an object located within the environment is invisible to the user;

the computer determining, in dependence upon the direction of the field of vision, a location of the object, and the model of the region of the environment, a location to display a representation of the object on a transparent display in front of the field of vision; and

the computer displaying the representation of the object at the determined location on the transparent display in front of the field of vision.

2. The method of claim 1 further comprising:

the computer displaying, with the representation of the object, contextual information describing the object.

3. The method of claim 1 wherein the computer determining, in dependence upon the model of the region of the environment, that an object located within the environment is invisible to the user comprises:

the computer identifying a location of the object within the environment;

the computer identifying a location of the user within the environment; and

the computer determining, from the model of the region of the environment, that there is an obstruction located between the user and the object.

4. The method of claim 1 wherein the computer determining, in dependence upon the model of the region of the environment, that an object located within the environment is invisible to the user comprises:

the computer identifying a location of the object within the environment; and

the computer determining, from the model of the region of the environment, that the object is located in an area of low light.

5. The method of claim 1 wherein the computer determining, in dependence upon the direction of the field of vision, the location of the object, and the model of the region of the environment, a location to display a representation of the object on a transparent display in front of the field of vision comprises:

the computer identifying a grid location of the object on the model of the region of the environment; and

the computer mapping the grid location of the object on the model of the region of the environment to a grid location on the transparent display.

6. The method of claim 1 wherein the computer displaying the representation of the object at the determined location on the transparent display in front of the field of vision comprises the computer projecting the representation of the object on the transparent display.

7. A computer system for augmenting reality for a user, the apparatus comprising:

one or more processors, one or more computer-readable memories, and one or more computer-readable tangible storage devices;

program instructions, stored on at least one of the one or more storage devices for execution by at least one of the one or more processors via at least one of the one or more memories, to create a model of a region of an environment of the user in a direction of a field of vision of the user;

program instructions, stored on at least one of the one or more storage devices for execution by at least one of the one or more processors via at least one of the one or more memories, to determine, in dependence upon the model of the region of the environment, that an object located within the environment is invisible to the user;

program instructions, stored on at least one of the one or more storage devices for execution by at least one of the one or more processors via at least one of the one or more memories, to determine, in dependence upon the direction of the field of vision, a location of the object, and the model of the region of the environment, a location to display a representation of the object on a transparent display in front of the field of vision; and

program instructions, stored on at least one of the one or more storage devices for execution by at least one of the one or more processors via at least one of the one or more memories, to display the representation of the object at the determined location on the transparent display in front of the field of vision.

8. The computer system of claim 7 further comprising program instructions, stored on at least one of the one or more storage devices for execution by at least one of the one or more processors via at least one of the one or more memories, to display, with the representation of the object, contextual information describing the object.

9. The computer system of claim 7 wherein the program instructions to determine that an object located within the environment is invisible to the user:

identify a location of the object within the environment;

identify a location of the user within the environment; and

determine, from the model of the region of the environment, that there is an obstruction located between the user and the object.

10. The computer system of claim 7 wherein the program instructions to determine that an object located within the environment is invisible to the user:

identify a location of the object within the environment; and

determine, from the model of the region of the environment, that the object is located in an area of low light.

11. The computer system of claim 7 wherein the program instructions to determine, in dependence upon the direction of the field of vision, the location of the object, and the model of the region of the environment, a location to display a representation of the object on a transparent display in front of the field of vision:

identify a grid location of the object on the model of the region of the environment; and

map the grid location of the object on the model of the region of the environment to a grid location on the transparent display.

12. The computer system of claim 7 wherein the program instructions to display the representation of the object at the determined location on the transparent display in front of the field of vision project the representation of the object on the transparent display.

13. A computer program product for augmenting reality for a user, the computer program product comprising:

one or more computer-readable tangible storage devices;

program instructions, stored on at least one of the one or more storage devices, to create a model of a region of an environment of the user in a direction of a field of vision of the user;

program instructions, stored on at least one of the one or more storage devices, to determine, in dependence upon the model of the region of the environment, that an object located within the the environment is invisible to the user;

program instructions, stored on at least one of the one or more storage devices, to determine, in dependence upon the direction of the field of vision, a location of the object, and the model of the region of the environment, a location to display a representation of the object on a transparent display in front of the field of vision; and

program instructions, stored on at least one of the one or more storage devices, to display the representation of the object at the determined location on the transparent display in front of the field of vision.

14. The computer program product of claim 13 further comprising program instructions, stored on at least one of the one or more storage devices, to display, with the representation of the object, contextual information describing the object.

15. The computer program product of claim 13 wherein program instructions to determine that an object located within the environment is invisible to the user:

identify a location of the object within the environment;

identify a location of the user within the environment; and

determine, from the model of the region of the environment, that there is an obstruction located between the user and the object.

16. The computer program product of claim 13 wherein program instructions to determine that an object located within the environment is invisible to the user:

identify a location of the object within the environment; and

determine, from the model of the region of the environment, that the object is located in an area of low light.

17. The computer program product of claim 13 wherein program instructions to determine, in dependence upon the direction of the field of vision, the location of the object, and the model of the region of the environment, a location to display a representation of the object on a transparent display in front of the field of vision:

identify a grid location of the object on the model of the region of the environment; and

map the grid location of the object on the model of the region of the environment to a grid location on the transparent display.

18. The computer program product of claim 13 wherein program instructions to display the representation of the object at the determined location on the transparent display in front of the field of vision project the representation of the object on the transparent display.