Dynamic integration of computer generated and real world images

• US 20020044152A1
• Filed: 06/11/2001
• Published: 04/18/2002
• Est. Priority Date: 10/16/2000
• Status: Abandoned Application

• Current Assignee: Tangis Corporation
• Original Assignee: Tangis Corporation
• Inventors: Robarts, James O., Newell, Dan, Abbott, Kenneth H. III
• Application Number:
  US09/879,827
• Publication Number:
  US 20020044152A1
• Time in Patent Office: Days
• US Class Current: 345/629
• CPC Class Codes: G02B 2027/0112 comprising device for genereting colour display
• CPC Class Codes: G02B 2027/0118 comprising devices for improving the contrast of the display / brillance control visibility
• CPC Class Codes: G02B 2027/014 comprising information/image processing systems
• CPC Class Codes: G02B 2027/0187 slaved to motion of at least a part of the body of the user, e.g. head, eye
• CPC Class Codes: G02B 27/017 Head mounted
• CPC Class Codes: G06F 3/011 Arrangements for interaction with the human body, e.g. for user immersion in virtual reality blind teaching G09B21/00
• CPC Class Codes: G06F 3/012 Head tracking input arrangements
• CPC Class Codes: G06F 3/013 Eye tracking input arrangements G06F3/015 takes precedence
• CPC Class Codes: G06T 11/00 2D [Two Dimensional] image generation
• CPC Class Codes: G06T 19/006 Mixed reality object pose determination, tracking or camera calibration for mixed reality G06T7/00

Abstract

A system integrates virtual information with real world images presented on a display, such as a head-mounted display of a wearable computer. The system modifies how the virtual information is presented to alter whether the virtual information is more or less visible relative to the real world images. The modification may be made dynamically, such as in response to a change in the user'"'"'s context, or user'"'"'s eye focus on the display, or a user command. The virtual information may be modified in a number of ways, such as adjusting the transparency of the information, modifying the color of the virtual information, enclosing the information in borders, and changing the location of the virtual information on the display. Through these techniques, the system provides the information to the user in a way that minimizes distraction of the user'"'"'s view of the real world images.

Claims

1. A method comprising:

presenting computer-generated information on a display that permits viewing of a real world context; and

assigning a degree of transparency to the information to enable display of the information to a user without impeding the user'"'"'s view of the real world context.

2. A method as recited in claim 1, further comprising dynamically adjusting the degree of transparency of the information.

3. A method as recited in claim 1, further comprising:

receiving data pertaining to the user'"'"'s context; and

dynamically adjusting the degree of transparency upon changes in the user'"'"'s context.

4. A method as recited in claim 1, further comprising:

receiving data pertaining to the user'"'"'s eye focus on the display; and

dynamically adjusting the degree of transparency due to change in the user'"'"'s eye focus.

5. A method as recited in claim 1, further comprising:

selecting an initial location on the display to present the information; and

subsequently moving the information from the initial location to a second location.

6. A method as recited in claim 1, farther comprising presenting a border around the information.

7. A method as recited in claim 1, further comprising presenting the information within a marquee.

8. A method as recited in claim 1, further comprising presenting the information as a faintly visible graphic overlaid on the real world context.

9. A method as recited in claim 1, further comprising modifying a color of the information to alternately blend or distinguish the information from the real world context.

10. A method as recited in claim 1, wherein the information is presented against a background, and further comprising adjusting transparency of the background.

22. A method as recited in claim 21, further comprising maintaining the mapping object relative to geographic coordinates so that the mapping object appears to track a particular real image direction relative to a particular real image even though the display is moved relative to the particular real image.

24. A method as recited in claim 23, wherein the border comprises a geometrical element that encloses the virtual object.

25. A method as recited in claim 23, wherein the border comprises a marquee.

26. A method as recited in claim 23, further comprising:

detecting one or more edges of the virtual object; and

dynamically generating the border along the edges.

27. A method as recited in claim 23, further comprising:

displaying the virtual object with a first degree of transparency; and

displaying the border with a second degree of transparency that is different from the first degree of transparency.

28. A method as recited in claim 23, further comprising:

fading out the virtual object at a first rate;

fading out the border at a second rate so that the border is visible on the display after the virtual object becomes too faint to view.

30. A method as recited in claim 29, wherein the information is at least partially transparent.

31. A method as recited in claim 29, wherein said modifying is performed in response to change in user context.

32. A method as recited in claim 29, wherein said modifying is performed in response to change in user eye focus on the display.

33. A method as recited in claim 29, wherein said modifying is performed in response to change of importance attributed to the information.

34. A method as recited in claim 29, wherein said modifying is performed in response to a user command.

35. A method as recited in claim 29, further comprising presenting a border around the information.

36. A method as recited in claim 29, further comprising presenting the information as a faintly visible graphic overlaid on the real world images.

38. A method as recited in claim 37, wherein said modifying is performed in response to change in user attention between the information and the real world images.

39. A method as recited in claim 37, wherein said modifying is performed in response to change in user context.

40. A method as recited in claim 37, wherein said modifying is performed in response to change of importance attributed to the information.

41. A method as recited in claim 37, wherein said modifying is performed in response to a user command.

42. A method as recited in claim 37, wherein said modifying comprises adjusting transparency of the information.

43. A method as recited in claim 37, wherein said modifying comprises moving the information to another location on the display.

11. A method comprising:

presenting information on a screen that permits viewing real images, the information being presented in a first degree of transparency; and

modifying presentation of the information to a second degree of transparency.

12. A method as recited in claim 11, wherein the first degree of transparency is more transparent than the second degree of transparency.

13. A method as recited in claim 11, wherein the transparency ranges from fully transparent to fully opaque.

14. A method as recited in claim 11, wherein said modifying is performed in response to change of importance attributed to the information.

15. A method as recited in claim 11, wherein said modifying is performed in response to a user command.

16. A method as recited in claim 11, wherein said modifying is performed in response to a change in user context.

18. A method as recited in claim 17, wherein the faintly visible virtual object is transparent.

20. A method as recited in claim 19, wherein the information is presented in an at least partially transparent manner.

21. A method as recited in claim 19, wherein the user'"'"'s context pertains to geographical location and the information comprises at least one mapping object that provides geographical guidance to the user:

the monitoring comprising detecting a direction that the user is facing; and

presenting the mapping object when the user is facing a first direction and not presenting the mapping object when the user is facing in a second direction.

17. A method for operating a display that permits a view of real images, comprising:

generating a notification event; and

presenting, on the display, a faintly visible virtual object atop the real images to notify a user of the notification event.

19. A method for operating a display that permits a view of real images, comprising:

monitoring a user'"'"'s context; and

alternately presenting information on the display together with the real images when the user is in a first context and not presenting the information on the display when the user is in a second context.

23. A method comprising:

presenting a virtual object on a display together with a view of real world surroundings; and

graphically depicting the virtual object within a border to visually distinguish the virtual object from the view of the real world surroundings.

29. A method comprising:

presenting information on a display that permits a view of real world images; and

modifying color of the information to alternately blend or distinguish the information from the real world images.

37. A method for operating a display that permits a view of real world images, comprising:

presenting information on the display with a first level of prominence; and

modifying the prominence from the first level to a second level.

44. A method comprising:

presenting a virtual object on a screen together with a view of a real world environment;

positioning the virtual object in a first location to entice a user to focus on the virtual object;

monitoring the user'"'"'s focus; and

migrating the virtual object to a second location less noticeable than the first location when the user shifts focus from the virtual object to the real world environment.

46. A method as recited in claim 45, wherein the virtual object is transparent and the modifying comprise changing a degree of transparency.

47. A method as recited in claim 45, wherein the modifying comprises altering a color of the virtual object.

48. A method as recited in claim 45, wherein the modifying comprises changing a location of the virtual object relative to the real world images.

50. A computer as recited in claim 49, wherein the software module adjusts transparency within a range from fully transparent to fully opaque.

51. A computer as recited in claim 49, further comprising:

context sensors to detect a user'"'"'s context; and

the software module being configured to adjust transparency of the information presented by the user interface in response to changes in the user'"'"'s context.

52. A computer as recited in claim 49, further comprising:

a sensor to detect a user'"'"'s eye focus; and

the software module being configured to adjust transparency of the information presented by the user interface in response to changes in the user'"'"'s eye focus.

53. A computer as recited in claim 49, wherein the software module is configured to adjust transparency of the information presented by the user interface in response to a user command.

54. A computer as recited in claim 49, wherein the software module moves the information on the display to make the information alternately more or less noticeable.

55. A computer as recited in claim 49, wherein the user interface presents a border around the information.

56. A computer as recited in claim 49, wherein the user interface presents the information within a marquee.

57. A computer as recited in claim 49, wherein the user interface modifies a color of the information presents to alternately blend or distinguish the information from the real world images.

58. A computer as recited in claim 49, embodied as a wearable computer that can be worn by the user.

60. A computer as recited in claim 59, wherein the notification is a graphical element.

61. A computer as recited in claim 59, wherein the notification is transparent.

62. A computer as recited in claim 59, embodied as a wearable computer that can be worn by the user.

45. A method comprising:

presenting at least one virtual object on a view of real world images; and

modifying how the virtual object is presented to alter whether the virtual object is more or less visible relative to the real world images.

49. A computer comprising:

a display that facilitates a view of real world images;

a processing unit; and

a software module that executes on the processing unit to present a user interface on the display, the user interface presenting information in a transparent manner to allow a user to view the information without impeding the user'"'"'s view of the real world images.

59. A computer comprising:

a display that facilitates a view of real world images;

a processing unit;

one or more software programs that execute on the processing unit, at least one of the programs generating an event; and

a user interface depicted on the display, where in response to the event, the user interface presents a faintly visible notification overlaid on the real world images to notify the user of the event.

63. One or more computer-readable media storing computer-executable instructions that, when executed, direct a computer to:

display information overlaid on real world images; and

present the information transparently to reduce obstructing a view of the real world images.

64. One or more computer-readable media as recited in claim 63, further storing computer-executable instructions that, when executed, direct a computer to dynamically adjust transparency of the transparent information.

65. One or more computer-readable media as recited in claim 63, further storing computer-executable instructions that, when executed, direct a computer to display a border around the information.

66. One or more computer-readable media as recited in claim 63, further storing computer-executable instructions that, when executed, direct a computer to modify a color of the information to alternately blend or contrast the information with the real world images.

67. One or more computer-readable media storing computer-executable instructions that, when executed, direct a computer to:

receive a notification event; and

in response to the notification event, display a watermark object atop real world images to notify a user of the notification event.

68. One or more computer-readable media storing computer-executable instructions that, when executed, direct a computer to:

ascertain a user'"'"'s context;

display information transparently atop a view of real world images; and

adjust transparency of the information in response to a change in the user'"'"'s context.

69. One or more computer-readable media storing computer-executable instructions that, when executed, direct a computer to:

display information transparently atop a view of real world images;

assign a level of prominence to the information that dictates how prominently the information is displayed to the user; and

adjust the level of prominence assigned to the information.

70. A user interface, comprising:

at least one virtual object overlaid on a view of real world images, the virtual object being transparent; and

a transparency component to dynamically adjust transparency of the virtual object.

71. A user interface as recited in claim 70, wherein the transparency ranges from fully transparent to fully opaque.

72. A system, comprising:

means for presenting at least one virtual object on a view of real world images; and

means for modifying how the virtual object is presented to alter whether the virtual object is more or less visible relative to the real world images.

73. A system as recited in claim 72, wherein the virtual object is transparent and the modifying means alters a degree of transparency.

74. A system as recited in claim 72, wherein the modifying means alters a color of the virtual object.

75. A system as recited in claim 72, wherein the modifying means alters a location of the virtual object relative to the real world images.