Enhanced information delivery using a transparent display

• US 9,646,522 B2
• Filed: 06/29/2012
• Issued: 05/09/2017
• Est. Priority Date: 06/29/2012
• Status: Active Grant

• Current Assignee: Intel Corporation
• Original Assignee: Intel Corporation
• Inventors: Ratcliff, Joshua J., Lyons, Kenton M.
• Primary Examiner(s): Amini, Javid A
• Application Number:
  US13/538,914
• Publication Number:
  US 20140002486A1
• Time in Patent Office: 1,775 Days
• Field of Search: 345633
• CPC Class Codes: G06F 3/011 Arrangements for interaction with the human body, e.g. for user immersion in virtual reality blind teaching G09B21/00
• CPC Class Codes: G06F 3/017 Gesture based interaction, e.g. based on a set of recognized hand gestures interaction based on gestures traced on a digitiser G06F3/04883
• CPC Class Codes: G06F 3/04815 Interaction with a metaphor-based environment or interaction object displayed as three-dimensional, e.g. changing the user viewpoint with respect to the environment or object
• CPC Class Codes: G09G 3/003 to produce spatial visual effects

Abstract

Information is delivered about a particular external environment using a transparent display. In one embodiment, a method includes determining a position of a mobile transparent display, determining an orientation of the display, retrieving information about the environment of the display using the determined position and orientation, and overlaying the retrieved information over a view on the transparent display.

Claims

1. A method comprising:

determining a position of a mobile transparent display;

determining an orientation of the display;

determining information using the determined position and orientation about a particular celestial object in a user'"'"'s view of the sky through the display;

receiving a gesture toward the particular object using a camera associated with the mobile transparent display;

retrieving a first stored image of the particular object from an astronomy database and projecting the first image as an overlay on the display;

retrieving a second stored image of the particular object from the database, the second stored image being a zoomed in image with respect to the first image, and projecting the second image as an overlay on the display instead of the first image in response to the zoom gesture; and

observing user hand gestures using the camera and interpreting the observed gestures as commands to control the presentation of information on the transparent display to present different images of the articular object.

2. The method of claim 1, wherein determining an orientation comprises determining the direction in which the display is oriented and inferring what may be seen through the transparent display.

3. The method of claim 1, wherein determining an orientation comprises using at least one of a compass, accelerometers, a pitch sensor and a camera facing the direction of the view on the transparent display.

4. The method of claim 1, wherein determining an orientation comprises using a camera facing the direction of the view on the transparent display, comparing an image from the camera to reference images at the determined position, each reference image being associated with an orientation, selecting a reference image corresponding to the image from the camera, and using the orientation associated with the selected reference image as the determined orientation.

5. The method of claim 1, wherein determining an orientation comprises tracking movement of a user'"'"'s head, the method further comprising adjusting an apparent position of the overlayed information to compensate for the tracked movements.

6. The method of claim 1, wherein overlaying the information comprises using a camera oriented in the direction of the view on the transparent display to determine reference points in the view and positioning the overlaid information with respect to the reference points.

7. The method of claim 1, further comprising:

receiving a command through a user gesture to call up a particular type of information on the display;

retrieving information, such as names, categories, and connections, about the environment of the display using the determined position and orientation in response to the command; and

overlaying the retrieved information over a view on the transparent display.

18. The method of claim 7, wherein the particular object is identified as a star and the information is information concerning a constellation that includes the identified object including lines that connect to other stars of the constellation.

19. The method of claim 1, further comprising receiving a user gesture toward the particular object using a camera associated with the mobile transparent display and wherein retrieving and projecting the first stored image is in response to the user gesture.

8. A machine-readable non-transitory medium having instructions stored thereon, that when operated on by the machine cause the machine to perform operations comprising:

determining a position of a mobile transparent display;

determining an orientation of the display;

determining information using the determined position and orientation about a particular celestial object in a user'"'"'s view of the sky through the display;

receiving a gesture toward the particular object using a camera associated with the mobile transparent display;

retrieving a first stored image of the particular object from an astronomy database and projecting the first image as an overlay on the display; and

retrieving a second stored image of the particular object from the database, the second stored image being a zoomed in image with respect to the first image, and projecting the second image as an overlay on the display instead of the first image in response to the zoom gesture; and

observing user hand gestures using the camera and interpreting the observed gestures as commands to control the presentation of information on the transparent display to present different images of the particular object.

9. The medium of claim 8, wherein determining an orientation comprises determining the direction in which the display is oriented using compass and inferring what may be seen through the transparent display.

10. The medium of claim 8, wherein determining an orientation comprises comparing an image from the camera to reference images at the determined position, each reference image being associated with an orientation, selecting a reference image corresponding to the image from the camera, and using the orientation associated with the selected reference image as the determined orientation.

11. The medium of claim 8, wherein the particular object is identified as a celestial object, the operations further comprising:

receiving a command through a user gesture to call up a particular type of information on the display;

retrieving information about the environment of the display using the determined position and orientation in response to the command; and

overlaying the retrieved information over a view on the transparent display, wherein the information is information concerning an environment, surface, and size of the identified celestial object.

12. An apparatus comprising:

a transparent display;

a positioning system to determine a position of the display;

an orientation system to determine an orientation of the display;

a computing system to receive the determined position and orientation, to determine information about celestial objects in view of a user of the sky viewing through the transparent display, to receive a gesture toward a particular object in a user'"'"'s view through the display using a camera associated with the mobile transparent display, to retrieve a first stored image of the particular object from an astronomy database and project the first image as an overlay on the display in response to the received gesture, to receive a gesture to zoom in on the particular object using the camera associated with the mobile transparent display, to retrieve a second stored image of the particular object from the database in response to the received zoom gesture, the second stored image being a zoomed in image with respect to the first image, and to project the second image as an overlay on the display instead of the first image, to observe user hand gestures using the camera and interpreting the observed gestures as commands to control the presentation of information on the transparent display to resent different images of the particular object; and

a wireless communications interface coupled to the computing system to send the position and orientation information to a remote server and to receive the information from the database about objects from the remote server.

13. The apparatus of claim 12, further comprising a gesture to screen synchronization module to receive gesture information from a camera of the apparatus to relate the gesture to the particular object and to interpret the gesture.

14. The apparatus of claim 12, further comprising a camera to observe real objects in view of the display, wherein the computing system determines reference points in the objects observed by the camera and positions the overlaid information with respect to the reference points.

15. The apparatus of claim 12, further comprising a camera oriented in the direction of the view of the user through the transparent display to observe user gestures and wherein the computing system interprets the observed user gestures as commands to control the presentation of information on the transparent display.

16. The apparatus of claim 12, wherein the transparent display is head-mounted.

17. The apparatus of claim 16, wherein the transparent display is in the form of eyeglasses that project information in the view of the user through the lenses of the eyeglasses.