Context-aware imaging device

• US 20030095681A1
• Filed: 11/21/2001
• Published: 05/22/2003
• Est. Priority Date: 11/21/2001
• Status: Abandoned Application

• Current Assignee: Hewlett-Packard Development Company, L.P. (HP Inc.)
• Original Assignee: Hewlett-Packard Development Company, L.P. (HP Inc.)
• Inventors: Burg, Bernard, Sayers, Craig P.
• Application Number:
  US09/989,181
• Publication Number:
  US 20030095681A1
• Time in Patent Office: Days
• US Class Current: 382/100
• CPC Class Codes: G06F 16/9537 Spatial or temporal dependent retrieval, e.g. spatiotemporal queries
• CPC Class Codes: H04N 1/00204 with a digital computer or a digital computer system, e.g. an internet server programmed control between transmitter and receiver or between image input and image output device H04N1/32561
• CPC Class Codes: H04N 1/00323 with a measuring, monitoring or signaling apparatus, e.g. for transmitting measured information to a central location
• CPC Class Codes: H04N 2101/00 Still video cameras

Abstract

A context-aware imaging device is described. The device includes an image capturing and display system having an image capturing system that captures an image containing a landmark of interest, and an image display that displays the captured image. A context interpretation engine is provided to generate contextual information relating to the landmark. A context rendering module is coupled to the context interpretation engine to render the contextual information to the user of the imaging device. All modules of the context-aware imaging device form a physically integrated unit.

Claims

1. A context-aware imaging device, comprising:

an image capturing and display system that captures and displays an image containing a landmark of interest;

a context interpretation engine that generates contextual information relating to the landmark, wherein the image capturing and display system and the context interpretation engine form a physically integrated unit.

2. The imaging device of claim 1, wherein context interpretation engine generates the contextual information by

determining geographical information of the landmark in the captured image;

searching a landmark database with the geographical information of the landmark to obtain the contextual information of the landmark.

3. The imaging device of claim 1, wherein the context interpretation engine further comprises

an area determination system that determines the geographical information of the landmark;

a landmark database that stores geographical information of landmarks and their corresponding contextual information to provide the contextual information of the landmark if accessed with the geographical information of the landmark.

4. The imaging device of claim 3, wherein the area determination system further comprises

a location sensor that provides location information of the imaging device;

an orientation sensor that determines the direction in which the image capturing and display system is aiming;

a context interpreter that generates the geographical information of the landmark by defining a segmented viewing volume within which the landmark is located using the location, the direction, and the orientation information.

5. The imaging device of claim 3, wherein the area determination system further comprises

a location sensor that provides location information of the imaging device;

an orientation sensor that determines the direction in which the image capturing and display system is aiming;

a zoom sensor that determines the viewing angle of the image capturing and display system;

a context interpreter that generates the geographical information of the landmark by defining a segmented viewing volume within which the landmark is located using the location, the viewing direction, and the zoom information provided by the sensors.

6. The imaging device of claim 3, wherein the area determination system further comprises

a location sensor that provides location information of the imaging device;

a distance sensor that determines the distance to the landmark from the image capturing and display system;

a context interpreter that generates the geographical information of the landmark from the location and distance information provided by the sensors.

7. The imaging device of claim 3, wherein the area determination system further comprises

a location sensor that provides location information of the imaging device;

an orientation sensor that determines the direction in which the image capturing and display system is aiming;

a distance sensor that determines the distance from the image capturing and display system to the landmark;

a context interpreter that generates the geographical information of the landmark from the location, the viewing direction, and the distance information provided by the sensors.

8. The imaging device of claim 3, wherein the area determination system further comprises

an image feature extractor that extracts searchable image features from the landmark in the captured image;

a context interpreter that uses the image features to search the landmark database for any landmark image with matching image features.

9. The imaging device of claim 3, wherein the area determination system further comprises

a location sensor that provides location information of the imaging device;

an image feature extractor that extracts searchable image features from the landmark in the captured image;

a context interpreter that uses the image features and the location information to search the landmark database for any landmark image with matching image features and similar location information.

10. The imaging device of claim 3, wherein the area determination system further comprises

an orientation sensor that determines the direction in which the image capturing and display system is aiming;

an image feature extractor that extracts searchable image features from the landmark in the captured image;

a context interpreter that uses the image features and the direction information to search the landmark database for any landmark image with matching image features and along the same direction.

11. The imaging device of claim 3, wherein the area determination system further comprises

a zoom sensor that determines the viewing scope of the image capturing and display system;

an image feature extractor that extracts searchable image features from the landmark in the captured image;

a context interpreter that uses the image features and the viewing scope information to search the landmark database for any landmark image with matching image features and within the viewing scope specified by the zoom sensor.

12. The imaging device of claim 3, wherein the area determination system further comprises

a distance sensor that determines the distance from the image capturing and display system to the landmark;

an image feature extractor that extracts searchable image features from the landmark in the captured image;

a context interpreter that uses the image features and the distance information to search the landmark database for any landmark image with matching image features and within the distance specified by the distance sensor.

13. The imaging device of claim 3, wherein the area determination system further comprises

a zoom and distance sensor that determines the projection angle of the image capturing and display system, and the distance from the image capturing and display system to a geographical point at which the image capturing and display system is focused;

an image feature extractor that extracts searchable image features from the landmark in the captured image;

a context interpreter that uses the image features, the projection angle and the distance information to search the landmark database for any landmark image with matching image features and within the projection angle and distance specified by the zoom and distance sensor.

14. The imaging device of claim 3, wherein the area determination system further comprises

a location sensor that provides location information of the imaging device;

an orientation sensor that determines the direction in which the image capturing and display system is aiming;

a zoom and distance sensor that determines the projection angle of the image capturing and display system, and the distance from the image capturing and display system to a geographical point at which the image capturing and display system is focused;

an image feature extractor that extracts searchable image features from the landmark in the captured image;

a context interpreter that uses the image features, the projection angle and the distance information to search the landmark database for any landmark image with matching image features and within the projection angle and distance specified by the zoom and distance sensor.

15. The imaging device of claim 3, wherein the area determination system further comprises a geographical information extractor coupled to the image capturing and display system to extract the geographical information of the landmark from the captured image.

16. The imaging device of claim 15, wherein the image capturing and display system further comprises an image sensor that captures the image with the landmark, wherein the image sensor also collects the geographical information of the landmark and attaches the geographical information to the image.

17. The imaging device of claim 1, wherein the context interpretation engine further comprises an updating module that can provide real time updates to the contextual and geographical information of each of the landmarks stored in the engine.

18. The imaging device of claim 17, wherein the updating module further comprises

a wireless communication interface that establishes wireless communication with external wireless network;

an update request module that browsers external Internet via the wireless communication interface to obtain the real time updates.

19. The imaging device of claim 1, wherein the image capturing and display system and the context interpretation engine reside inside a single enclosure.

20. The imaging device of claim 1, wherein the image capturing and display system and the context interpretation engine reside in different enclosures, but still physically attached to each other to form the physically integrated unit.

21. The imaging device of claim 1, wherein modules of the context interpretation engine reside in different enclosures, and have intermittent connectivity with each other.

22. The imaging device of claim 1, further comprising a context rendering module coupled to the context interpretation engine to render the contextual information relating to the landmark to the user of the imaging device.

23. The imaging device of claim 22, wherein the context rendering module is a display that can be either separated from or overlaid with a display of the image capturing and display system.

24. The imaging device of claim 22, wherein the context rendering module is an audio player.

25. The imaging device of claim 1, wherein the image capturing and display system can be selected from a group comprising a binoculars system, a telescope system, an eyeglass system, a camera system, a digital camera system, and a video camera system.

26. The imaging device of claim 1, wherein the context interpretation engine further comprises

a user interface that allows user inputs to the context interpretation engine;

a storage that stores user inputs from the user interface, wherein the storage also stores the captured image of the landmark and its contextual information.