Systems and methods for linking content to individual image features

• US 8,670,617 B2
• Filed: 05/14/2008
• Issued: 03/11/2014
• Est. Priority Date: 05/14/2008
• Status: Active Grant

• Current Assignee: Terrago Technologies Incorporated (In-Q-Tel)
• Original Assignee: Terrago Technologies Incorporated (In-Q-Tel)
• Inventors: Knoblock, Craig A., Shahabi, Cyrus, Chen, Ching-Chien, Kapoor, Dipsy
• Primary Examiner(s): Li, Ruiping
• Application Number:
  US12/152,546
• Publication Number:
  US 20090285487A1
• Time in Patent Office: 2,127 Days
• Field of Search: 382/190, 707/E17.107, 707/E17.11, 707/E17.116, 707/E17.117, 707/999.003, 707/999.006
• US Class Current: 382/190
• CPC Class Codes: G06F 16/444 Spatial browsing, e.g. 2D maps, 3D or virtual spaces
• CPC Class Codes: G06V 20/176 Urban or other man-made structures

Abstract

A method, computer program, and system for linking content to individual image features are provided. A section of an image is identified. A plurality of features associated with the section of the image is determined. Each of the plurality of features corresponds to at least one position within the section of the image. Content associated with the plurality of features is retrieved from a content repository. The content is linked to the plurality of features based on at least one rule. The content is then presented.

Claims

1. A method comprising:

receiving an identification of a section of an image;

determining a plurality of features included in a geoknowledgebase that are associated with the section of the image, each of the plurality of features in the geoknowledgebase corresponding to at least one position in the geoknowledgebase that is within the section of the image;

determining information in the geoknowledgebase related to the at least one position corresponding to each of the plurality of features;

extracting the related information from the geoknowledgebase for use as keywords in searching a content repository;

searching content in the content repository for the keywords to identify content comprising descriptions of the determined plurality of features;

retrieving, via a computing device, content that includes the keywords from the content repository, wherein the retrieved content comprises video, and wherein speech recognition software is used to convert speech included in an audio file of the video to text for searching using an extraction module;

linking the retrieved content to the determined plurality of features;

filtering the plurality of features associated with the identified section of the image based on a user-specified filter parameter to select at least one feature that satisfies conditions indicated by the user-specified filter parameter, using a filter module; and

presenting content linked to the at least one feature selected from the determined plurality of features.

2. The method of claim 1, further comprising determining a spatial extent of each of the plurality of features and aligning a coordinate system of each of the features in the geoknowledgebase with the coordinate system of the image.

3. The method of claim 2, wherein the extracted keywords are the features.

4. The method of claim 1, further comprising selecting the at least one feature using a rule based on a type of the content.

5. The method of claim 1, wherein the image is a frame of a video, and an alignment of the features is based on the geoknowledgebase.

6. The method of claim 1, further comprising marking at least one of the plurality of features with at least one icon.

7. The method of claim 6, wherein the appearance of the at least one icon is based on a characteristic of the content or the plurality of features.

8. A non-transitory computer readable storage medium having embodied thereon a program, the program being executable by a processor for performing a method, the method comprising:

receiving an identification of a section of an image comprising features;

delimiting a spatial query of a geoknowledgebase to only those features within the section of the image;

determining a plurality of features from the delimited query of the geoknowledgebase, the plurality of features corresponding to at least one position within the section of the image;

using the determined plurality of features as keywords for searching content in a content repository for occurrences of the keywords;

assigning content to an index in the content repository;

searching content in the content repository for occurrences of keywords in the content using the index;

retrieving content from the content repository based on the index, the retrieved content including at least one occurrence of the keywords, wherein the retrieved content comprises video, and wherein speech recognition software is used to convert speech included in an audio file of the video to text for searching using an extraction module;

linking the retrieved content to the plurality of features;

presenting content that has been linked to at least one of the plurality of features associated with the section of the image;

filtering the plurality of features associated with the identified section of the image based on a user-specified filter parameter to select at least one feature that satisfies conditions indicated by the user-specified filter parameter, using a filter module; and

presenting an indication of the linking of at least one feature to the presented content.

9. The computer readable storage medium of claim 8, further comprising determining a spatial extent of each of the plurality of features.

10. The computer readable storage medium of claim 9, further comprising linking the retrieved content to the plurality of features, based on at least one rule, the at least one rule is based on the spatial extent.

11. The computer readable storage medium of claim 8, further comprising determining a type of the content.

12. The computer readable storage medium of claim 11, wherein the section of the image is a floor plan of a building and the type of the content is plumbing, the features of the building being aligned with a geographical coordinate system defined by a geoknowledgebase such that depictions of the features in the section of the image correspond to actual physical locations of the features.

13. The computer readable storage medium of claim 8, wherein the geoknowledgebase includes a plurality of datasets, and determining the plurality of features is based on features described by parcel data included in a parcel dataset of the geoknowledgebase.

14. The computer readable storage medium of claim 8, further comprising marking at least one of the plurality of features with at least one icon.

15. The computer readable storage medium of claim 14, wherein the appearance of the at least one icon is based on a characteristic of the content or the plurality of features.

16. A system comprising:

an interface module configured to receive an identification of a section of an image;

a features module configured to determine a plurality of features included in a geoknowledgebase that are associated with the section of the image, each of the plurality of features in the geoknowledgebase corresponding to at least one position included in the geoknowledgebase, the at least one position being within the section of the image, such that only features including a position within the section of the image are determined by the features module;

a filter module configured to reduce a number of features determined by the features module using filter parameters received from a user such that only features that satisfy conditions indicated by the filter parameters and within the section of the image may be determined by the features module, and wherein the features module is further configured to determine a spatial extent of each of the plurality of features;

an extraction module configured to use the determined plurality of features as keywords to search documents stored in a content repository, and to retrieve content from the content repository, the retrieved content including one or more keywords and comprising a description of features portrayed in the section of the image, wherein the content retrieved by the extraction module comprises video, and wherein speech recognition software is used to convert speech included in an audio file of the video to text for searching using the extraction module;

a rules module configured to determine a link between each of the retrieved content and each of the plurality of features based on at least one rule; and

the interface module further configured to present at least one content and linked feature.

17. The system of claim 16, wherein the section of the image is a blueprint of a building and the at least one rule is based on the spatial extent of features in the blueprint.

18. The system of claim 16, wherein the image is a frame of a video and the extraction module is further configured to determine a type of the content of the video.

19. The system of claim 16, wherein the features module is further configured to determine the plurality of features based on features described by parcel data included in a parcel dataset of the geoknowledgebase.

20. The system of claim 16, wherein the interface module is further configured to mark at least one of the plurality of features with at least one icon.

21. The system of claim 20, wherein the appearance of the at least one icon is based on a characteristic of the content or the plurality of features.