Text and imagery spatial correlator

• US 7,117,192 B2
• Filed: 05/23/2001
• Issued: 10/03/2006
• Est. Priority Date: 05/23/2001
• Status: Expired due to Fees

• Current Assignee: MDA Information Systems LLC (Galileo Parent, Inc.)
• Original Assignee: Veridian Erim International Inc.
• Inventors: Waltz, Edward, Berthiaume, Richard A.
• Primary Examiner(s): Knight, Anthony
• Assistant Examiner(s): Chang, Sunray
• Application Number:
  US09/863,513
• Publication Number:
  US 20020178134A1
• Time in Patent Office: 1,959 Days
• Field of Search: 706/48, 706/45, 717/117, 702/76, 382/219, 382/217, 382/155, 382/113, 707/6, 707/3
• US Class Current: 706/48
• CPC Class Codes: G06F 16/29 Geographical information databases
• CPC Class Codes: G06F 16/30 of unstructured textual data document management systems G06F16/93
• CPC Class Codes: Y10S 707/99933 Query processing, i.e. searching
• CPC Class Codes: Y10S 707/99936 Pattern matching access

Abstract

A text and imagery spatial correlator automatically relates the geographic locations where events referenced in text occur to those same geographic locations appearing in imagery. In the preferred embodiment, the system deploys adaptable, context-sensitive agents for the automatic interpretation of text, and the application of those agents to classify and geolocate textual references. The results are integrated with similar spatial references from imagery in a common data model, accessible by both spatial and non-spatial data correlation. The unique combination of elements enables the system to automatically develop spatial relationships between text and imagery. Although the invention finds utility in numerous applications, the system and method are particularly useful in correlating remote sensor imagery.

Claims

1. A method of correlating text and imagery, comprising the steps of:

specifying a target concept;

providing textual material and imagery;

training a text search detector to examine the textual material for text regions which relate to the target concept, and creating a text target detection record in a database A in the event of a match or other meaningful association;

training a discriminating feature detector to search for locations within the imagery which relate to the target concept, and creating a location target detection record in a database B in the event of a match or other meaningful association; and

comparing the records in both databases to declare an approximate correlation, if any, indicative of a common target concept.

2. The method of claim 1, wherein the target concept is an event or object.

3. The method of claim 1, wherein the discriminating features within the imagery include infrared, multispectral or spatial features.

4. The method of claim 1, wherein the step of training the text search detector includes the steps of:

a) defining a search phrase;

b) testing the phrase against a validation set, and

c) repeating a) and b) until all relevant targets in the validation set are detected.

5. The method of claim 1, wherein the examination of the textual material includes searching the text regions for geographic location text associated with the target concept.

6. The method of claim 1, further including the step of generating a concept identifier code in both the text and image target detection records using a lookup table in the event of a match or other meaningful association.

7. The method or claim 6, wherein the searching of the text regions is accomplished by reference to a Gazeteer of place names and their corresponding lat-long locations.

8. The method of claim 7, wherein the text target detection record contains:

a text document ID number,

an index to locate a paragraph or passage within the document.

the target concept identifier code (CIC), and

the latitude-longitude (LL) value.

9. The method of claim 6, wherein the search for locations within the imagery includes extracting a lat-long location.

10. The method of claim 9, wherein the location target detection record contains:

an image ID) number,

an index to locate the target within the image,

the target concept identifier code (CIC), and

the latitude-longitude (LL) value.

11. The method of claim 1, wherein the target concept within the imagery is in the form of a pixel index.

12. The method of claim 10, wherein:

the searching of the rest regions is accomplished by reference to a Gazercer of place names and their corresponding lat-long locations; and

the text target detection record contains;

a text document ID number,

an index to locate a paragraph or passage within the document,

the target concept identifier code (CIC), and

the latitude-longitude (LL) value.

13. The method of claim 2, wherein the event may be characterized as an emergency, tragedy, disaster or crisis.

14. The method of claim 2, wherein the object involves an environmental asset, structure, or mode of transportation.

15. The method of claim 1, wherein either or both of the steps associated with examining the textual material or searching for locations within the imagery are carried out in a batch mode or as part of a recursive flow.

16. A text and imagery spatial correlator, comprising:

a document text parsing and interpretation engine which uses a context-based search to generate topical information;

an imagery engine operative to associate the components of an image with known spatial features and generate location information; and

a matching subsystem operative to associate the topical information with the location information and present a result to a user.

17. The text and imagery spatial correlator of claim 16, wherein text parsing and interpretation engine includes a user-trainable agent to define the context of interest in a current search.

18. The text and imagery spatial correlator of claim 16, wherein the topical information concerns an event or an object.

19. The text and imagery spatial correlator of claim 18, wherein the event may be characterized as an emergency, tragedy, disaster or crisis.

20. The text and imagery spatial correlastorof claim 18, whcitin to object involves an environmental asset, structure or mode of transportation.

21. The text and imagery spadal correlator of claim 16, wherein to matching subsystem is operative to associate the topical information with the location information in a batch mode or as part of a recursive flow.