Dynamic information extraction with self-organizing evidence construction

• US 20050154701A1
• Filed: 12/01/2004
• Published: 07/14/2005
• Est. Priority Date: 12/01/2003
• Status: Abandoned Application

• Current Assignee: Techteam Government Solutions Incorporated (Jacobs Solutions, Inc.)
• Original Assignee: Techteam Government Solutions Incorporated (Jacobs Solutions, Inc.)
• Inventors: Brueckner, Sven, Parunak, H. Van Dyke, Weinstein, Peter, Sauter, John
• Application Number:
  US11/001,555
• Publication Number:
  US 20050154701A1
• Time in Patent Office: Days
• US Class Current: 1/1
• CPC Class Codes: G06F 16/35 Clustering; Classification
• CPC Class Codes: G06F 16/367 Ontology

Abstract

A data analysis system with dynamic information extraction and self-organizing evidence construction finds numerous applications in information gathering and analysis, including the extraction of targeted information from voluminous textual resources. One disclosed method involves matching text with a concept map to identify evidence relations, and organizing the evidence relations into one or more evidence structures that represent the ways in which the concept map is instantiated in the evidence relations. The text may be contained in one or more documents in electronic form, and the documents may be indexed on a paragraph level of granularity. The evidence relations may self-organize into the evidence structures, with feedback provided to the user to guide the identification of evidence relations and their self-organization into evidence structures. A method of extracting information from one or more documents in electronic form includes the steps of clustering the document into clustered text; identifying patterns in the clustered text; and matching the patterns with the concept map to identify evidence relations such that the evidence relations self-organize into evidence structures that represent the ways in which the concept map is instantiated in the evidence relations.

Claims

1. A method of extracting information from text, comprising the steps of:

matching the text with a concept map to identify evidence relations; and

organizing the evidence relations into one or more evidence structures that represent the ways in which the concept map is instantiated in the evidence relations.

2. The method of claim 1, wherein the text is contained in one or more documents in electronic form.

3. The method of claim 2, wherein the documents are indexed on a paragraph level of granularity.

4. The method of claim 1, including the step of allowing the evidence relations to self-organize into the evidence structures.

5. The method of claim 4, including the use of feedback from the user to guide the identification of evidence relations and their self-organization into evidence structures.

6. The method of claim 1, further including the steps of:

identifying patterns in the text; and

matching the text with the concept map using the patterns.

7. The method of claim 6, wherein the patterns use linguistically-oriented regular expressions to recognize relations in the text.

8. The method of claim 1, wherein the text is preprocessed to identify basic grammatical constituents such as noun phrases and verb phrases.

9. The method of claim 8, further including the step of resolving pronoun references and similar linguistic phenomena that have a significant presence in the test.

10. The method of claim 1, wherein the evidence relations include a reference to a document, a paragraph, or metadata.

11. The method of claim 1, wherein the evidence relations include a reference to the pattern used to match the concept map relation, and the terms in the document text that were matched to the pattern.

12. The method of claim 1, wherein the evidence relations include a reference to the exact terms in the text that match to the concept map concepts and relations.

13. The method of claim 12, wherein the terms are as specific as or more specific than the corresponding concepts and relations in the concept map.

14. The method of claim 1, wherein the evidence relations include an estimate as to the confidence in the evidence relation, based on the match of the relation to the textual data.

15. The method of claim 14, wherein the confidence estimate is based in part on a measure of the absence of supporting evidence.

16. The method of claim 15, wherein the confidence reflects the degree to which the evidence relation fits with other evidence into the larger pattern defined by the concept map.

17. The method of claim 1, further including the step of clustering the text prior to matching the text with the concept map.

18. The method of claim 17, wherein the evidence structures represent the ways in which the concept map is instantiated in the document evidence by providing mutually compatible evidence relations connected to each other according to the template provided by the concept map.

23. The method of claim 1, wherein each document is preprocessed to identify basic grammatical constituents such as noun phrases and verb phrases.

24. The method of claim 23, further including the step of resolving pronoun references and similar linguistic phenomena that have a significant presence in the test.

19. A method of extracting information from one or more documents in electronic form, comprising the steps of:

clustering the document into clustered text;

identifying patterns in the clustered text; and

matching the patterns with the concept map to identify evidence relations, whereby the evidence relations self-organize into evidence structures that represent the ways in which the concept map is instantiated in the evidence relations.

20. The method of claim 19, including the use of feedback from the user to guide the identification of patterns, the matching of textual patterns with the concept map, and their self-organization into evidence structures.

21. The method of claim 20, wherein the documents are indexed on the paragraph level of granularity.

22. The method of claim 20, wherein the patterns use linguistically-oriented regular expressions to recognize relations in the text.

25. The method of claim 19, wherein the evidence relations include a reference to a document, a paragraph, or metadata.

26. The method of claim 19, wherein the evidence relations include a reference to the pattern used to match the concept map relation, and the terms in the document text that were matched to the pattern.

27. The method of claim 19, wherein the evidence relations include a reference to the exact terms in the text that match to the concept map concepts and relations.

28. The method of claim 27, wherein the terms are as specific, or more specific, than the corresponding concepts and relations in the concept map.

29. The method of claim 19, wherein the evidence relations include an estimate as to the confidence in the evidence relation, based on the match of the relation to the textual data.

30. The method of claim 29, wherein the confidence estimate is based in part on a measure of the absence of supporting evidence.

31. The method of claim 29, wherein the confidence reflects the degree to which the evidence relation fits with other evidence into the larger pattern defined by the concept map.

32. The method of claim 19, wherein the evidence structures represent the ways in which the concept map is instantiated in the document evidence by providing mutually compatible evidence relations connected to each other according to the template provided by the concept map.