INFORMATION MINING USING DOMAIN SPECIFIC CONCEPTUAL STRUCTURES

• US 20080243889A1
• Filed: 06/03/2008
• Published: 10/02/2008
• Est. Priority Date: 02/13/2007
• Status: Active Grant

• Current Assignee: International Business Machines Corporation
• Original Assignee: International Business Machines Corporation
• Inventors: Chen, Ying, Spangler, William Scott, Rhodes, James J., Kreulen, Jeffrey Thomas
• Granted Patent: US 8,805,843 B2
• Time in Patent Office: Days
• US Class Current: 707/100
• CPC Class Codes: G06F 16/355 Class or cluster creation or modification
• CPC Class Codes: G06F 16/358 Browsing; Visualisation therefor
• CPC Class Codes: G06N 5/022 Knowledge engineering; Knowledge acquisition

Abstract

A method and analytics tools for information mining incorporating domain specific knowledge and conceptual structures are disclosed, the method including: providing a first set of documents related to a first topic of interest; using a first taxonomy to categorize the first set of documents into a set of categories; providing a second set of documents related to a second topic of interest; categorizing the second set of documents according to the set of categories of the first set of documents; using an element of domain knowledge to re-categorize the first set of documents; and examining a category to identify a document of interest.

Claims

1. A method for use with 1) a first set of documents related to a first topic of interest and 2) a second set of documents related to a second topic of interest, the method comprising the steps of:

using a first taxonomy to categorize the first set of documents into a set of categories;

categorizing the second set of documents according to the set of categories of the first set of documents; and

examining a category to identify a document of interest, the document of interest being a representative document within the category.

2. The method of claim 1, wherein the first taxonomy is created through a feature space derived from the first set of documents.

3. The method of claim 1, wherein:

the first taxonomy is created through a feature space derived from the first set of documents; and

the feature space includes at least one of unstructured data, structured data, and annotations derived from text of the first set of documents.

4. The method of claim 1, further comprising:

using an element of domain knowledge to re-categorize the first set of documents.

5. The method of claim 1, wherein categorizing the second set of documents further includes:

categorizing the second set of documents according to a criterion chosen from the group consisting of;

text within the second set of documents, structure within the second set of documents, and annotations derived from text within the second set of documents.

6. The method of claim 1, further comprising examining the set of categories to identify correlations between categories.

7. A method for use with a set of documents related to a first topic of interest, the method comprising:

creating a first set of categories of the set of documents according to an automatically generated taxonomy;

creating a second set of categories of the set of documents according to at least one of unstructured data, structured data, and annotations derived from text in the set of documents;

constructing a contingency table having the first set of categories along a first axis and the second set of categories along a second axis; and

identifying a relationship between at least two different categories using the contingency table.

8. The method of claim 7, wherein the contingency table includes cells having respective actual values and for which respective expected values are computed.

9. The method of claim 7, wherein:

the contingency table includes a cell having an actual value and for which an expected value is computed; and

further comprising comparing the expected value against the actual value to identify a category of interest.

10. The method of claim 7, further comprising a step of:

using the contingency table and trending information to identify a recent category with respect to some pre-determined date.

11. The method of claim 7, wherein:

the contingency table includes a cell having an actual value; and

further comprising computing a degree of significance for the actual value.

12. The method of claim 7, wherein:

the contingency table includes a cell having trending information.

13. A method comprising:

extracting a set of documents related to a specified topic from a data warehouse;

generating a taxonomy for the set of documents that provides a first partition of the set of documents according to the taxonomy;

using domain-specific knowledge to re-partition the set of documents to provide a second partition of the set of documents; and

creating a refined taxonomy for the set of documents according to the second partition so that the refined taxonomy incorporates the domain-specific knowledge.

14. The method of claim 13, further comprising:

classifying the set of documents into classification classes independent of the second partition; and

generating a contingency table for comparing the categories of the refined taxonomy with the classification classes of the set of documents.

15. The method of claim 13, further comprising:

classifying the set of documents according to at least one of structured fields, annotations, and a second taxonomy for the set of documents; and

generating a contingency table for comparing the categories of the refined taxonomy with the classification of the set of documents.

16. The method of claim 13, further comprising:

classifying the set of documents into classes independent of the second partition;

generating a contingency table for comparing the categories of the refined taxonomy with the classes of the set of documents; and

identifying a set of most closely-related categories using the contingency table.

17. The method of claim 13, further comprising:

classifying the set of documents into classes independent of the second partition;

generating a contingency table for comparing the categories of the refined taxonomy with the classes of the set of documents; and

identifying a set of recent categories using the contingency table.

18. The method of claim 13, further comprising:

classifying the set of documents into a first set of classes independent of the second partition;

generating a first contingency table for comparing the categories of the refined taxonomy with the first classes of the set of documents;

classifying the set of documents into a second set of classes independent of the second partition; and

generating a second contingency table for comparing the categories of the refined taxonomy with the second classes of the set of documents.

19. A computer program product for use with 1) a first set of documents related to a first topic of interest and 2) a second set of documents related to a second topic of interest, the computer program product comprising a computer useable medium including a computer readable program, wherein the computer readable program when executed on a computer causes the computer to:

categorize the first set of documents into a set of categories using a first taxonomy;

categorize the second set of documents according to the set of categories of the first set of documents; and

examine a category to identify a document of interest, wherein the document of interest typifies the category by most nearly matching a mathematical definition of the category.

20. A computer program product comprising a computer useable medium including a computer readable program, wherein the computer readable program when executed on a computer causes the computer to:

extract a set of documents related to a specified topic from a data warehouse;

generate a taxonomy for the set of documents that provides a first partition of the set of documents according to the taxonomy;

use domain-specific knowledge to re-partition the set of documents to provide a second partition of the set of documents; and

create a refined taxonomy for the set of documents according to the second partition so that the refined taxonomy incorporates the domain-specific knowledge.