SYSTEM, METHOD, AND COMPUTER PROGRAM PRODUCT FOR DATA MINING AND AUTOMATICALLY GENERATING HYPOTHESES FROM DATA REPOSITORIES

• US 20090083208A1
• Filed: 09/15/2008
• Published: 03/26/2009
• Est. Priority Date: 03/15/2006
• Status: Abandoned Application

• Current Assignee: Araicom Research LLC
• Original Assignee: Araicom Research LLC
• Inventors: Raghavan, Vijay V., Xie, Ying, Prestigiacomo, Anthony
• Application Number:
  US12/210,253
• Publication Number:
  US 20090083208A1
• Time in Patent Office: Days
• US Class Current: 706/47
• CPC Class Codes: G06F 16/30 of unstructured textual data document management systems G06F16/93
• CPC Class Codes: G06F 16/35 Clustering; Classification
• CPC Class Codes: G06F 2216/03 Data mining
• CPC Class Codes: G06N 5/02 Knowledge representation; Symbolic representation
• CPC Class Codes: G16H 50/70 for mining of medical data, e.g. analysing previous cases of other patients
• CPC Class Codes: G16H 70/60 relating to pathologies
• CPC Class Codes: Y02A 90/10 Information and communication technologies [ICT] supporting adaptation to climate change, e.g. for weather forecasting or climate simulation

Abstract

Various embodiments of the present invention provide systems, methods, and computer programs for generating a hypothesis. Specifically, some method embodiments include steps for accessing a system for extracting relationships and determining a relationship rule defining a relationship among a plurality of phrases and a plurality of concepts stored in the system for extracting relationships. Such embodiments further provide steps for parsing a plurality of documents in a data repository according to the relationship rule and generating a hypothesis comprising a previously unknown combination of phrases and concepts being at least partially determined from the parsed plurality of documents. Various embodiments also provide a step for presenting the hypothesis to a user so as to indicate the previously unknown combination.

Claims

1. A method for generating a hypothesis, the method comprising:

accessing a system for extracting relationships, the system for extracting relationships comprising a plurality of phrases and a plurality of concepts;

determining a relationship rule defining a relationship among at least a portion of the plurality of phrases and at least a portion of the plurality of concepts;

parsing a plurality of documents in a data repository according to the relationship rule, the plurality of documents each comprising at least a portion of one of the plurality of phrases and the plurality of concepts;

generating a hypothesis comprising a previously unknown combination, the previously unknown combination including one of at least one of the plurality of phrases and at least one of the plurality of concepts, the previously unknown combination being at least partially determined from the parsed plurality of documents; and

presenting the hypothesis so as to indicate the previously unknown combination.

2. A method according to claim 1, wherein the relationship rule is selected from the group consisting of:

an assignment of at least one of the plurality of phrases to at least one of the plurality of concepts;

an assignment of at least one of the plurality of phrases to a relationship identifier, the relationship identifier linking a first one of the plurality of concepts to a second one of the plurality of concepts;

an assignment of at least one of the plurality of concepts to a semantic category;

an arrangement of at least a portion of the plurality of concepts in a hierarchical relationship, wherein a first one of the portion of concepts comprises a child concept and a second one of the portion of concepts comprises a parent concept; and

combinations thereof.

3. A method according to claim 1, wherein at least a portion of the plurality of documents comprises at least one of a first concept, a second concept, and a third concept, and

wherein the parsing step further comprises:

detecting a first relationship between the first and second concepts;

detecting a second relationship between the second and third concepts;

detecting a third relationship between the first and third concepts; and

determining a potential chain relationship among the first second, and third concepts at least partially from the detected first, second, and third relationships; and

wherein the generating step further comprises generating a chain hypothesis comprising the previously unknown combination of the first, second, and third concepts.

4. A method according to claim 1, wherein at least a portion of the plurality of documents comprises at least one of a first concept, a second concept, and a plurality of linking concepts, and

wherein the parsing step further comprises:

detecting a first relationship between the first concept and a first portion of the plurality of linking concepts;

detecting a second relationship between the second concept and a second portion of the plurality of linking concepts; and

determining a potential substitution relationship between the first concept and the second concept at least partially from the detected first and second relationships and a number of overlapping concepts present in both the first portion and the second portion of the plurality of linking concepts; and

wherein the generating step further comprises generating a substitution hypothesis comprising the previously unknown combination of at least one of the first and second concepts with a portion of the plurality of linking concepts not present in the number of overlapping concepts.

5. A method according to claim 4, wherein the parsing step further comprises determining a strength of the potential substitution relationship between the first and second concepts based at least in part on the number of overlapping concepts present in both the first portion of the second portion of the plurality of linking concepts.

6. A method according to claim 1, wherein at least a portion of the plurality of documents comprises at least one of a first concept, a second concept, and a third concept, and

wherein the parsing step further comprises:

detecting a first relationship between the first concept and the second concept;

detecting a second relationship between the second concept and the third concept; and

determining a potential pairwise relationship between the first concept and the third concept at least partially from the detected first and second relationships; and

wherein the generating step further comprises generating a pairwise hypothesis comprising the previously unknown combination of the first and third concepts.

7. A method according to claim 6, wherein the parsing step further comprises assessing a strength of the potential relationship between the first and third concepts at least partially from a known secondary relationship between the first and third concepts.

8. A method according to claim 7, wherein the known secondary relationship comprises a common semantic category including both the first and third concepts.

9. A method according to claim 8, wherein the relationship rule comprises the common semantic category.

10. A method according to claim 1, further comprising:

identifying a portion of the plurality of documents in the data repository associated with a user;

creating a user profile based at least in part on the identified documents, the user profile being indicative of a user information need; and

modifying the hypothesis in response to the user profile such that the modified hypothesis at least partially corresponds to the user information need.

11. A method according to claim 10, wherein the user profile comprises at least one semantic category and wherein the method further comprises filtering the presented hypothesis such that the previously unknown combination includes only at least one phrase and at least one concept corresponding substantially to the at least one semantic category.

12. A method according to claim 1, wherein presenting the hypothesis comprises presenting a display to a user comprising a visual representation of the previously unknown combination including one of at least one of the plurality of phrases and at least one of the plurality of concepts.

13. A method according to claim 12, wherein the visual representation comprises an interactive icon configured to be selectable by the user, the interactive icon being further configured to modify the display when selected by the user.

14. A method according to claim 1, wherein the system for extracting relationships is selected from the group consisting of:

a vocabulary database corresponding to a selected subject area;

a predetermined lexicon;

a semantic network;

a metathesaurus; and

combinations thereof.

15. A method according to claim 1, wherein the data repository is selected from the group consisting of:

a biomedical literature database;

a medical records database;

a chemical literature database;

a computer science literature database;

a physics literature database;

a legal literature database;

a psychology literature database;

a social science literature database;

a news periodical database;

a business journal database; and

combinations thereof.

16. A method according to claim 1, further comprising storing the determined relationship rule for later or repeated use in the subsequent parsing step.

17. A method according to claim 1, further comprising verifying the hypothesis using at least one independent resource.

18. A computer program product for generating a hypothesis based on a plurality of documents in a data repository in a manner that reduces the burden on the data repository, said computer program product comprising a computer-readable storage medium having computer-readable program code portions stored therein, the computer-readable program code portions comprising:

a first set of computer instructions for accessing a system for extracting relationships, the system for extracting relationships comprising a plurality of phrases and a plurality of concepts;

a second set of computer instructions for determining a relationship rule defining a relationship among at least a portion of the plurality of phrases and at least a portion of the plurality of concepts;

a third set of computer instructions for parsing the plurality of documents in the data repository according to the relationship rule, the plurality of documents each comprising at least a portion of one of the plurality of phrases and the plurality of concepts;

a fourth set of computer instructions for generating a hypothesis comprising a previously unknown combination, the previously unknown combination including one of at least one of the plurality of phrases and at least one of the plurality of concepts, the previously unknown combination being at least partially determined from the parsed plurality of documents; and

a fifth set of computer instructions for presenting the hypothesis so as to indicate the previously unknown combination.

19. A computer program product according to claim 18, wherein the relationship rule is selected from the group consisting of:

an assignment of at least one of the plurality of phrases to at least one of the plurality of concepts;

an assignment of at least one of the plurality of phrases to a relationship identifier, the relationship identifier linking a first one of the plurality of concepts to a second one of the plurality of concepts;

an assignment of at least one of the plurality of concepts to a semantic category;

an arrangement of at least a portion of the plurality of concepts in a hierarchical relationship, wherein a first one of the portion of concepts comprises a child concept and a second one of the portion of concepts comprises a parent concept; and

combinations thereof.

20. A computer program product according to claim 18, wherein at least a portion of the plurality of documents comprises at least one of a first concept, a second concept, and a third concept, and

wherein the third set of computer instructions for parsing further comprises:

a sixth set of computer instructions for detecting a first relationship between the first and second concepts;

a seventh set of computer instructions for detecting a second relationship between the second and third concepts;

an eighth set of computer instructions for detecting a third relationship between the first and third concepts; and

a ninth set of computer instructions for determining a potential chain relationship among the first second, and third concepts at least partially from the detected first, second, and third relationships; and

wherein the fourth set of computer instructions for generating further comprises a tenth set of computer instructions for generating a chain hypothesis comprising the previously unknown combination of the first, second, and third concepts.

21. A computer program product according to claim 18, wherein at least a portion of the plurality of documents comprises at least one of a first concept, a second concept, and a plurality of linking concepts, and

wherein the third set of computer instructions for parsing further comprises:

an eleventh set of computer instructions for detecting a first relationship between the first concept and a first portion of the plurality of linking concepts;

a twelfth set of computer instructions for detecting a second relationship between the second concept and a second portion of the plurality of linking concepts; and

a thirteenth set of computer instructions for determining a potential substitution relationship between the first concept and the second concept at least partially from the detected first and second relationships and a number of overlapping concepts present in both the first portion and the second portion of the plurality of linking concepts; and

wherein the fourth set of computer instructions for generating further comprises a fourteenth set of computer instructions for generating a substitution hypothesis comprising the previously unknown combination of at least one of the first and second concepts with a portion of the plurality of linking concepts not present in the number of overlapping concepts.

22. A computer program product according to claim 21, wherein the third set of computer instructions for parsing further comprises a fifteenth set of computer instructions for determining a strength of the potential substitution relationship between the first and second concepts based at least in part on the number of overlapping concepts present in both the first portion of the second portion of the plurality of linking concepts.

23. A computer program product according to claim 18, wherein at least a portion of the plurality of documents comprises at least one of a first concept, a second concept, and a third concept, and

wherein the third set of computer instructions for parsing further comprises:

a sixteenth set of computer instructions for detecting a first relationship between the first concept and the second concept;

a seventeenth set of computer instructions for detecting a second relationship between the second concept and the third concept; and

an eighteenth set of computer instructions for determining a potential pairwise relationship between the first concept and the third concept at least partially from the detected first and second relationships; and

wherein the fourth set of computer instructions for generating further comprises a nineteenth set of computer instructions for generating a pairwise hypothesis comprising the previously unknown combination of the first and third concepts.

24. A computer program product according to claim 23, wherein the third set of computer instructions for parsing further comprises a twentieth set of computer instructions for assessing a strength of the potential relationship between the first and third concepts at least partially from a known secondary relationship between the first and third concepts.

25. A computer program product according to claim 24, wherein the known secondary relationship comprises a common semantic category including both the first and third concepts.

26. A computer program product according to claim 25, wherein the relationship rule comprises the common semantic category.

27. A computer program product according to claim 18, further comprising:

a twenty-first set of computer instructions for identifying a portion of the plurality of documents in the data repository associated with a user;

a twenty-second set of computer instructions for creating a user profile based at least in part on the identified documents, the user profile being indicative of a user information need; and

a twenty-third set of computer instructions for modifying the hypothesis in response to the user profile such that the modified hypothesis at least partially corresponds to the user information need.

28. A computer program product according to claim 27, wherein the user profile comprises at least one semantic category, the computer program product further comprising a twenty-fourth set of computer instructions for filtering the presented hypothesis such that the previously unknown combination includes only at least one phrase and at least one concept corresponding substantially to the at least one semantic category.

29. A computer program product according to claim 18, wherein fifth set of computer instructions for presenting the hypothesis comprises a twenty-fifth set of computer instructions for presenting a display to a user comprising a visual representation of the previously unknown combination including one of at least one of the plurality of phrases and at least one of the plurality of concepts.

30. A computer program product according to claim 29, wherein the visual representation comprises an interactive icon configured to be selectable by the user, the interactive icon being further configured to modify the display when selected by the user.

31. A computer program product according to claim 18, wherein the system for extracting relationships is selected from the group consisting of:

a vocabulary database corresponding to a selected subject area;

a predetermined lexicon;

a semantic network;

a semantic database;

a metathesaurus; and

combinations thereof.

32. A computer program product according to claim 18, wherein the data repository is selected from the group consisting of:

a biomedical literature database;

a medical records database;

a chemical literature database;

a computer science literature database;

a physics literature database;

a legal literature database;

a psychology literature database;

a social science literature database;

a news periodical database;

a business journal database; and

combinations thereof.

33. A computer program product according to claim 18, further comprising a twenty-sixth set of computer instructions for storing the determined relationship rule for later or repeated use in the subsequent parsing step.

34. A computer program product according to claim 18, further comprising a twenty-seventh set of computer instructions for verifying the hypothesis using at least one independent resource.

35. A system for mining information from a data repository comprising a plurality of documents to produce a hypothesis, the system comprising:

a system for extracting relationships comprising a plurality of phrases and a plurality of concepts;

a host computing element in communication with said system for extracting relationships for accessing said system for extracting relationships;

wherein said host computing element determines a relationship rule defining a relationship among at least a portion of the plurality of phrases and at least a portion of the plurality of concepts;

wherein said host computing element parses the plurality of documents in a data repository according to the relationship rule, the plurality of documents each comprising at least a portion of one of the plurality of phrases and the plurality of concepts; and

wherein said host computing element generates the hypothesis comprising a previously unknown combination, the previously unknown combination including one of at least one of the plurality of phrases and at least one of the plurality of concepts, the previously unknown combination being at least partially determined from the parsed plurality of documents; and

a user interface in communication with said host computing element, said user interface configured for presenting the hypothesis so as to indicate the previously unknown combination.

36. A system according to claim 35, wherein said host computing element determines a relationship rule selected from the group consisting of:

an assignment of at least one of the plurality of phrases to at least one of the plurality of concepts;

an assignment of at least one of the plurality of phrases to a relationship identifier, the relationship identifier linking a first one of the plurality of concepts to a second one of the plurality of concepts;

an assignment of at least one of the plurality of concepts to a semantic category;

an arrangement of at least a portion of the plurality of concepts in a hierarchical relationship, wherein a first one of the portion of concepts comprises a child concept and a second one of the portion of concepts comprises a parent concept; and

combinations thereof.

37. A system according to claim 35,

wherein said host computing element identifies a portion of the plurality of documents in the data repository associated with a user;

wherein said host computing element creates a user profile based at least in part on the identified documents, the user profile being indicative of a user information need; and

wherein said host computing element modifies the hypothesis in response to the user profile such that the modified hypothesis at least partially corresponds to the user information need.

38. A system according to claim 37, wherein the user profile comprises at least one semantic category and wherein said host computing element filters the presented hypothesis such that the previously unknown combination includes only at least one phrase and at least one concept corresponding substantially to the at least one semantic category.

39. A system according to claim 35, wherein said user interface presents the hypothesis as a display to a user comprising a visual representation of the previously unknown combination including one of at least one of the plurality of phrases and at least one of the plurality of concepts.

40. A system according to claim 39, wherein said user interface presents the visual representation comprising an interactive icon configured to be selectable by the user, the interactive icon being further configured to modify the display when selected by the user.

41. A system according to claim 35, wherein said system for extracting relationships is selected from the group consisting of:

a vocabulary database corresponding to a selected subject area;

a predetermined lexicon;

a semantic network;

a semantic database;

a metathesaurus; and

combinations thereof.

42. A system according to claim 35, wherein said host computing element is in communication with a data repository selected from the group consisting of:

a biomedical literature database;

a medical records database;

a chemical literature database;

a computer science literature database;

a physics literature database;

a legal literature database;

a psychology literature database;

a social science literature database;

a news periodical database;

a business journal database; and

combinations thereof.

43. A system according to claim 35, further comprising a memory device in communication with said host computing element, said memory device configured for storing the determined relationship rule for later or repeated use in the subsequent parsing step.

44. A system according to claim 35, further comprising an independent resource in communication with said host computing device, said independent resource configured for verifying the generated hypothesis.