Significance of relationships discovered in a corpus

• US 10,706,362 B2
• Filed: 10/09/2017
• Issued: 07/07/2020
• Est. Priority Date: 12/02/2015
• Status: Active Grant

• Current Assignee: International Business Machines Corporation
• Original Assignee: International Business Machines Corporation
• Inventors: Gordon, John B., Hogan, John P., Kottaram, Sanjay F.
• Primary Examiner(s): Chaki, Kakali
• Assistant Examiner(s): Seck, Ababacar
• Application Number:
  US15/727,693
• Publication Number:
  US 20180032887A1
• Time in Patent Office: 1,002 Days
• Field of Search: 706 12
• CPC Class Codes: G06F 40/40 Processing or translation of natural language natural language analysis G06F40/20; semantic analysis G06F40/30
• CPC Class Codes: G06F 9/453 Help systems
• CPC Class Codes: G06N 20/00 Machine learning
• CPC Class Codes: G06N 3/02 Neural networks
• CPC Class Codes: G06N 5/02 Knowledge representation; Symbolic representation
• CPC Class Codes: G06N 5/022 Knowledge engineering; Knowledge acquisition
• CPC Class Codes: G06N 5/04 Inference or reasoning models
• CPC Class Codes: G09B 7/04 characterised by modifying the teaching programme in response to a wrong answer, e.g. repeating the question, supplying a further explanation

Abstract

Certain relationships representing material insights are identified from among a set of discovered relationships. Cognitive discovery of relationships in a knowledge base, or corpus, are ranked according to one or more metrics indicative of material insights, including recentness and degree of alignment.

Claims

1. A method comprising:

identifying a focus within a natural language question;

mining a first body of information for disclosure of entities within the first body of information;

determining relationships among the entities based on the focus;

generating a targeting document containing a first relationship and a second relationship, the first and second relationships being relationships among the entities determined from the first body of information;

recording mining data for relationships in the targeting document in support of an importance criteria;

ranking the first relationship with respect to the second relationship based on the recorded mining data and according to the importance criteria as a set of ranking data; and

storing the targeting document including the set of ranking data and mining data in the first body of information for on-demand access during a question-answer session corresponding to the domain of knowledge;

wherein;

the importance criteria is a first degree to which first relationship is known and a second degree to which the second relationship is known according to a dictionary of commonly known relationships for the domain of knowledge; and

the first relationship being ranked as more important than the second relationship because the first degree is smaller than the second degree.

2. The method of claim 1, further comprising:

parsing the natural language question to identify the focus.

3. The method of claim 1, wherein the importance criteria is further based upon an age of the first relationship compared to an age of the second relationship.

4. The method of claim 1, wherein the importance criteria relates to a comparison between a first set of evidence that identifies the first relationship with a second set of evidence that identifies the second relationship.

5. The method of claim 4, wherein the comparison between the first set of evidence and the second set of evidence is based at least in part upon a number of evidentiary documents within each of the first set of evidence and the second set of evidence.

6. The method of claim 1, further comprising:

registering the corresponding set of relationships in a relationship store having a set of relationships identified from the first body of information.

7. The method of claim 1 further comprising:

reporting to a user a set of top-ranked relationships in the targeting document.

8. The method of claim 1, further comprising:

classifying an identified relationship as a less-trusted fact; and

modifying the classification of the identified relationship from the less-trusted fact to a more-trusted fact based at least in part upon a count of trusted sources that support the identified relationship.

9. The method of claim 1, wherein the natural language question is presented as a training question corresponding to a new document to be added to the first body of information.

10. The method of claim 9, further comprising:

incorporating the new document into the first body of information to form a second body of information.

11. The method of claim 1, further comprising:

reporting the set of ranking data as input to determine a restated question.

12. A computer program product comprising a computer-readable storage medium having a set of instructions stored therein which, when executed by a processor, causes the processor to determine a significance ranking for discovered relationships by:

identifying a focus within a natural language question;

mining a first body of information for disclosure of entities within the first body of information;

determining relationships among the entities based on the focus;

generating a targeting document containing a first relationship and a second relationship, the first and second relationships being relationships among the entities determined from the first body of information;

recording mining data for relationships in the targeting document in support of an importance criteria;

ranking the first relationship with respect to the second relationship based on the recorded mining data and according to the importance criteria as a set of ranking data; and

storing the targeting document including the set of ranking data and mining data in the first body of information for on-demand access during a question-answer session corresponding to the domain of knowledge;

wherein;

the importance criteria is a first degree to which first relationship is known and a second degree to which the second relationship is known according to a dictionary of commonly known relationships for the domain of knowledge; and

the first relationship being ranked as more important than the second relationship because the first degree is smaller than the second degree.

13. The computer program product of claim 12, further comprising:

parsing the natural language question to identify the focus.

14. The computer program product of claim 12, wherein the importance criteria is further based upon an age of the first relationship compared to an age of the second relationship.

15. The computer program product of claim 12, wherein the importance criteria relates to a comparison between a first set of evidence that identifies the first relationship with a second set of evidence that identifies the second relationship.

16. The computer program product of claim 15, wherein the comparison between the first set of evidence and the second set of evidence is based at least in part upon a number of evidentiary documents within each of the first set of evidence and the second set of evidence.

17. A computer system comprising:

a processor set; and

a computer readable storage medium;

wherein;

the processor set is structured, located, connected, and/or programmed to run program instructions stored on the computer readable storage medium; and

the program instructions which, when executed by the processor set, cause the processor set to determine a significance ranking for discovered relationships by;

identifying a focus within a natural language question;

mining a first body of information for disclosure of entities within the first body of information;

determining relationships among the entities based on the focus;

generating a targeting document containing a first relationship and a second relationship, the first and second relationships being relationships among the entities determined from the first body of information;

recording mining data for relationships in the targeting document in support of an importance criteria;

ranking the first relationship with respect to the second relationship based on the recorded mining data and according to the importance criteria as a set of ranking data; and

storing the targeting document including the set of ranking data and mining data in the first body of information for on-demand access during a question-answer session corresponding to the domain of knowledge;

wherein;

the importance criteria is a first degree to which first relationship is known and a second degree to which the second relationship is known according to a dictionary of commonly known relationships for the domain of knowledge; and

the first relationship being ranked as more important than the second relationship because the first degree is smaller than the second degree.

18. The computer system of claim 17, further comprising:

parsing the natural language question to identify the focus.

19. The computer system of claim 17, wherein the importance criteria is further based upon an age of the first relationship compared to an age of the second relationship.

20. The computer system of claim 17, wherein the importance criteria relates to a comparison between a first set of evidence that identifies the first relationship with a second set of evidence that identifies the second relationship.