Identifying and displaying relationships between candidate answers

• US 9,558,263 B2
• Filed: 12/05/2013
• Issued: 01/31/2017
• Est. Priority Date: 12/05/2013
• Status: Active Grant

• Current Assignee: International Business Machines Corporation
• Original Assignee: International Business Machines Corporation
• Inventors: Visotski, William G., Wilson, David E., Yates, Robert L.
• Primary Examiner(s): Jacob, Ajith
• Application Number:
  US14/097,697
• Publication Number:
  US 20150161242A1
• Time in Patent Office: 1,153 Days
• US Class Current: 1/1
• CPC Class Codes: G06F 16/3329 Natural language query formulation or dialogue systems
• CPC Class Codes: G06F 16/3334 Selection or weighting of terms from queries, including natural language queries
• CPC Class Codes: G06F 16/3344 using natural language analysis
• CPC Class Codes: G06F 16/338 Presentation of query results
• CPC Class Codes: G06F 16/36 Creation of semantic tools, e.g. ontology or thesauri

Abstract

Mechanisms are provided for identifying commonalities between candidate answers generated by a Question and Answer (QA) system in response to an input question. The mechanisms receive a plurality of candidate answers for an input question from the QA system and identify terms present in the candidate answers. The mechanisms determine relationships between terms in each of the candidate answers and determine a common relationship between a first term and a second term, the common relationship being common amongst at least a subset of the plurality of candidate answers, based on the determined relationships between terms in each of the candidate answers. The mechanisms present the plurality of candidate answers and the common relationship to a user.

Claims

1. A computer program product comprising a computer readable storage medium having a computer readable program stored therein, wherein the computer readable program, when executed on a computing device, causes the computing device to:

receive a plurality of candidate answers for an input question from a Question and Answer (QA) system;

identify terms present in the plurality of candidate answers;

determine relationships between terms the plurality of candidate answers;

determine a common relationship between terms in a subset of the plurality of candidate answers, the common relationship being between a first term and a second term in each candidate answer of the subset of the plurality of candidate answers based on the determined relationships between terms the plurality of candidate answers; and

present the plurality of candidate answers and the common relationship to a user.

2. The computer program product of claim 1, wherein the computer readable program further causes the computing device to determine the common relationship by determining an intersection of the relationships between terms in the subset of the plurality of candidate answers.

3. The computer program product of claim 1, wherein the computer readable program further causes the computing device to determine the common relationship:

searching a relationship data store comprising a plurality of entries, each entry corresponding to a relationship between a first found term and at least one second found term found during pre-processing of documents of at least one corpus; and

for a term found in a candidate answer in the plurality of candidate answers, identifying one or more entries in the relationship data store that match the term.

4. The computer program product of claim 3, wherein each entry in the relationship data store comprises a pairwise relationship between a first term found during pre-processing of at least one document of a corpus, and a second term found during pre-processing of the at least one document of the corpus, based on relationships identified in the at least one document of the corpus through natural language processing of the at least one document of the corpus.

5. The computer program product of claim 4, wherein each entry in the relationship data store further comprises a corresponding frequency of occurrence value indicating a frequency at which a corresponding pairwise relationship was found in the at least one document of the corpus during the pre-processing.

6. The computer program product of claim 5, wherein the computer readable program further causes the computing device to present the plurality of candidate answers and the common relationship to the user by generating a visual display of the plurality of candidate answers and the common relationship along with the corresponding frequency of occurrence value for the common relationship.

7. The computer program product of claim 6, wherein the visual display further comprises at least one evidence text passage from at least one document of a corpus that supports the common relationship, and wherein portions of at least one of the subset of the plurality of candidate answers or the evidence text passages are accentuated in the visual display based on a value corresponding to the frequency of occurrence value for the common relationship.

8. The computer program product of claim 1, wherein the computer readable program further causes the computing device to determine the common relationship between terms in the subset of the plurality of candidate answers by:

presenting a graphical user interface (GUI) providing a first portion of the GUI for outputting the plurality of candidate answers, a second portion of the GUI for outputting a plurality of user selectable options for specifying a desired common relationship that the user wishes to have identified between the plurality of candidate answers, a third portion of the GUI for outputting the relationships between terms in the plurality of candidate answers and the common relationship between terms in the subset of the plurality of candidate answers, and a fourth portion of the GUI for outputting evidence text passages from documents of a corpus that support the common relationship between terms in the subset of the plurality of candidate answers.

9. The computer program product of claim 1, wherein the computer readable program further causes the computing device to present the plurality of candidate answers and the common relationship between terms in the subset of the plurality of candidate answers to the user by generating a visual display of the plurality of candidate answers and at least one evidence text passage from at least one document of a corpus that support the common relationship between terms in the subset of the plurality of candidate answers, with portions of at least one of the subset of the plurality of candidate answers, or the at least one evidence text passage, being accentuated in the visual display.

10. An apparatus comprising:

a processor; and

a memory coupled to the processor, wherein the memory comprises instructions which, when executed by the processor, cause the processor to;

receive a plurality of candidate answers for an input question from a Question and Answer (QA) system;

identify terms present in the plurality of candidate answers;

determine relationships between terms the plurality of candidate answers;

determine a common relationship between terms in a subset of the plurality of candidate answers, the common relationship being between a first term and a second term in each candidate answer of the subset of the plurality of candidate answers based on the determined relationships between terms the plurality of candidate answers; and

present the plurality of candidate answers and the common relationship to a user.

11. The apparatus of claim 10, wherein the instructions further cause the processor to determine the common relationship by determining an intersection of the relationships between terms in the subset of the plurality of candidate answers.

12. The apparatus of claim 10, wherein the instructions further cause the processor to determine the common relationship by:

searching a relationship data store comprising a plurality of entries, each entry corresponding to a relationship between a first found term and at least one second found term found during pre-processing of documents of at least one corpus; and

for a term found in a candidate answer in the plurality of candidate answers, identifying one or more entries in the relationship data store that match the term.

13. The apparatus of claim 12, wherein each entry in the relationship data store comprises a pair-wise relationship between a first term found during pre-processing of at least one document of a corpus, and a second term found during pre-processing of the at least one document of the corpus, based on relationships identified in the at least one document of the corpus through natural language processing of the at least one document of the corpus.

14. The apparatus of claim 13, wherein each entry in the relationship data store further comprises a corresponding frequency of occurrence value indicating a frequency at which a corresponding pairwise relationship was found in the at least one document of the corpus during the pre-processing.

15. The apparatus of claim 14, wherein the instructions further cause the processor to present the plurality of candidate answers and the common relationship to the user by generating a visual display of the plurality of candidate answers and the common relationship along with the corresponding frequency of occurrence value for the common relationship.

16. The apparatus of claim 15, wherein the visual display further comprises at least one evidence text passage from at least one document of a corpus that supports the common relationship, and wherein portions of at least one of the subset of the plurality of candidate answers or the evidence text passages are accentuated in the visual display based on a value corresponding to the frequency of occurrence value for the common relationship.

17. The apparatus of claim 10, wherein the instructions further cause the processor to determine the common relationship between terms in the subset of the plurality of candidate answers by:

presenting a graphical user interface (GUI) providing a first portion of the GUI for outputting the plurality of candidate answers, a second portion of the GUI for outputting a plurality of user selectable options for specifying a desired common relationship that the user wishes to have identified between the plurality of candidate answers, a third portion of the GUI for outputting the relationships between terms in the plurality of candidate answers and the common relationship between terms in the subset of the plurality of candidate answers, and a fourth portion of the GUI fir outputting evidence text passages from documents of a corpus that support the common relationship between terms in the subset of the plurality of candidate answers.

18. The apparatus of claim 10, wherein the instructions further cause the processor to present the plurality of candidate answers and the common relationship between terms in the subset of the plurality of candidate answers to the user by generating a visual display of the plurality of candidate answers and at least one evidence text passage from at least one document of a corpus that support the common relationship between terms in the subset of the plurality of candidate answers, with portions of at least one of the subset of the plurality of candidate answers, or the at least one evidence text passage, being accentuated in the visual display.