Evidence Aggregation Across Heterogeneous Links for Intelligence Gathering Using a Question Answering System

• US 20160140439A1
• Filed: 11/18/2014
• Published: 05/19/2016
• Est. Priority Date: 11/18/2014
• Status: Active Grant

• Current Assignee: International Business Machines Corporation
• Original Assignee: International Business Machines Corporation
• Inventors: Adderly, Darryl M., Allen, Corville O., Tucker, Robert K.
• Granted Patent: US 11,204,929 B2
• Time in Patent Office: Days
• US Class Current: 1/1
• CPC Class Codes: G06F 16/24578 using ranking
• CPC Class Codes: G06F 16/3329 Natural language query formulation or dialogue systems
• CPC Class Codes: G06F 16/367 Ontology
• CPC Class Codes: G06F 40/134 Hyperlinking
• CPC Class Codes: G06F 40/30 Semantic analysis
• CPC Class Codes: G06N 5/022 Knowledge engineering; Knowledge acquisition

Abstract

Mechanisms, in a Question Answering (QA) system comprising a processor and a memory, for evaluating a hypothetical link in an ontology are provided. An initial analysis of the ontology is performed to identify a set of information concept entities and links between information concept entities in the ontology. The hypothetical link between a first information concept entity and a second information concept entity in the ontology is generated based on the initial analysis of the ontology. Natural language questions corresponding to the hypothetical link are processed to generate answer results directed to a plurality of links between a plurality of information concept entities. The answer results are aggregated across the plurality of links to determine an aggregate answer result for the hypothetical link. An indication of whether or not the hypothetical link is a valid link is output based on the aggregate answer result for the hypothetical link.

Claims

1. A method, in a Question Answering (QA) system comprising a processor and a memory, for evaluating a hypothetical link in an ontology, the method comprising:

performing, by a QA system, an initial analysis of the ontology to identify a set of information concept entities and links between information concept entities in the ontology;

generating, by the QA system, the hypothetical link between a first information concept entity and a second information concept entity in the ontology based on the initial analysis of the ontology;

processing, by the QA system, one or more natural language questions corresponding to the hypothetical link, wherein the processing of the one or more natural language questions by the QA system comprises generating answer results directed to a plurality of links between a plurality of information concept entities;

aggregating, by the QA system, the answer results across the plurality of links between the plurality of information concept entities to determine an aggregate answer result for the hypothetical link; and

generating, by the QA system, an indication of whether or not the hypothetical link is a valid link between the first information concept entity and the second information concept entity based on the aggregate answer result for the hypothetical link.

2. The method of claim 1, wherein the answer results directed to the plurality of links comprises a plurality of numerical scores, and wherein each score is indicative of evidential support for a corresponding link in the plurality of links.

3. The method of claim 1, wherein the plurality of links comprises a plurality of links having heterogeneous link types between a plurality of different information concept entities.

4. The method of claim 3, wherein the answer results directed to the plurality of links comprises a plurality of numerical scores, wherein each score is indicative of evidential support for a corresponding link in the plurality of links, and wherein each score is calculated based on a weight value associated with a link type of the corresponding link.

5. The method of claim 4, wherein determining the answer results further comprises, for each link in the plurality of links:

determining a weight value based on the link type and one or more characteristics of at least one of the link or information concept entities associated with the link; and

applying the weight value to an evidence score associated with the link to generate a score for the link.

6. The method of claim 5, wherein the one or more characteristics comprise characteristics of the link, wherein the characteristics of the link comprise at least one of a location associated with the link, a time of day associated with the link, a frequency of action associated with the link, or a direction of action associated with the link.

7. The method of claim 5, wherein determining the weight value based on the link type and one or more characteristics of at least one of the link or information concept entities associated with the link comprises:

correlating a characteristic of the link with a link type characteristic associated with a link type of the link based on a data structure correlating characteristics with types of links;

retrieving an adjustment value associated with the link type characteristic; and

adjusting the weight value based on the retrieved adjustment value.

8. The method of claim 1, further comprising:

determining, by the QA system, whether or not to modify the ontology based on the aggregate answer result; and

in response to determining to modify the ontology, modifying the ontology to include the hypothetical link as an actual link in the ontology to generate an updated ontology.

9. The method of claim 8, wherein generating, by the QA system, an indication of whether or not the hypothetical link is a valid link between the first information concept entity and the second information concept entity based on the aggregate answer result for the hypothetical link comprises outputting a graphical user interface representation of the updated ontology.

10. The method of claim 9, wherein the graphical user interface representation comprises graphical user interface elements for drilling down into information about supporting evidence for the hypothetical link in the updated ontology.

11. A computer program product comprising a computer readable storage medium having a computer readable program stored therein, wherein the computer readable program, when executed on a data processing system, causes the data processing system to implement a Question and Answer (QA) system configured to:

perform an initial analysis of the ontology to identify a set of information concept entities and links between information concept entities in the ontology;

generate the hypothetical link between a first information concept entity and a second information concept entity in the ontology based on the initial analysis of the ontology;

process one or more natural language questions corresponding to the hypothetical link, wherein the processing of the one or more natural language questions by the QA system comprises generating answer results directed to a plurality of links between a plurality of information concept entities;

aggregate the answer results across the plurality of links between the plurality of information concept entities to determine an aggregate answer result for the hypothetical link; and

generate an indication of whether or not the hypothetical link is a valid link between the first information concept entity and the second information concept entity based on the aggregate answer result for the hypothetical link.

12. The computer program product of claim 11, wherein the answer results directed to the plurality of links comprises a plurality of numerical scores, and wherein each score is indicative of evidential support for a corresponding link in the plurality of links.

13. The computer program product of claim 11, wherein the plurality of links comprises a plurality of links having heterogeneous link types between a plurality of different information concept entities.

14. The computer program product of claim 13, wherein the answer results directed to the plurality of links comprises a plurality of numerical scores, wherein each score is indicative of evidential support for a corresponding link in the plurality of links, and wherein each score is calculated based on a weight value associated with a link type of the corresponding link.

15. The computer program product of claim 14, wherein the QA system is further configured to determine the answer results further, for each link in the plurality of links, at least by:

determining a weight value based on the link type and one or more characteristics of at least one of the link or information concept entities associated with the link; and

applying the weight value to an evidence score associated with the link to generate a score for the link.

16. The computer program product of claim 15, wherein the one or more characteristics comprise characteristics of the link, wherein the characteristics of the link comprise at least one of a location associated with the link, a time of day associated with the link, a frequency of action associated with the link, or a direction of action associated with the link.

17. The computer program product of claim 15, wherein determining the weight value based on the link type and one or more characteristics of at least one of the link or information concept entities associated with the link comprises:

correlating a characteristic of the link with a link type characteristic associated with a link type of the link based on a data structure correlating characteristics with types of links;

retrieving an adjustment value associated with the link type characteristic; and

adjusting the weight value based on the retrieved adjustment value.

18. The computer program product of claim 11, wherein the QA system is further configured to:

determine whether or not to modify the ontology based on the aggregate answer result; and

in response to determining to modify the ontology, modify the ontology to include the hypothetical link as an actual link in the ontology to generate an updated ontology.

19. The computer program product of claim 18, wherein the QA system generates an indication of whether or not the hypothetical link is a valid link between the first information concept entity and the second information concept entity based on the aggregate answer result for the hypothetical link at least by outputting a graphical user interface representation of the updated ontology.

20. An apparatus comprising:

a processor; and

a memory coupled to the processor, wherein the memory comprises instructions which, when executed by the processor, cause the processor to implement a Question and Answer (QA) system configured to;

perform an initial analysis of the ontology to identify a set of information concept entities and links between information concept entities in the ontology;

generate the hypothetical link between a first information concept entity and a second information concept entity in the ontology based on the initial analysis of the ontology;

process one or more natural language questions corresponding to the hypothetical link, wherein the processing of the one or more natural language questions by the QA system comprises generating answer results directed to a plurality of links between a plurality of information concept entities;

aggregate the answer results across the plurality of links between the plurality of information concept entities to determine an aggregate answer result for the hypothetical link; and

generate an indication of whether or not the hypothetical link is a valid link between the first information concept entity and the second information concept entity based on the aggregate answer result for the hypothetical link.