Intelligence gathering and analysis using a question answering system

• US 9,892,362 B2
• Filed: 11/18/2014
• Issued: 02/13/2018
• Est. Priority Date: 11/18/2014
• Status: Active Grant

• Current Assignee: International Business Machines Corporation
• Original Assignee: International Business Machines Corporation
• Inventors: Adderly, Darryl M., Allen, Corville O., Tucker, Robert K.
• Primary Examiner(s): Gonzales, Vincent
• Application Number:
  US14/546,417
• Publication Number:
  US 20160140453A1
• Time in Patent Office: 1,183 Days
• CPC Class Codes: G06N 5/022 Knowledge engineering; Knowledge acquisition

Abstract

Mechanisms are provided for identifying and evaluating hypothetical ontological links in an ontology and outputting intelligence information. An ontology is received that comprises a plurality of information concept objects and one or more actual links between the information concept objects. An indication of a selected information concept object for which a hypothetical ontological link is to be evaluated is received. One or more natural language questions are generated for processing by a Question Answering (QA) system pipeline based on at least an identification of a type of the selected information concept object. The one or more natural language questions are processed to generate answer results and a score for the hypothetical ontological link is calculated based on the answer results. Intelligence information associated with the hypothetical ontological link is output based on the score for the hypothetical ontological link.

Claims

1. A method, in a data processing system comprising a processor and a memory, for identifying and evaluating hypothetical ontological links in an ontology and outputting information, the method comprising:

receiving, by the data processing system, an ontology comprising a plurality of information concept objects and one or more actual links between the information concept objects;

receiving, by the data processing system, an indication of at least a selected information concept object for which a hypothetical ontological link is to be evaluated, wherein the hypothetical ontological link is a potential link that is not already present as an actual link in the ontology;

automatically generating, by the data processing system, one or more natural language questions for processing by a Question Answering (QA) system pipeline based on at least an identification of a type of the selected information concept object;

processing, by the QA system pipeline, the one or more natural language questions to generate answer results;

calculating, by the data processing system, a score for the hypothetical ontological link based on the answer results; and

outputting, by the data processing system, information associated with the hypothetical ontological link based on the score for the hypothetical ontological link, wherein calculating a score for the hypothetical ontological link comprises;

calculating a score for each answer result in the generated answer results;

generating a weighted score for each of the answer results; and

combining the weighted scores for each of the answer results to generate the score for the hypothetical ontological link.

2. The method of claim 1, wherein receiving an indication of at least a selected information concept object for which a hypothetical ontological link is to be evaluated comprises receiving a user selection of an information concept object from the ontology and an input of a hypothetical ontological link type associated with the selected information concept object.

3. The method of claim 1, further comprising:

automatically identifying, by the data processing system, the hypothetical ontological link to be evaluated based on the indication of at least a selected information concept object.

4. The method of claim 1, wherein automatically generating one or more natural language questions for processing by the QA system pipeline comprises:

retrieving one or more question templates, corresponding to a type of the selected information concept object and a type of the hypothetical ontological link, from a question template database; and

generating the one or more natural language questions as instances of the one or more question templates.

5. The method of claim 1, wherein calculating a score for the hypothetical ontological link based on the answer results comprises calculating the score based on an evaluation of evidential support for and against the hypothetical ontological link being an actual link between the selected information concept and another information concept, in at least one corpus of evidence data.

6. The method of claim 1, further comprising:

classifying the hypothetical ontological link into one of a plurality of classifications for a type of the hypothetical ontological link, based on an evaluation of evidential support for the hypothetical ontological link.

7. The method of claim 1, wherein the selected information concept object is a data representation of one of a person, a location, an event, or a physical object, and wherein the hypothetical ontological link is a data representation of a relationship between the selected information concept object and another information concept object of a same or different type.

8. The method of claim 1, wherein outputting information associated with the hypothetical ontological link based on the score for the hypothetical ontological link comprises updating the ontology to include the hypothetical ontological link, as an actual link corresponding to the hypothetical ontological link, in the ontology between the selected information concept object and another information concept object in the ontology, in response to the score for the hypothetical ontological link meeting or exceeding a predetermined criterion.

9. The method of claim 8, wherein the actual link corresponding to the hypothetical ontological link, comprises at least one of a corresponding link type from a plurality of possible link types, a link category, from a plurality of possible link categories, associated with the corresponding link type, and the score for the hypothetical ontological link.

10. A computer program product comprising a computer readable storage medium having a computer readable program stored therein, wherein the computer readable program, when executed on a data processing system, causes the data processing system to:

receive an ontology comprising a plurality of information concept objects and one or more actual links between the information concept objects;

receive an indication of at least a selected information concept object for which a hypothetical ontological link is to be evaluated, wherein the hypothetical ontological link is a potential link that is not already present as an actual link in the ontology;

automatically generate one or more natural language questions for processing by a Question Answering (QA) system pipeline based on at least an identification of a type of the selected information concept object;

process, by the QA system pipeline, the one or more natural language questions to generate answer results;

calculate a score for the hypothetical ontological link based on the answer results; and

output information associated with the hypothetical ontological link based on the score for the hypothetical ontological link, wherein the computer readable program further causes the data processing system to calculate a score for the hypothetical ontological link at least by;

calculating a score for each answer result in the generated answer results;

generating a weighted score for each of the answer results; and

combining the weighted scores for each of the answer results to generate the score for the hypothetical ontological link.

11. The computer program product of claim 10, wherein the computer readable program further causes the data processing system to receive an indication of at least a selected information concept object for which a hypothetical ontological link is to be evaluated at least by receiving a user selection of an information concept object from the ontology and an input of a hypothetical ontological link type associated with the selected information concept object.

12. The computer program product of claim 10, wherein the computer readable program further causes the data processing system to:

automatically identify the hypothetical ontological link to be evaluated based on the indication of at least a selected information concept object.

13. The computer program product of claim 10, wherein the computer readable program further causes the data processing system to automatically generate one or more natural language questions for processing by the QA system pipeline at least by:

retrieving one or more question templates, corresponding to a type of the selected information concept object and a type of the hypothetical ontological link, from a question template database; and

generating the one or more natural language questions as instances of the one or more question templates.

14. The computer program product of claim 10, wherein the computer readable program further causes the data processing system to calculate a score for the hypothetical ontological link based on the answer results at least by calculating the score based on an evaluation of evidential support for and against the hypothetical ontological link being an actual link between the selected information concept and another information concept, in at least one corpus of evidence data.

15. The computer program product of claim 10, wherein the computer readable program further causes the data processing system to:

classify the hypothetical ontological link into one of a plurality of classifications for a type of the hypothetical ontological link, based on an evaluation of evidential support for the hypothetical ontological link.

16. The computer program product of claim 10, wherein the selected information concept object is a data representation of one of a person, a location, an event, or a physical object, and wherein the hypothetical ontological link is a data representation of a relationship between the selected information concept object and another information concept object of a same or different type.

17. The computer program product of claim 10, wherein the computer readable program further causes the data processing system to output information associated with the hypothetical ontological link based on the score for the hypothetical ontological link at least by updating the ontology to include the hypothetical ontological link, as an actual link corresponding to the hypothetical ontological link, in the ontology between the selected information concept object and another information concept object in the ontology, in response to the score for the hypothetical ontological link meeting or exceeding a predetermined criterion.

18. The computer program product of claim 17, wherein the actual link corresponding to the hypothetical ontological link, comprises at least one of a corresponding link type from a plurality of possible link types, a link category, from a plurality of possible link categories, associated with the corresponding link type, and the score for the hypothetical ontological link.

19. An apparatus comprising:

a processor; and

a memory coupled to the processor, wherein the memory comprises instructions which, when executed by the processor, cause the processor to;

receive an ontology comprising a plurality of information concept objects and one or more actual links between the information concept objects;

receive an indication of at least a selected information concept object for which a hypothetical ontological link is to be evaluated, wherein the hypothetical ontological link is a potential link that is not already present as an actual link in the ontology;

automatically generate one or more natural language questions for processing by a Question Answering (QA) system pipeline based on at least an identification of a type of the selected information concept object;

process, by the QA system pipeline, the one or more natural language questions to generate answer results;

calculate a score for the hypothetical ontological link based on the answer results; and

output information associated with the hypothetical ontological link based on the score for the hypothetical ontological link, wherein the instructions further cause the processor to calculate a score for the hypothetical ontological link at least by;

calculating a score for each answer result in the generated answer results,

generating a weighted score for each of the answer results; and

combining the weighted scores for each of the answer results to generate the score for the hypothetical ontological link.