Knowledge canvassing using a knowledge graph and a question and answer system

• US 10,586,156 B2
• Filed: 06/25/2015
• Issued: 03/10/2020
• Est. Priority Date: 06/25/2015
• Status: Active Grant

• Current Assignee: International Business Machines Corporation
• Original Assignee: International Business Machines Corporation
• Inventors: Merdivan, Erinc, Riendeau, John A., Sirotkovic, Jadran, Whitley, Michael D., Zhang, Le
• Primary Examiner(s): Hershley, Mark E
• Assistant Examiner(s): Ohba, Mellissa M.
• Application Number:
  US14/749,733
• Publication Number:
  US 20160378851A1
• Time in Patent Office: 1,720 Days
• Field of Search: None
• CPC Class Codes: G06F 16/3329 Natural language query formulation or dialogue systems
• CPC Class Codes: G06F 16/3344 using natural language analysis
• CPC Class Codes: G06F 16/338 Presentation of query results
• CPC Class Codes: G06F 16/9024 Graphs; Linked lists G06F16/9027 takes precedence
• CPC Class Codes: G06F 40/247 Thesauruses; Synonyms
• CPC Class Codes: G06N 5/022 Knowledge engineering; Knowledge acquisition

Abstract

Mechanisms for processing a knowledge canvassing request receive a request specifying an entity of interest from an originator of the request and analyze the request to extract a feature of the request. The mechanisms determine whether the request is a targeted natural language question to be answered or a knowledge canvassing request, based on the extracted feature. In response to determining that the request is a knowledge canvassing request, the mechanisms process the request by identifying entities represented in a knowledge graph data structure as being related to the entity of interest. The mechanisms output results of the processing of the request to the originator of the request.

Claims

1. A computer program product comprising a computer readable storage medium having a computer readable program stored therein, wherein the computer readable program, when executed on a data processing system, causes the data processing system to implement a cognitive system for processing a knowledge canvassing request, wherein the computer readable program causes the data processing system to:

receive, by the cognitive system, a request specifying at least one entity of interest from an originator of the request, wherein the cognitive system comprises a request classification engine, a factoid QA system pipeline, and a knowledge canvassing pipeline;

analyze, by the cognitive system, the request to extract one or more features of the request;

determine, by the request classification engine, whether the request is a targeted natural language question to be answered or a knowledge canvassing request, based on a comparison of the one or more extracted features against one or more classification rules or patterns;

in response to determining that the request is a targeted natural language question, route, by the request classification engine, the request to the factoid QA system pipeline which processes the request as a natural language question using natural language processing (NLP) mechanisms;

in response to determining that the request is a knowledge canvassing request, route, by the request classification engine, the request to the knowledge canvassing pipeline and process, by the knowledge canvassing pipeline, the request by identifying entities represented in a knowledge graph data structure as being related to the at least one entity of interest to suggest other areas of potential interest to the originator of the request; and

output, by the cognitive system, results of the processing of the request to the originator of the request,

wherein processing the request comprises;

identifying, by the cognitive system, entities in the request;

performing, by relationship search logic within the knowledge canvassing pipeline, a search of the knowledge graph to find corresponding nodes to the identified entities and identifying related nodes in the knowledge graph that have links connecting the identified entities with the related nodes representing related entities;

analyzing, by passage relevancy scoring logic within the knowledge canvassing pipeline, evidence passages of a corpus of documents that are associated with the related nodes to determine whether there is support for relationships between the identified entities and the related entities;

selecting, by candidate relationship generation logic within the knowledge canvassing pipeline, a set of candidate relationships between identified entities and related entities based on candidate relationship selection criteria;

performing, by independent relevancy scoring logic within the knowledge canvassing pipeline, analysis of the set of candidate relationships that is context independent;

identifying, by context dependent relevancy scoring logic within the knowledge canvassing pipeline, portions of the corpus of documents where entities of the set of candidate relationships are mentioned and evaluates a context of the portions of the corpus of documents to determine a context dependent metric to associate with each candidate relationship;

determining, by final merging and ranking logic within the knowledge canvassing pipeline, a ranked listing of the set of candidate relationships based on context dependent and context independent metrics; and

retrieving, by evidence passage retrieval logic within the knowledge canvassing pipeline, evidential passages from the corpus of documents that reference the entities in the ranked set of candidate relationships.

2. The computer program product of claim 1, wherein the knowledge graph data structure comprises a plurality of nodes representing different entities identified in a corpus of information ingested by the data processing system, and edges between nodes representing relationships between the entities corresponding to the nodes as identified by analyzing a context of references to the entities in the corpus of information.

3. The computer program product of claim 2, wherein the computer readable program further causes the data processing system to process the request by identifying entities represented in the knowledge graph data structure as being related to the at least one entity of interest at least by identifying at least one node in the knowledge graph corresponding to the at least one entity of interest and related entities connected to the at least one node by one or more edges in the knowledge graph data structure.

4. The computer program product of claim 3, wherein the computer readable program further causes the data processing system to process the request at least by ranking the related entities by scoring each related entity in accordance with a relatedness metric associated with edges connecting the related entity to the at least one entity of interest, and wherein the computer readable program further causes the data processing system to output results of the processing at least by outputting a ranked listing of the related entities based on the ranking of the related entities.

5. The computer program product of claim 4, wherein the relatedness metric is an inverse document frequency (IDF) metric representing a degree of rarity of the relationship between the related entity and the entity of interest in the corpus of information.

6. The computer program product of claim 4, wherein the ranking of the related entities is performed in accordance with a domain specific ranking criteria indicating a criteria that is of greater desirability in the context of the domain.

7. The computer program product of claim 4, wherein the computer readable program further causes the data processing system to receive a user input selecting a related entity in the ranked listing of related entities to be a new entity of interest, and repeating the analyze, determine, process, and output operations with the new entity of interest being the entity of interest associated with a new request to identify related entities.

8. The computer program product of claim 4, wherein scoring each related entity comprises generating, for the related entity, a context independent score and a context dependent score, and calculating a score for the related entity based on a combination of the context independent score and the context dependent score.

9. The computer program product of claim 1, wherein the computer readable program further causes the data processing system to output results of the processing of the request to the originator of the request at least by outputting results comprising a listing of one or more related entities and, for each related entity of the one or more related entities, a portion of at least one text passage from a corpus of information in which a relationship of the related entity with the entity of interest is referenced.

10. An apparatus comprising:

a processor; and

a memory coupled to the processor, wherein the memory comprises instructions which, when executed by the processor, cause the processor to implement a cognitive system for processing a knowledge canvassing request, wherein the instructions causes the processor to;

receive, by the cognitive system, a request specifying at least one entity of interest from an originator of the request, wherein the cognitive system comprises a request classification engine, a factoid QA system pipeline, and a knowledge canvassing pipeline;

analyze, by the cognitive system, the request to extract one or more features of the request;

determine, by the request classification engine, whether the request is a targeted natural language question to be answered or a knowledge canvassing request, based on a comparison of the one or more extracted features against one or more classification rules or patterns;

in response to determining that the request is a targeted natural language question, route, by the request classification engine, the request to the factoid QA system pipeline which processes the request as a natural language question using natural language processing (NLP) mechanisms;

in response to determining that the request is a knowledge canvassing request, route, by the request classification engine, the request to the knowledge canvassing pipeline and process, by the knowledge canvassing pipeline, the request by identifying entities represented in a knowledge graph data structure as being related to the at least one entity of interest to suggest other areas of potential interest to the originator of the request; and

output, by the cognitive system, results of the processing of the request to the originator of the request,

wherein processing the request comprises;

identifying, by the cognitive system, entities in the request;

performing, by relationship search logic within the knowledge canvassing pipeline, a search of the knowledge graph to find corresponding nodes to the identified entities and identifying related nodes in the knowledge graph that have links connecting the identified entities with the related nodes representing related entities;

analyzing, by passage relevancy scoring logic within the knowledge canvassing pipeline, evidence passages of a corpus of documents that are associated with the related nodes to determine whether there is support for relationships between the identified entities and the related entities;

selecting, by candidate relationship generation logic within the knowledge canvassing pipeline, a set of candidate relationships between identified entities and related entities based on candidate relationship selection criteria;

performing, by independent relevancy scoring logic within the knowledge canvassing pipeline, analysis of the set of candidate relationships that is context independent;

identifying, by context dependent relevancy scoring logic within the knowledge canvassing pipeline, portions of the corpus of documents where entities of the set of candidate relationships are mentioned and evaluates a context of the portions of the corpus of documents to determine a context dependent metric to associate with each candidate relationship;

determining, by final merging and ranking logic within the knowledge canvassing pipeline, a ranked listing of the set of candidate relationships based on context dependent and context independent metrics; and

retrieving, by evidence passage retrieval logic within the knowledge canvassing pipeline, evidential passages from the corpus of documents that reference the entities in the ranked set of candidate relationships.

11. The apparatus of claim 10, wherein the knowledge graph data structure comprises a plurality of nodes representing different entities identified in a corpus of information ingested by the data processing system, and edges between nodes representing relationships between the entities corresponding to the nodes as identified by analyzing a context of references to the entities in the corpus of information.

12. The apparatus of claim 11, wherein the instructions further cause the processor to process the request by identifying entities represented in the knowledge graph data structure as being related to the at least one entity of interest at least by identifying at least one node in the knowledge graph corresponding to the at least one entity of interest and related entities connected to the at least one node by one or more edges in the knowledge graph data structure.

13. The apparatus of claim 12, wherein the instructions further cause the processor to process the request at least by ranking the related entities by scoring each related entity in accordance with a relatedness metric associated with edges connecting the related entity to the at least one entity of interest, and wherein the computer readable program further causes the data processing system to output results of the processing at least by outputting a ranked listing of the related entities based on the ranking of the related entities.

14. The apparatus of claim 13, wherein the relatedness metric is an inverse document frequency (IDF) metric representing a degree of rarity of the relationship between the related entity and the entity of interest in the corpus of information.

15. The apparatus of claim 13, wherein the ranking of the related entities is performed in accordance with a domain specific ranking criteria indicating a criteria that is of greater desirability in the context of the domain.

16. The apparatus of claim 13, wherein the instructions further cause the processor to receive a user input selecting a related entity in the ranked listing of related entities to be a new entity of interest, and repeating the analyze, determine, process, and output operations with the new entity of interest being the entity of interest associated with a new request to identify related entities.

17. The apparatus of claim 13, wherein scoring each related entity comprises generating, for the related entity, a context independent score and a context dependent score, and calculating a score for the related entity based on a combination of the context independent score and the context dependent score.

18. The apparatus of claim 10, wherein the instructions further cause the processor to output results of the processing of the request to the originator of the request at least by outputting results comprising a listing of one or more related entities and, for each related entity of the one or more related entities, a portion of at least one text passage from a corpus of information in which a relationship of the related entity with the entity of interest is referenced.