Entity-relation based passage scoring in a question answering computer system

• US 9,904,667 B2
• Filed: 11/20/2014
• Issued: 02/27/2018
• Est. Priority Date: 11/20/2014
• Status: Expired due to Fees

• Current Assignee: International Business Machines Corporation
• Original Assignee: International Business Machines Corporation
• Inventors: Kalyanpur, Aditya A., Murdock, IV, James W.
• Primary Examiner(s): MORRISON, JAY A
• Application Number:
  US14/548,359
• Publication Number:
  US 20160147871A1
• Time in Patent Office: 1,195 Days
• CPC Class Codes: G06F 16/24522 Translation of natural language queries to structured queries
• CPC Class Codes: G06F 16/288 Entity relationship models
• CPC Class Codes: G06F 16/3329 Natural language query formulation or dialogue systems
• CPC Class Codes: G06F 16/3344 using natural language analysis
• CPC Class Codes: G06F 40/205 Parsing
• CPC Class Codes: G06F 40/211 Syntactic parsing, e.g. based on context-free grammar [CFG] or unification grammars
• CPC Class Codes: G06F 40/284 Lexical analysis, e.g. tokenisation or collocates

Abstract

According to an aspect, a query and a passage are parsed by a language parser to detect noun-centric phrases and verb-centric phrases in the query and the passage. Entities, including at least one untyped entity, are identified based on the noun-centric phrases and relations are identified based on the verb-centric phrases. Entity pairs are created that include an entity identified in the query and an entity identified in the passage, each pair satisfies a matching criteria. Relation pairs are created that include a relation identified in the query and a relation identified in the passage, each pair satisfies a matching criteria. A passage score that indicates the likelihood that an answer to the query is contained in the passage is determined based on the entity pairs, the matching criteria satisfied by each entity pair, the elation pairs, and the matching criteria satisfied by each relation pair.

Claims

1. A computer program product comprising:

a computer readable storage medium readable by a processing circuit of a question answering computer system and storing instructions for execution by the processing circuit to perform a method comprising;

parsing, by a language parser, a query and a passage to detect noun-centric phrases and verb-centric phrases in the query and the passage;

identifying entities, for both of the query and the passage, based on the noun-centric phrases detected from the parsing, the entities from each of the query and the passage including at least one untyped entity that includes a noun and at least one modifier;

identifying relations, for both of the query and the passage, based on the verb-centric phrases detected from the parsing;

creating one or more entity pairs that each include an entity identified in the query and an entity identified in the passage, each entity pair satisfying a matching criteria with respect to entities of the entity pair and at least one entity pair including an untyped entity identified in the query and an untyped entity identified in the passage, the matching criteria with respect to entities of the entity pair including a degree of statistical similarity between the entities of the entity pair;

creating one or more relation pairs that each include a relation identified in the query and a relation identified in the passage, each relation pair satisfying a matching criteria with respect to relations of the relation pair;

calculating, by the question answering computer system, for each of the relation pairs, a relation match confidence score that indicates a level of confidence that the relations of the relation pair match;

determining, by the question answering computer system, a passage score for the passage that indicates a likelihood that an answer to the query is contained in the passage, the determining based on the one or more entity pairs, the matching criteria satisfied by each entity pair, the one or more relation pairs, the matching criteria satisfied by each relation pair, and the one or more of the relation match confidence scores,

wherein the passage score is accessible to a computer system for determining a relevance of the passage to the query, the query received from an agent external to the computer system.

2. The computer program product of claim 1, wherein the method further comprises:

calculating, by the question answering computer system, for each of the entity pairs, an entity match confidence score that indicates a level of confidence that the entities of the entity pair match,

wherein the determining is further based on the one or more entity match confidence scores.

3. The computer program product of claim 1, wherein at least one of the relations is an untyped relation.

4. The computer program product of claim 1, wherein the language parser includes a part-of-speech parser and a dependency parser.

5. The computer program product of claim 1, wherein the identifying of at least one of the relations is further based on an inference using a rule obtained from a knowledge base.

6. The computer program product of claim 1, wherein each entity includes one or more terms.

7. The computer program product of claim 1, wherein the statistical similarity is based on at least one of whether the entity identified in the query is the same as the entity identified in the passage, whether the entity identified in the query is a synonym of the entity identified in the passage, and whether the entity identified in the query is determined by latent semantic analysis (LSA) to be statistically similar to the entity identified in the passage.

8. The computer program product of claim 1, wherein the matching criteria with respect to relations of the relation pair includes at least one of the relation identified in the query is the same as the relation identified in the passage, the relation identified in the query is a synonym of the relation identified in the passage, and the relation identified in the query is determined by LSA to be statistically similar to the relation identified in the passage.

9. A system comprising:

a memory having computer readable instructions; and

a processor for executing the computer readable instructions, the computer readable instructions including;

parsing, by a language parser, a query and a passage to detect noun-centric phrases and verb-centric phrases in the query and the passage;

identifying entities, for both of the query and the passage, based on the noun-centric phrases detected from the parsing, the entities from each of the query and the passage including at least one untyped entity that includes a noun and at least one modifier;

identifying relations, for both of the query and the passage, based on the verb-centric phrases detected from the parsing;

creating one or more entity pairs that each include an entity identified in the query and an entity identified in the passage, each entity pair satisfying a matching criteria with respect to entities of the entity pair and at least one entity pair including an untyped entity identified in the query and an untyped entity identified in the passage, the matching criteria with respect to entities of the entity pair including a degree of statistical similarity between the entities of the entity pair;

creating one or more relation pairs that each include a relation identified in the query and a relation identified in the passage, each relation pair satisfying a matching criteria with respect to relations of the relation pair;

calculating, by the question answering computer system, for each of the relation pairs, a relation match confidence score that indicates a level of confidence that the relations of the relation pair match; and

determining, by the question answering computer system, a passage score for the passage that indicates a likelihood that an answer to the query is contained in the passage, the determining based on the one or more entity pairs, the matching criteria satisfied by each entity pair, the one or more relation pairs, the matching criteria satisfied by each relation pair, and the one or more of the relation match confidence scores,

wherein the passage score is accessible to a computer system for determining a relevance of the passage to the query, the query received from an agent external to the computer system.

10. The system of claim 9, wherein the computer readable instructions further include:

calculating, by the question answering computer system, for each of the entity pairs, an entity match confidence score that indicates a level of confidence that the entities of the entity pair match,

wherein the determining is further based on the one or more entity match confidence scores.

11. The system of claim 9, wherein at least one of the relations is an untyped relation.

12. The system of claim 9, wherein the identifying of at least one of the relations is further based on an inference using a rule obtained from a knowledge base.