CONCEPT NOISE REDUCTION IN DEEP QUESTION ANSWERING SYSTEMS

• US 20140129492A1
• Filed: 11/08/2012
• Published: 05/08/2014
• Est. Priority Date: 11/08/2012
• Status: Active Grant

• Current Assignee: International Business Machines Corporation
• Original Assignee: International Business Machines Corporation
• Inventors: Petri, John A., Stevens, Richard J.
• Granted Patent: US 9,092,740 B2
• Time in Patent Office: Days
• US Class Current: 706/12
• CPC Class Codes: G06N 20/00 Machine learning
• CPC Class Codes: G09B 19/0053 Computers, e.g. programming

Abstract

Method, computer program product, and system to perform an operation for a deep question answering system. The operation begins by computing a concept score for a first concept in a first case received by the deep question answering system, the concept score being based on a machine learning concept model for the first concept. The operation then excludes the first concept from consideration when analyzing a candidate answer and an item of supporting evidence to generate a response to the first case upon determining that the concept score does not exceed a predefined concept minimum weight threshold. The operation then increases a weight applied to the first concept when analyzing the candidate answer and the item of supporting evidence to generate the response to the first case when the concept score exceeds a predefined maximum weight threshold.

Claims

1. A computer program product for a deep question answering system, the computer program product comprising:

a computer-readable storage medium having computer-readable program code embodied therewith, the computer-readable program code comprising;

computer-readable program code configured to compute a concept score for a first concept in a first case received by the deep question answering system, wherein the concept score is based on a machine learning concept model for the first concept;

computer-readable program code configured to exclude the first concept from consideration when analyzing a candidate answer and an item of supporting evidence to generate a response to the first case when the concept score does not exceed a predefined concept minimum weight threshold; and

computer-readable program code configured to increase a weight applied to the first concept when analyzing the candidate answer and the item of supporting evidence to generate the response to the first case when the concept score exceeds a predefined maximum weight threshold.

2. The computer program product of claim 1, further comprising:

computer-readable program code configured to compute a plurality of concept matching scores for a plurality of concepts based on a respective weight factor established by the machine learning concept model; and

computer-readable program code configured to aggregate the plurality of computed concept matching scores into a single relevancy score, wherein the single relevancy score is used to generate a response to the first case.

3. The computer program product of claim 1, further comprising computer-readable program code configured to train the deep question answering system to generate the machine learning concept model, wherein the computer-readable program code configured to train the deep question answering system comprises:

computer-readable program code configured to provide the deep question answering system with a sample case and a sample answer;

computer-readable program code configured to identify the first concept in the sample case;

computer-readable program code configured to compute a concept matching score for the first concept in the sample case based on the sample answer; and

computer-readable program code configured to include the concept matching score in the machine learning concept model.

4. The computer program product of claim 3, wherein producing the machine learning concept model further comprises identifying a type of question included in the sample case, wherein the type of question includes the first concept, wherein the concept matching score is based a degree of relevancy for the first concept and the type of question.

5. The computer program product of claim 1, further comprising:

computer-readable program code configured to generate, based on the first concept score, a response to the first case.

6. The computer program product of claim 1, wherein the concept score is based on at least one of:

(i) the presence of the first concept in both the first case and the candidate answer, and (ii) the presence of a relationship between the first concept in the first case and a second concept in the in the candidate answer.

7. The computer program product of claim 1, further comprising:

computer-readable program code configured to detect the first concept in the case.

8. The computer program product of claim 1, wherein the first concept is stored in an ontology comprising a plurality of concepts and concept relationships.

9. A system, comprising:

one or more computer processors; and

a memory containing a program, which, when executed by the one or more computer processors, performs an operation for a deep question answering system, the operation comprising;

computing a concept score for a first concept in a first case received by the deep question answering system, wherein the concept score is based on a machine learning concept model for the first concept;

excluding the first concept from consideration when analyzing a candidate answer and an item of supporting evidence to generate a response to the first case when the concept score does not exceed a predefined concept minimum weight threshold; and

increasing a weight applied to the first concept when analyzing the candidate answer and the item of supporting evidence to generate the response to the first case when the concept score exceeds a predefined maximum weight threshold.

10. The system of claim 9, the operation further comprising:

computing a plurality of concept matching scores for a plurality of concepts based on a respective weight factor established by the machine learning concept model; and

aggregating the plurality of computed concept matching scores into a single relevancy score, wherein the single relevancy score is used to generate a response to the first case.

11. The system of claim 9, the operation further comprising training the deep question answering system to generate the machine learning concept model, wherein training the deep question answering system comprises:

providing the deep question answering system with a sample case and a sample answer;

identifying the first concept in the sample case;

computing a concept matching score for the first concept in the sample case based on the sample answer; and

including the concept matching score in the machine learning concept model.

12. The system of claim 11, wherein producing the machine learning concept model further comprises identifying a type of question included in the sample case, wherein the type of question includes the first concept, wherein the concept matching score is based a degree of relevancy for the first concept and the type of question.

13. The system of claim 9, the operation further comprising generating, based on the first concept score, a response to the first case.

14. The system of claim 9, wherein the concept score is based on at least one of:

(i) the presence of the first concept in both the first case and the candidate answer, and (ii) the presence of a relationship between the first concept in the first case and a second concept in the in the candidate answer.

15. The system of claim 9, the operation further comprising detecting the first concept in the case.

16. The system of claim 9, wherein the first concept is stored in an ontology comprising a plurality of concepts and concept relationships.