CONCEPT NOISE REDUCTION IN DEEP QUESTION ANSWERING SYSTEMS

• US 20140127647A1
• Filed: 12/05/2012
• Published: 05/08/2014
• Est. Priority Date: 11/08/2012
• Status: Active Grant

• Current Assignee: International Business Machines Corporation
• Original Assignee: International Business Machines Corporation
• Inventors: Stevens, Richard J., Petri, John A.
• Granted Patent: US 9,087,304 B2
• Time in Patent Office: Days
• US Class Current: 434/118
• CPC Class Codes: G06N 20/00 Machine learning
• CPC Class Codes: G09B 19/0053 Computers, e.g. programming

Abstract

Method, computer program product, and system to perform an operation for a deep question answering system. The operation begins by computing a concept score for a first concept in a first case received by the deep question answering system, the concept score being based on a machine learning concept model for the first concept. The operation then excludes the first concept from consideration when analyzing a candidate answer and an item of supporting evidence to generate a response to the first case upon determining that the concept score does not exceed a predefined concept minimum weight threshold. The operation then increases a weight applied to the first concept when analyzing the candidate answer and the item of supporting evidence to generate the response to the first case when the concept score exceeds a predefined maximum weight threshold.

Claims

1. A method for a deep question answering system, the method comprising:

computing a concept score for a first concept in a first case received by the deep question answering system, wherein the concept score is based on a machine learning concept model for the first concept;

excluding the first concept from consideration when analyzing a candidate answer and an item of supporting evidence to generate a response to the first case when the concept score does not exceed a predefined concept minimum weight threshold; and

increasing, by operation of one or more computer processors, a weight applied to the first concept when analyzing the candidate answer and the item of supporting evidence to generate the response to the first case when the concept score exceeds a predefined maximum weight threshold.

2. The method of claim 1, further comprising:

computing a plurality of concept matching scores for a plurality of concepts based on a respective weight factor established by the machine learning concept model; and

aggregating the plurality of computed concept matching scores into a single relevancy score, wherein the single relevancy score is used to generate a response to the first case.

3. The method of claim 1, further comprising training the deep question answering system to generate the machine learning concept model, wherein training the deep question answering system comprises:

providing the deep question answering system with a sample case and a sample answer;

identifying the first concept in the sample case;

computing a concept matching score for the first concept in the sample case based on the sample answer; and

including the concept matching score in the machine learning concept model.

4. The method of claim 3, wherein producing the machine learning concept model further comprises identifying a type of question included in the sample case, wherein the type of question includes the first concept, wherein the concept matching score is based a degree of relevancy for the first concept and the type of question.

5. The method of claim 1, further comprising generating, based on the first concept score, a response to the first case.

6. The method of claim 1, wherein the concept score is based on at least one of:

(i) the presence of the first concept in both the first case and the candidate answer, and (ii) the presence of a relationship between the first concept in the first case and a second concept in the in the candidate answer.

7. The method of claim 1, further comprising detecting the first concept in the case.

8. The method of claim 1, wherein the first concept is stored in an ontology comprising a plurality of concepts and concept relationships.