Concept noise reduction in deep question answering systems

• US 9,087,304 B2
• Filed: 12/05/2012
• Issued: 07/21/2015
• Est. Priority Date: 11/08/2012
• Status: Active Grant

• Current Assignee: International Business Machines Corporation
• Original Assignee: International Business Machines Corporation
• Inventors: Petri, John E., Stevens, Richard J.
• Primary Examiner(s): Chaki, Kakali
• Assistant Examiner(s): Pellett, Daniel
• Application Number:
  US13/705,345
• Publication Number:
  US 20140127647A1
• Time in Patent Office: 958 Days
• Field of Search: None
• US Class Current: 1/1
• CPC Class Codes: G06N 20/00 Machine learning
• CPC Class Codes: G09B 19/0053 Computers, e.g. programming

Abstract

Method, computer program product, and system to perform an operation for a deep question answering system. The operation begins by computing a concept score for a first concept in a first case received by the deep question answering system, the concept score being based on a machine learning concept model for the first concept. The operation then excludes the first concept from consideration when analyzing a candidate answer and an item of supporting evidence to generate a response to the first case upon determining that the concept score does not exceed a predefined concept minimum weight threshold. The operation then increases a weight applied to the first concept when analyzing the candidate answer and the item of supporting evidence to generate the response to the first case when the concept score exceeds a predefined maximum weight threshold.

Claims

1. A method, comprising:

computing a concept score for a first concept in a first case, wherein the concept score is based on whether the first concept is present in at least one of a candidate answer and an item of supporting evidence;

adjusting the concept score based on a coefficient for the first concept in a machine learning concept model for the first concept, wherein the coefficient reflects a relevance of the first concept in generating a correct response to the first case;

excluding the first concept from consideration when analyzing the candidate answer and the item of supporting evidence to generate a response to the first case upon determining at least one of;

(i) that the adjusted concept score does not exceed a predefined minimum weight threshold and (ii) that the coefficient does not exceed a predefined noise threshold; and

considering the first concept when analyzing the candidate answer and the item of supporting evidence to generate the response to the first case upon determining at least one of the concept score and the coefficient exceeds a predefined maximum weight threshold.

2. The method of claim 1, further comprising:

computing a plurality of concept matching scores for a plurality of concepts in the candidate answer based on a respective weight factor specified by the machine learning concept model; and

aggregating the plurality of computed concept matching scores into a single relevancy score, wherein the single relevancy score is used to generate a response to the first case.

3. The method of claim 1, further comprising training the deep question answering system to generate the machine learning concept model, wherein training the deep question answering system comprises:

providing the deep question answering system with a sample case and a sample answer;

identifying the first concept in the sample case;

computing the concept matching score for the first concept in the sample case based on the sample answer; and

including the concept matching score in the machine learning concept model.

4. The method of claim 3, wherein producing the machine learning concept model further comprises identifying a type of question included in the sample case, wherein the type of question includes the first concept, wherein the concept matching score is based a degree of relevancy for the first concept and the type of question.

5. The method of claim 1, further comprising generating, based on the first concept score, a response to the first case.

6. The method of claim 1, wherein the concept score is further computed based on the presence of a relationship between the first concept in the first case and a second concept in the in the candidate answer.

7. The method of claim 1, further comprising detecting the first concept in the case.

8. The method of claim 1, wherein the first concept is stored in an ontology comprising a plurality of concepts and concept relationships.