Avoiding supporting evidence processing when evidence scoring does not affect final ranking of a candidate answer
First Claim
Patent Images
1. A method to provide selective supporting evidence in a question answering (QA) system, comprising:
- initiating processing of a first question using an execution pipeline of the QA system, wherein the execution pipeline comprises (i) a first stage configured to determine candidate answers to the first question without consideration of any items of supporting evidence and (ii) a second stage, downstream from the first stage, configured to process items of supporting evidence relating to the candidate answers in order to generate updated confidence scores;
upon determining a first candidate answer for a first question during the first stage of the execution pipeline and prior to executing the second stage of the execution pipeline;
generating, by the QA system, using a first machine learning (ML) model, a first confidence score value for the first candidate answer, wherein the first confidence score value reflects a degree to which the first candidate answer is a correct response to the first question, wherein the first ML model does not consider supporting evidence features for the first candidate answer;
generating, by the QA system, using a second ML model, a measure of expected change to the first confidence score value based at least in part on supporting evidence features for the first candidate answer; and
upon determining that the measure of expected change does not exceed a first threshold, returning, by the QA system, at least the first candidate answer in response to the first question, without processing the first question using the second stage of the execution pipeline.
1 Assignment
0 Petitions
Accused Products
Abstract
Methods to provide selective supporting evidence processing by applying a first machine learning (ML) model to a first candidate answer to generate a first confidence score that does not consider supporting evidence for the first candidate answer, determining, from a second ML model, an expected contribution of processing supporting evidence for the first candidate answer, and upon determining that the expected contribution does not exceed a specified threshold, skipping supporting evidence processing for the first candidate answer.
-
Citations
12 Claims
-
1. A method to provide selective supporting evidence in a question answering (QA) system, comprising:
-
initiating processing of a first question using an execution pipeline of the QA system, wherein the execution pipeline comprises (i) a first stage configured to determine candidate answers to the first question without consideration of any items of supporting evidence and (ii) a second stage, downstream from the first stage, configured to process items of supporting evidence relating to the candidate answers in order to generate updated confidence scores; upon determining a first candidate answer for a first question during the first stage of the execution pipeline and prior to executing the second stage of the execution pipeline; generating, by the QA system, using a first machine learning (ML) model, a first confidence score value for the first candidate answer, wherein the first confidence score value reflects a degree to which the first candidate answer is a correct response to the first question, wherein the first ML model does not consider supporting evidence features for the first candidate answer; generating, by the QA system, using a second ML model, a measure of expected change to the first confidence score value based at least in part on supporting evidence features for the first candidate answer; and upon determining that the measure of expected change does not exceed a first threshold, returning, by the QA system, at least the first candidate answer in response to the first question, without processing the first question using the second stage of the execution pipeline. - View Dependent Claims (2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12)
-
Specification