Scoring attributes in deep question answering systems based on algorithmic source code influences
First Claim
Patent Images
1. A system, comprising:
- one or more computer processors; and
a memory containing a program which when executed by the processors performs an operation comprising;
identifying a first variable in a source code of a question answering (QA) system;
upon determining that a weight applied to a value of the first variable by a first rule in the source code increases a confidence score for candidate answers generated by the QA system beyond a threshold;
computing an influence score for the first variable based on;
(i) the weight applied to the value of the first variable by the first rule in the source code, (ii) a number of rules specifying weights applied to values of the first attribute, (iii) a location of the first attribute in each rule, (iv) a number of times the first variable is used in each rule, (v) a type of operation applied to the value of the first variable by each respective rule, and (vi) an identified phase of a processing pipeline of the QA system in which each respective rule is applied;
computing an importance score for the first variable based at least in part on the computed influence score; and
upon determining that the importance score exceeds a predefined threshold, storing an indication that the first variable is an important variable relative to other variables specified in the source code;
receiving, by the QA system, a case that does not specify a value for the first variable; and
refraining, by the QA system, from processing the case.
1 Assignment
0 Petitions
Accused Products
Abstract
Systems and computer program products to perform an operation comprising: identifying a first attribute of a source code in a deep question answering system, computing an influence score for the first attribute based on a rule in the source code used to compute a confidence score for each of a plurality of candidate answers generated by the deep question answering system, computing an importance score for the first attribute based at least in part on the computed influence score, and upon determining that the importance score exceeds a predefined threshold, storing an indication that the first attribute is an important attribute relative to other attributes specified in the source code.
-
Citations
15 Claims
-
1. A system, comprising:
-
one or more computer processors; and a memory containing a program which when executed by the processors performs an operation comprising; identifying a first variable in a source code of a question answering (QA) system; upon determining that a weight applied to a value of the first variable by a first rule in the source code increases a confidence score for candidate answers generated by the QA system beyond a threshold; computing an influence score for the first variable based on;
(i) the weight applied to the value of the first variable by the first rule in the source code, (ii) a number of rules specifying weights applied to values of the first attribute, (iii) a location of the first attribute in each rule, (iv) a number of times the first variable is used in each rule, (v) a type of operation applied to the value of the first variable by each respective rule, and (vi) an identified phase of a processing pipeline of the QA system in which each respective rule is applied;computing an importance score for the first variable based at least in part on the computed influence score; and upon determining that the importance score exceeds a predefined threshold, storing an indication that the first variable is an important variable relative to other variables specified in the source code; receiving, by the QA system, a case that does not specify a value for the first variable; and refraining, by the QA system, from processing the case. - View Dependent Claims (2, 3, 4, 5, 6, 7, 8)
-
-
9. A computer program product, comprising:
a non-transitory computer-readable storage medium having computer-readable program code embodied therewith, the computer-readable program code executable by one or more computer processors to perform an operation comprising; identifying a first variable in a source code of a question answering (QA) system; upon determining that a weight applied to a value of the first variable by a first rule in the source code increases a confidence score for candidate answers generated by the QA system beyond a threshold; computing an influence score for the first variable based on;
(i) the weight applied to the value of the first variable by the first rule in the source code, (ii) a number of rules specifying weights applied to values of the first attribute, (iii) a location of the first attribute in each rule, (iv) a number of times the first variable is used in each rule, (v) a type of operation applied to the value of the first variable by each respective rule, and (vi) an identified phase of a processing pipeline of the QA system in which each respective rule is applied;computing an importance score for the first variable based at least in part on the computed influence score; and upon determining that the importance score exceeds a predefined threshold, storing an indication that the first variable is an important variable relative to other variables specified in the source code; receiving, by the QA system, a case that does not specify a value for the first variable; and refraining, by the QA system, from processing the case. - View Dependent Claims (10, 11, 12, 13, 14, 15)
Specification