Confidence threshold tuning
First Claim
Patent Images
1. A computer-implemented method comprising:
- identifying possible outcomes for at least one step in a speech application and assigning a cost for each possible outcome in terms of expected dialog turns;
accessing recognition results and corresponding confidence scores for the at least one step in the speech application;
associating each recognition result and corresponding confidence score with one of the possible outcomes based on at least one application parameter;
calculating a cost for the at least one step based on the association of each recognition result and corresponding confidence score with one of the possible outcomes and the cost for each possible outcome; and
adjusting the at least one parameter of the speech application to reduce the cost for the at least one step.
3 Assignments
0 Petitions
Accused Products
Abstract
An expected dialog-turn (ED) value is estimated for evaluating a speech application. Parameters such as a confidence threshold setting can be adjusted based on the expected dialog-turn value. In a particular example, recognition results and corresponding confidence scores are used to estimate the expected dialog-turn value. The recognition results can be associated with a possible outcome for the speech application and a cost for the possible outcome can be used to estimate the expected dialog-turn value.
-
Citations
20 Claims
-
1. A computer-implemented method comprising:
-
identifying possible outcomes for at least one step in a speech application and assigning a cost for each possible outcome in terms of expected dialog turns;
accessing recognition results and corresponding confidence scores for the at least one step in the speech application;
associating each recognition result and corresponding confidence score with one of the possible outcomes based on at least one application parameter;
calculating a cost for the at least one step based on the association of each recognition result and corresponding confidence score with one of the possible outcomes and the cost for each possible outcome; and
adjusting the at least one parameter of the speech application to reduce the cost for the at least one step. - View Dependent Claims (2, 3, 4, 5, 6, 7)
-
-
8. A computer-implemented method, comprising:
-
accessing recognition results and corresponding confidence scores for a speech application;
estimating a first expected dialog-turn value of the speech application for a first confidence threshold based on the recognition results and corresponding confidence scores;
estimating a second expected dialog-turn value of the speech application for a second confidence threshold based on the recognition results and corresponding confidence scores; and
comparing the first expected dialog turn value and the second expected dialog turn value. - View Dependent Claims (9, 10, 11, 12, 13)
-
-
14. A computer-implemented method comprising:
-
identifying possible outcomes for a speech application and assigning a cost to each possible outcome;
accessing recognition results and corresponding confidence scores for the speech application;
associating each recognition result and corresponding confidence score with one of the possible outcomes for a plurality of confidence thresholds; and
estimating an expected dialog-turn value for each of the plurality of confidence thresholds based on how many recognition results are associated with each possible outcome and the cost of each possible outcome. - View Dependent Claims (15, 16, 17, 18, 19, 20)
-
Specification