METHOD AND APPARATUS OF CONFIDENCE MEASURE CALCULATION

• US 20140195238A1
• Filed: 03/01/2012
• Published: 07/10/2014
• Est. Priority Date: 07/01/2011
• Status: Active Grant

• Current Assignee: NEC Corporation, University of Washington
• Original Assignee: NEC Corporation, University of Washington
• Inventors: Terao, Makoto, Ostendorf, Mari
• Granted Patent: US 9,336,769 B2
• Time in Patent Office: Days
• US Class Current: 704/251
• CPC Class Codes: G10L 15/02 Feature extraction for speech recognition; Selection of recognition unit
• CPC Class Codes: G10L 15/063 Training
• CPC Class Codes: G10L 15/10 using distance or distortion measures between unknown speech and reference templates

Abstract

An apparatus that calculates a confidence measure of a target word string specified in a recognition result includes: an alternative candidate generator which generates an alternative candidate word string in the position of the target word string; a classifier training unit which trains a classifier which is configured to discriminate between the target word string and the alternative candidate word string; a feature extractor which extracts a feature value representing an adjacent context in the position of the target word string; and a confidence measure calculator which determining whether the true word string in the position of the target word string is the target word string or the alternative candidate word string by using the classifier and the feature value and calculates a confidence measure of the target word string on the basis of the determination result.

Claims

1. A confidence measure calculation method of calculating a confidence measure of a target word string specified in a recognition result, the method comprising:

receiving the recognition result and the target word string;

generating an alternative candidate word string in a position of the target word string;

training a classifier which is configured to discriminate between the target word string and the alternative candidate word string;

extracting a feature value representing an adjacent context in the position of the target word string;

determining whether a true word string in the position of the target word string is the target word string or the alternative candidate word string by using the classifier and the feature value; and

calculating a confidence measure of the target word string on the basis of the determination result.

2. The method according to claim 1, wherein a plurality of the alternative candidate word strings are generated, the classifier is configured to discriminate between the target word string and any of the plurality of alternative candidate word strings, and whether the true word string in the position of the target word string is the target word string or any of the plurality of alternative candidate word strings is determined by using the classifier.

3. The method according to claim 1, wherein the classifier is configured to output a posterior probability of classification result, and the posterior probability that the target word string is correct is used as the confidence measure of the target word string.

4. The method according to claim 1, wherein, when the true word string in the position of the target word string is classified as the target word string, a first constant is used as the confidence measure of the target word string, whereas, when the true word string in the position of the target word string is classified as the alternative candidate word string, a second constant smaller than the first constant is used as the confidence measure of the target word string.

5. The method according to claim 1, wherein a hypothesis which is outputted by a recognition system in the position of the target word string and is other than the target word string is used as the alternative candidate word string.

6. The method according to claim 1, wherein a word string having pronunciation which is similar to pronunciation of the target word string is generated as the alternative candidate word string by referring to a pronunciation dictionary.

7. The apparatus according to claim 1, wherein a word string that a recognition system tends to confuse with the target word string is generated as the alternative candidate word string by referring to an inter-word confusion matrix.

8. The apparatus according to claim 1, wherein the feature value representing the adjacent context in the position of the target word string is calculated on the basis of frequencies of words appearing within a certain distance from the position of the target word string or frequencies of attributes of the words.

9. The method according to claim 1, further comprising:

judging the target word string to be a recognition error when the calculated confidence measure of the target word string is smaller than or equal to a predetermined threshold.

10. The method according to claim 1, further comprising:

judging the target word string to be a recognition error when a posterior probability of the target word string outputted from a recognition system is smaller than or equal to a first threshold and the calculated confidence measure of the target word string is smaller than or equal to a second threshold.

11. The method according to claim 1, further comprising:

carrying out the generating, the training, the extracting and the determining to obtain a confidence measure of the target word string and a confidence measure of the alternative candidate word string; and

outputs a word string having a largest confidence measure.

12. A confidence measure calculation apparatus that calculates a confidence measure of a target word string specified in a recognition result, the apparatus comprising:

a generator generating an alternative candidate word string in a position of the target word string;

a training unit training a classifier which is configured to discriminate between the target word string and the alternative candidate word string;

an extractor extracting a feature value representing an adjacent context in the position of the target word string; and

a calculator determining whether a true word string in the position of the target word string is the target word string or the alternative candidate word string by using the classifier and the feature value, and calculating a confidence measure of the target word string on the basis of the determination result.

13. The apparatus according to claim 12, wherein the generator generates a plurality of the alternative candidate word strings, the classifier is configured to discriminate between the target word string and any of the plurality of alternative candidate word strings, and, upon determination the calculator determines whether the true word string in the position of the target word string is the target word string or any of the plurality of alternative candidate word strings.

14. The apparatus according to claim 12, wherein the classifier is configured to output a posterior probability of classification result, and the calculator uses the posterior probability that the target word string is correct as the confidence measure of the target word string.

15. The apparatus according to claim 12, wherein the calculator uses a first constant as the confidence measure of the target word string when the true word string in the position of the target word string is classified as the target word string, whereas the calculator uses a second constant smaller than the first constant as the confidence measure of the target word string when the true word string in the position of the target word string is classified as the alternative candidate word string.

16. The apparatus according to claim 12, wherein the generator uses, as the alternative candidate word string, a hypothesis which is outputted by a recognition system in the position of the target word string and is other than the target word string.

17. The apparatus according to claim 12, wherein the generator refers to a pronunciation dictionary and generates, as the alternative candidate word string, a word string having pronunciation of which is similar to pronunciation of the target word string.

18. The apparatus according to claim 12, wherein the generator refers to an inter-word confusion matrix to generate, as the alternative candidate word string, a word string that a recognition system tends to confuse with the target word string.

19. The apparatus according to claim 12, wherein the feature value representing the adjacent context in the position of the target word string is calculated on the basis of frequencies of words appearing within a certain distance from the position of the target word string or frequencies of attributes of the words.

20. The apparatus according to claim 12, wherein, when the calculated confidence measure of the target word string is smaller than or equal to a predetermined threshold, the target word string is judged to be a recognition error.

21. The apparatus according to claim 12, wherein, when a posterior probability of the target word string outputted from a recognition system is smaller than or equal to a first threshold and the calculated confidence measure of the target word string is smaller than or equal to a second threshold, the target word string is judged to be a recognition error.

22. The apparatus according to claim 12, wherein the apparatus calculates a confidence measure of the target word string and a confidence measure of the alternative candidate word string, and outputs a word string having a largest confidence measure.

23. A confidence measure calculation apparatus that calculates a confidence measure of a target word string specified in a recognition result, the apparatus comprising:

means for generating an alternative candidate word string in a position of the target word string;

means for training a classifier which is configured to discriminate between the target word string and the alternative candidate word string;

means for extracting a feature value representing an adjacent context in the position of the target word string; and

means for determining whether a true word string in the position of the target word string is the target word string or the alternative candidate word string by using the classifier and the feature value to calculate a confidence measure of the target word string on the basis of the determination result.

24. A non-transitory computer-readable recoding medium that stores a program which causes a computer to execute the processes of:

receiving a recognition result and a target word string which is specified in the recognition result;

generating an alternative candidate word string in a position of the target word string;

training a classifier which is configured to discriminate between the target word string and the alternative candidate word string;

extracting a feature value representing an adjacent context in the position of the target word string;

determining whether a true word string in the position of the target word string is the target word string or the alternative candidate word string by using the classifier and the feature value; and

calculating a confidence measure of the target word string on the basis of the determination result.

25. The recording medium according to claim 24, wherein the program causes the computer to generate a plurality of the alternative candidate word strings in the generating process, to train a classifier which is configured to discriminate between the target word string and any of the plurality of alternative candidate word strings in the training process, and determine whether the true word string in the position of the target word string is the target word string or any of the plurality of alternative candidate word strings by using the classifier in the determination process.

26. The recording medium according to claim 24, wherein the classifier is configured to output a posterior probability of classification result, and the program causes the computer to use the posterior probability that the target word string is correct as the confidence measure of the target word string in the calculation process.

27. The recording medium according to claim 24, wherein the program causes the computer, in the calculation process, to use a first constant as the confidence measure of the target word string when the true word string in the position of the target word string is classified as the target word string, whereas use a second constant smaller than the first constant as the confidence measure of the target word string when the true word string in the position of the target word string is classified as the alternative candidate word string.

28. The recording medium according to claim 24, wherein the program causes the computer to use, as the alternative candidate word string, a hypothesis being outputted by a recognition system in the position of the target word string and being other than the target word string.

29. The recording medium according to claim 24, wherein the program causes the computer to refer to a pronunciation dictionary and generate a word string having pronunciation which is similar to pronunciation of the target word string as the alternative candidate word string, in the generating process.

30. The recording medium according to claim 24, wherein the program causes the computer to refer to an inter-word confusion matrix and generate a word string that a recognition system tends to confuse with the target word string as the alternative candidate word string, in the generating process.

31. The recording medium according to claim 24, wherein the program causes the computer to calculate the feature value representing the adjacent context in the position of the target word string on the basis of frequencies of words appearing within a certain distance from the position of the target word string or frequencies of attributes of the words.

32. The recording medium according to claim 24, wherein the program further causes the computer to execute the process of judging the target word string to be a recognition error when the calculated confidence measure of the target word string is smaller than or equal to a predetermined threshold.

33. The recording medium according to claim 24, wherein the program further causes the computer to execute the process of judging the target word string to be a recognition error when a posterior probability of the target word string outputted from a recognition system is smaller than or equal to a first threshold and the calculated confidence measure of the target word string is smaller than or equal to a second threshold.

34. The recording medium according to claim 24, wherein the program further causes the computer to execute the processes of:

executing the processes of the generating, the training, the extracting and the determining to obtain a confidence measure of the target word string and a confidence measure of the alternative candidate word string, and

outputting a word string having a largest confidence measure.