Methods and systems for identifying errors in a speech recognition system

• US 8,868,421 B2
• Filed: 10/11/2010
• Issued: 10/21/2014
• Est. Priority Date: 02/04/2005
• Status: Active Grant

• Current Assignee: Vocollect, Inc. (Honeywell International Inc.)
• Original Assignee: Vocollect, Inc. (Honeywell International Inc.)
• Inventors: Braho, Keith P., Pike, Jeffrey P., Pike, Lori A.
• Primary Examiner(s): MCFADDEN, SUSAN IRIS
• Application Number:
  US12/901,864
• Publication Number:
  US 20110029312A1
• Time in Patent Office: 1,471 Days
• Field of Search: 704/251
• US Class Current: 704/251
• CPC Class Codes: G10L 15/01 Assessment or evaluation of speech recognition systems
• CPC Class Codes: G10L 15/065 Adaptation

Abstract

A method for model adaptation for a speech recognition system includes determining an error rate, corresponding to either recognition of instances of a word or recognition of instances of various words, without using a transcript of words input to the system. The method may further include adjusting an adaptation, of the model for the word or various models for the various words, based on the error rate. Apparatus are disclosed for identifying possible errors made by a speech recognition system without using a transcript of words input to the system. An apparatus for model adaptation for a speech recognition system includes a processor adapted to estimate an error rate, corresponding to either recognition of instances of a word or recognition of instances of various words, without using a transcript of words input to the system. The apparatus may further include a controller adapted to adjust an adaptation of the model for the word or various models for the various words, based on the error rate.

Claims

1. A method for identifying a possible error made by a speech recognition system comprising:

with an apparatus using at least one hardware-implemented processor, identifying an instance where the system rejects a first hypothesis of a first utterance, followed by the system accepting a second hypothesis of a second utterance, wherein the first and second hypotheses substantially match word-for-word.

2. The method of claim 1, wherein the first and second hypotheses substantially match word-for-word by matching word-for-word, except that one of the first and second hypotheses includes at least one additional recognized model that is negligible for purposes of identifying the possible error.

3. The method of claim 1, wherein a count of occurrences of the possible error is used in adjusting an adaptation of a model for a word associated with the possible error.

4. The method of claim 1, wherein the system rejects the first hypothesis due to a confidence factor of the first hypothesis not exceeding an acceptance threshold.

5. The method of claim 1, wherein the system rejects at least one word in the first hypothesis.

6. The method of claim 1, wherein the first and second utterances are spoken consecutively, proximately or within a predetermined amount of time.

7. A method for identifying a possible error made by a speech recognition system comprising:

with an apparatus using at least one hardware-implemented processor, identifying when the system generates first and second hypotheses of two utterances and the system accepts the second hypothesis, wherein the two hypotheses do not match word-for-word, but the hypotheses mostly match word-for-word.

8. The method of claim 7, wherein the hypotheses mostly match word-for-word by matching word-for-word except for a predetermined number of words.

9. The method of claim 7, wherein a count of occurrences of the possible error is used in adjusting an adaptation a model for a word associated with the possible error.

10. The method of claim 7, wherein the two utterances are spoken consecutively, proximately, or within a predetermined amount of time.

11. The method of claim 7, wherein there are no intervening utterances that indicate that the first of the two utterances was correctly recognized by the system.

12. The method of claim 7, wherein the two hypotheses differ in that a word in the second hypothesis is substituted with a word in the first hypothesis.

13. The method of claim 7, wherein the hypotheses differ in that a word in the second hypothesis is substituted with garbage in the first hypothesis.

14. A method for identifying a possible error made by a speech recognition system, comprising:

with an apparatus using at least one hardware-implemented processor, identifying when a hypothesis generated by the system does not match an expected response word-for-word, but the hypothesis mostly matches the expected response word-for-word.

15. The method of claim 14, wherein a count of occurrences of the possible errors is used in adjusting adaptation of a model for a word associated with the possible errors.

16. The method of claim 14, wherein the hypothesis mostly matches the expected response word-for-word by matching word-for-word except for one word.

17. An apparatus for identifying a possible error made by a speech recognition system comprising:

a processor that is operable to identify an instance where the system rejects a first hypothesis of a first utterance, followed by the system accepting a second hypothesis of a second utterance, wherein the first and second hypotheses substantially match word-for-word.

18. The apparatus of claim 17, wherein the first and second hypotheses substantially match word-for-word by matching word-for-word, except that one of the first and second hypotheses includes at least one additional recognized model that is negligible for purposes of identifying the possible error.

19. The apparatus of claim 17, wherein a count of occurrences of the possible error is used in adjusting an adaptation of a model for a word associated with the possible error.

20. The apparatus of claim 17, wherein the system rejects the first hypothesis due to a confidence factor of the first hypothesis not exceeding an acceptance threshold.

21. The apparatus of claim 17, wherein the system rejects at least one word in the first hypothesis.

22. The apparatus of claim 17, wherein the first and second utterances are spoken consecutively, proximately or within a predetermined amount of time.

23. An apparatus for identifying a possible error made by a speech recognition system comprising:

a processor that is operable to identify when the system generates first and second hypotheses of two utterances and the system accepts the second hypothesis, wherein the two hypotheses do not match word-for-word, but the hypotheses mostly match word-for-word.

24. The apparatus of claim 23, wherein the hypotheses mostly match word-for-word by matching word-for-word except for a predetermined number of words.

25. The apparatus of claim 23, wherein a count of occurrences of the possible error is used in adjusting an adaptation a model for a word associated with the possible error.

26. The apparatus of claim 23, wherein the two utterances are spoken consecutively, proximately, or within a predetermined amount of time.

27. The apparatus of claim 23, wherein there are no intervening utterances that indicate that the first of the two utterances was correctly recognized by the system.

28. The apparatus of claim 23, wherein the two hypotheses differ in that a word in the second hypothesis is substituted with a word in the first hypothesis.

29. The apparatus of claim 23, wherein the hypothesis differ in that a word in the second hypothesis is substituted with garbage in the first hypothesis.

30. An apparatus for identifying a possible error rate made by a speech recognition system, comprising:

a processor that is operable to identify when a hypothesis generated by the system does not match an expected response word-for-word, but the hypothesis mostly matches the expected response word-for-word.

31. The apparatus of claim 30, wherein a count of occurrences of the possible error is used in adjusting an adaptation of a model for a word associated with the possible error.

32. The apparatus of claim 30, wherein the hypothesis mostly matches the expected response word-for-word by matching word-for-word except for a predetermined number of words.