Methods and systems for identifying errors in a speech recognition system

• US 9,928,829 B2
• Filed: 10/17/2014
• Issued: 03/27/2018
• Est. Priority Date: 02/04/2005
• Status: Active Grant

• Current Assignee: Vocollect, Inc. (Honeywell International Inc.)
• Original Assignee: Vocollect, Inc. (Honeywell International Inc.)
• Inventors: Braho, Keith P., Pike, Jeffrey P., Pike, Lori A.
• Primary Examiner(s): MCFADDEN, SUSAN IRIS
• Application Number:
  US14/517,016
• Publication Number:
  US 20150039309A1
• Time in Patent Office: 1,257 Days
• Field of Search: 704251
• CPC Class Codes: G10L 15/01 Assessment or evaluation of speech recognition systems
• CPC Class Codes: G10L 15/065 Adaptation

Abstract

Methods are disclosed for identifying possible errors made by a speech recognition system without using a transcript of words input to the system. A method for model adaptation for a speech recognition system includes determining an error rate, corresponding to either recognition of instances of a word or recognition of instances of various words, without using a transcript of words input to the system. The method may further include adjusting an adaptation, of the model for the word or various models for the various words, based on the error rate. Apparatus are disclosed for identifying possible errors made by a speech recognition system without using a transcript of words input to the system. An apparatus for model adaptation for a speech recognition system includes a processor adapted to estimate an error rate, corresponding to either recognition of instances of a word or recognition of instances of various words, without using a transcript of words input to the system. The apparatus may further include a controller adapted to adjust an adaptation of the model for the word or various models for the various words, based on the error rate.

Claims

1. A method for identifying a possible error made by a speech recognition system comprising:

with an apparatus using at least one hardware-implemented processor, identifying when a hypothesis generated by the speech recognition system does not match an expected response word-for-word, but the hypothesis mostly matches the expected response word-for-word;

with the apparatus using the at least one hardware-implemented processor, identifying an instance where the speech recognition system rejects a first hypothesis of a first utterance received from a user, followed by the speech recognition system accepting a second hypothesis of a second utterance received from the user, wherein the first and second hypotheses substantially match word-for-word;

with the apparatus using the at least one hardware-implemented processor, incrementing a count of an occurrence of a possible error made by the speech recognition system; and

with the apparatus using the at least one hardware-implemented processor, adjusting an adaptation of a model used by the speech recognition system for a word associated with the possible error when the count exceeds an acceptance threshold.

2. The method of claim 1, wherein the first hypothesis and the second hypothesis substantially match word-for-word by matching word-for-word, except that one of the first hypothesis and the second hypothesis includes at least one additional recognized model that is negligible for purposes of identifying the possible error.

3. The method of claim 1, wherein the system rejects the first hypothesis due to a confidence factor of the first hypothesis not exceeding an acceptance threshold.

4. The method of claim 1, wherein the system rejects at least one word in the first hypothesis.

5. The method of claim 1, wherein the first and second utterances are spoken consecutively, proximately or within a predetermined amount of time.

6. The method of claim 1, wherein the hypothesis mostly matches the expected response word-for-word by matching word-for-word except for one word.

7. A method for identifying a possible error made by a speech recognition system comprising:

with an apparatus using at least one hardware-implemented processor, identifying when a hypothesis generated by the speech recognition system does not match an expected response word-for-word, but the hypothesis mostly matches the expected response word-for-word;

with the apparatus using the at least one hardware-implemented processor, identifying when the speech recognition system generates a first hypothesis and a second hypothesis of two utterances received from a user and the speech recognition system accepts the second hypothesis, wherein the first hypothesis and the second hypothesis do not match word-for-word, but the first hypothesis and the second hypothesis mostly match word-for-word;

with the apparatus using the at least one hardware-implemented processor, incrementing a count of an occurrence of a possible error made by the speech recognition system; and

with the apparatus using the at least one hardware-implemented processor, adjusting an adaptation of a model used by the speech recognition system for a word associated with the possible error when the count exceeds an acceptance threshold.

8. The method of claim 7, wherein the first hypothesis and the second hypothesis mostly match word-for-word by matching word-for-word except for a predetermined number of words.

9. The method of claim 7, wherein the two utterances are spoken consecutively, proximately, or within a predetermined amount of time.

10. The method of claim 7, wherein there are no intervening utterances that indicate that the first of the two utterances was correctly recognized by the system.

11. The method of claim 7, wherein the first hypothesis and the second hypothesis differ in that a word in the second hypothesis is substituted with a word in the first hypothesis.

12. The method of claim 7, wherein the first hypothesis and the second hypothesis differ in that a word in the second hypothesis is substituted with garbage in the first hypothesis.

13. The method of claim 7, wherein the hypothesis mostly matches the expected response word-for-word by matching word-for-word except for one word.

14. An apparatus for identifying a possible error made by a speech recognition system comprising:

a processor that is operable to;

identify when a hypothesis generated by the speech recognition system does not match an expected response word-for-word, but the hypothesis mostly matches the expected response word-for-word;

identify an instance where the speech recognition system rejects a first hypothesis of a first utterance received from a user, followed by the speech recognition system accepting a second hypothesis of a second utterance received from the user, wherein the first hypothesis and the second hypothesis substantially match word-for-word;

incrementing a count of an occurrence of a possible error made by the speech recognition system; and

adjusting an adaptation of a model used by the speech recognition system for a word associated with the possible error when the count exceeds an acceptance threshold.

15. The apparatus of claim 14, wherein:

the first hypothesis and the second hypothesis substantially match word-for-word by matching word-for-word, except that one of the first hypothesis and the second hypothesis includes at least one additional recognized model that is negligible for purposes of identifying the possible error; and

the system rejects the first hypothesis due to a confidence factor of the first hypothesis not exceeding an acceptance threshold.

16. The apparatus of claim 14, wherein the first and second utterances are spoken consecutively, proximately or within a predetermined amount of time.

17. The apparatus of claim 14, wherein the hypothesis mostly matches the expected response word-for-word by matching word-for-word except for a predetermined number of words.