Methods and systems for assessing and improving the performance of a speech recognition system

• US 8,374,870 B2
• Filed: 03/09/2011
• Issued: 02/12/2013
• Est. Priority Date: 02/04/2005
• Status: Active Grant

• Current Assignee: Vocollect, Inc. (Honeywell International Inc.)
• Original Assignee: Vocollect, Inc. (Honeywell International Inc.)
• Inventors: Braho, Keith, Pike, Jeffrey, El-Jaroudi, Amro, Pike, Lori, Laughery, Michael
• Primary Examiner(s): MCFADDEN, SUSAN IRIS
• Application Number:
  US13/044,028
• Publication Number:
  US 20110161082A1
• Time in Patent Office: 706 Days
• Field of Search: 704/251
• US Class Current: 704/251
• CPC Class Codes: G10L 15/01 Assessment or evaluation of speech recognition systems
• CPC Class Codes: G10L 15/06 Creation of reference templates; Training of speech recognition systems, e.g. adaptation to the characteristics of the speaker's voice G10L15/14 takes precedence

Abstract

A method for assessing a performance of a speech recognition system may include determining a grade, corresponding to either recognition of instances of a word or recognition of instances of various words among a set of words, wherein the grade indicates a level of the performance of the system and the grade is based on a recognition rate and at least one recognition factor. An apparatus for assessing a performance of a speech recognition system may include a processor that determines a grade, corresponding to either recognition of instances of a word or recognition of instances of various words among a set of words, wherein the grade indicates a level of the performance of the system and wherein the grade is based on a recognition rate and at least one recognition factor.

Claims

1. A method for model adaptation for a speech recognition system comprising:

using at least one processor, determining a performance assessment of the system, corresponding to either recognition of instances of a word or recognition of instances of various words among a set of words, the performance assessment of the system based on a plurality of recognition observations by the system; and

adjusting an adaptation of a model for the word or various models for the various words, by making a comparison of the determined performance assessment of the system to a performance assessment threshold and making a determination of whether to adapt the model or various models or whether to withhold adapting the model or various models, based on the comparison.

2. The method of claim 1, wherein the performance assessment is a grade.

3. The method of claim 1, wherein the performance assessment is at least one of an accuracy rate or an error rate.

4. The method of claim 1, wherein the performance assessment threshold is based on at least one of the perplexity of the grammar of the speech recognition system or the confusability of the words in the vocabulary of the speech recognition system.

5. The method of claim 1, wherein the performance assessment threshold is based on a performance assessment of a set of users of like systems.

6. The method of claim 2 wherein the grade is based on a numerical score.

7. The method of claim 2 wherein the grade is based on a recognition rate and at least one recognition factor.

8. The method of claim 1 wherein the performance assessment threshold is at least one of a predetermined value or a value settable by a user.

9. The method of claim 1, wherein the performance assessment threshold is based on a number of words in an utterance input to the speech recognition system.

10. The method of claim 1, wherein the performance assessment threshold is based on environmental factors.

11. An apparatus for model adaptation for a speech recognition system comprising:

a processor configured for determining a performance assessment of the system, corresponding to either recognition of instances of a word or recognition of instances of various words among a set of words, the performance assessment of the system based on a plurality of recognition observations by the system; and

a controller configured for adjusting an adaptation, of a model for the word or various models for the various words, by making a comparison of the determined performance assessment of the system to a performance assessment threshold and making a determination of whether to adapt the model or various models or whether to withhold adapting the model or various models, based on the comparison.

12. The apparatus of claim 11, wherein the adaptation is adjusted based on a performance assessment of a set of users of like systems.

13. The apparatus of claim 11 wherein the performance assessment is a grade.

14. The apparatus of claim 11, wherein the performance assessment is at least one of an accuracy rate or an error rate.

15. The apparatus of claim 11 wherein the performance assessment threshold is at least one of a predetermined value or a value settable by a user.

16. The apparatus of claim 11, wherein the performance assessment threshold is based on a number of words in an utterance input to the speech recognition system.

17. The apparatus of claim 11, wherein the performance assessment threshold is based on environmental factors.

18. The apparatus of claim 11, wherein the performance assessment threshold is based on at least one of the perplexity of the grammar of the speech recognition system or the confusability of the words in the vocabulary of the speech recognition system.