Testing A Grammar Used In Speech Recognition For Reliability In A Plurality Of Operating Environments Having Different Background Noise

• US 20090271189A1
• Filed: 04/24/2008
• Published: 10/29/2009
• Est. Priority Date: 04/24/2008
• Status: Active Grant

• Current Assignee: Cerence Operating Company (Cerence Inc.)
• Original Assignee: International Business Machines SA (International Business Machines Corporation)
• Inventors: Cross, Charles W. JR., Bodin, William K., Agapi, Ciprian, Mirt, Michael H.
• Granted Patent: US 8,082,148 B2
• Time in Patent Office: Days
• US Class Current: 704/233
• CPC Class Codes: G10L 15/01 Assessment or evaluation of speech recognition systems

Abstract

Methods, systems, and products for testing a grammar used in speech recognition for reliability in a plurality of operating environments having different background noise that include: receiving recorded background noise for each of the plurality of operating environments; generating a test speech utterance for recognition by a speech recognition engine using a grammar; mixing the test speech utterance with each recorded background noise, resulting in a plurality of mixed test speech utterances, each mixed test speech utterance having different background noise; performing, for each of the mixed test speech utterances, speech recognition using the grammar and the mixed test speech utterance, resulting in speech recognition results for each of the mixed test speech utterances; and evaluating, for each recorded background noise, speech recognition reliability of the grammar in dependence upon the speech recognition results for the mixed test speech utterance having that recorded background noise.

Claims

1. A computer-implemented method of testing a grammar used in speech recognition for reliability in a plurality of operating environments having different background noise, the method comprising:

receiving recorded background noise for each of the plurality of operating environments;

generating a test speech utterance for recognition by a speech recognition engine using a grammar;

mixing the test speech utterance with each recorded background noise, resulting in a plurality of mixed test speech utterances, each mixed test speech utterance having different background noise;

performing, for each of the mixed test speech utterances, speech recognition using the grammar and the mixed test speech utterance, resulting in speech recognition results for each of the mixed test speech utterances; and

evaluating, for each recorded background noise, speech recognition reliability of the grammar in dependence upon the speech recognition results for the mixed test speech utterance having that recorded background noise.

2. The method of claim 1 wherein:

performing, for each of the mixed test speech utterances, speech recognition using the grammar and the mixed test speech utterance, resulting in a speech recognition results for each of the mixed test speech utterances further comprises repeatedly performing, for one of the mixed test speech utterances, the speech recognition using the grammar and the mixed test speech utterance, resulting in a plurality of speech recognition results for the mixed test speech utterance; and

evaluating, for each recorded background noise, speech recognition reliability of the grammar in dependence upon the speech recognition results for the mixed test speech utterance having that recorded background noise further comprises calculating, in dependence upon the plurality of speech recognition results for one of the mixed test speech utterance, a reliability indicator for the grammar when recognizing speech having the recorded background noise included in that mixed test speech utterance.

3. The method of claim 1 wherein generating a test speech utterance for recognition by a speech recognition engine using a grammar further comprises sampling the test speech utterance from speech of a person.

4. The method of claim 1 wherein generating a test speech utterance for recognition by a speech recognition engine using a grammar further comprises synthesizing the test speech utterance using a text-to-speech engine.

5. The method of claim 1 wherein testing a grammar used in speech recognition for reliability in a plurality of operating environments having different background noise further comprises testing a plurality of grammars for use in speech recognition for a multimodal application, the multimodal application operating on a multimodal device supporting multiple modes of interaction that include a voice mode and one or more non-voice modes, the method further comprising:

identifying, by the multimodal application, a current background noise for a current operating environment in which the multimodal device operates; and

altering, by the multimodal application, flow of execution for the multimodal application in dependence upon the identified current background noise for the current operating environment in which the multimodal application operates.

6. The method of claim 5 wherein altering, by the multimodal application, flow of execution for the multimodal application in dependence upon the identified current background noise for the current operating environment in which the multimodal application operates further comprises:

selecting, by the multimodal application, one of the plurality of grammars tested in dependence upon the current background noise and the evaluation of the speech recognition reliability of the plurality of grammars using the recorded background noises;

receiving, by the multimodal application, a voice utterance from a user; and

performing, by the multimodal application, speech recognition in dependence upon the selected grammar and the voice utterance.

7. A system for testing a grammar used in speech recognition for reliability in a plurality of operating environments having different background noise, the system comprising one or more computer processors and computer memory operatively coupled to the computer processors, the computer memory having disposed within it computer program instructions capable of:

receiving recorded background noise for each of the plurality of operating environments;

generating a test speech utterance for recognition by a speech recognition engine using a grammar;

mixing the test speech utterance with each recorded background noise, resulting in a plurality of mixed test speech utterances, each mixed test speech utterance having different background noise;

performing, for each of the mixed test speech utterances, speech recognition using the grammar and the mixed test speech utterance, resulting in speech recognition results for each of the mixed test speech utterances; and

evaluating, for each recorded background noise, speech recognition reliability of the grammar in dependence upon the speech recognition results for the mixed test speech utterance having that recorded background noise.

8. The system of claim 7 wherein:

performing, for each of the mixed test speech utterances, speech recognition using the grammar and the mixed test speech utterance, resulting in a speech recognition results for each of the mixed test speech utterances further comprises repeatedly performing, for one of the mixed test speech utterances, the speech recognition using the grammar and the mixed test speech utterance, resulting in a plurality of speech recognition results for the mixed test speech utterance; and

evaluating, for each recorded background noise, speech recognition reliability of the grammar in dependence upon the speech recognition results for the mixed test speech utterance having that recorded background noise further comprises calculating, in dependence upon the plurality of speech recognition results for one of the mixed test speech utterance, a reliability indicator for the grammar when recognizing speech having the recorded background noise included in that mixed test speech utterance.

9. The system of claim 7 wherein generating a test speech utterance for recognition by a speech recognition engine using a grammar further comprises sampling the test speech utterance from speech of a person.

10. The system of claim 7 wherein generating a test speech utterance for recognition by a speech recognition engine using a grammar further comprises synthesizing the test speech utterance using a text-to-speech engine.

11. The system of claim 7 wherein testing a grammar used in speech recognition for reliability in a plurality of operating environments having different background noise further comprises testing a plurality of grammars for use in speech recognition for a multimodal application, the multimodal application operating on a multimodal device supporting multiple modes of interaction that include a voice mode and one or more non-voice modes, and wherein the computer memory has disposed within it computer program instructions capable of:

identifying a current background noise for a current operating environment in which the multimodal device operates; and

altering flow of execution for the multimodal application in dependence upon the identified current background noise for the current operating environment in which the multimodal application operates.

12. The system of claim 11 wherein altering flow of execution for the multimodal application in dependence upon the identified current background noise for the current operating environment in which the multimodal application operates further comprises:

selecting one of the plurality of grammars tested in dependence upon the current background noise and the evaluation of the speech recognition reliability of the plurality of grammars using the recorded background noises;

receiving a voice utterance from a user; and

performing speech recognition in dependence upon the selected grammar and the voice utterance.

13. A computer program product for testing a grammar used in speech recognition for reliability in a plurality of operating environments having different background noise, the computer program product disposed upon a computer readable medium, the computer program product comprising computer program instructions capable of:

receiving recorded background noise for each of the plurality of operating environments;

generating a test speech utterance for recognition by a speech recognition engine using a grammar;

mixing the test speech utterance with each recorded background noise, resulting in a plurality of mixed test speech utterances, each mixed test speech utterance having different background noise;

performing, for each of the mixed test speech utterances, speech recognition using the grammar and the mixed test speech utterance, resulting in speech recognition results for each of the mixed test speech utterances; and

evaluating, for each recorded background noise, speech recognition reliability of the grammar in dependence upon the speech recognition results for the mixed test speech utterance having that recorded background noise.

14. The computer program product of claim 13 wherein:

performing, for each of the mixed test speech utterances, speech recognition using the grammar and the mixed test speech utterance, resulting in a speech recognition results for each of the mixed test speech utterances further comprises repeatedly performing, for one of the mixed test speech utterances, the speech recognition using the grammar and the mixed test speech utterance, resulting in a plurality of speech recognition results for the mixed test speech utterance; and

evaluating, for each recorded background noise, speech recognition reliability of the grammar in dependence upon the speech recognition results for the mixed test speech utterance having that recorded background noise further comprises calculating, in dependence upon the plurality of speech recognition results for one of the mixed test speech utterance, a reliability indicator for the grammar when recognizing speech having the recorded background noise included in that mixed test speech utterance.

15. The computer program product of claim 13 wherein generating a test speech utterance for recognition by a speech recognition engine using a grammar further comprises sampling the test speech utterance from speech of a person.

16. The computer program product of claim 13 wherein generating a test speech utterance for recognition by a speech recognition engine using a grammar further comprises synthesizing the test speech utterance using a text-to-speech engine.

17. The computer program product of claim 13 wherein testing a grammar used in speech recognition for reliability in a plurality of operating environments having different background noise further comprises testing a plurality of grammars for use in speech recognition for a multimodal application, the multimodal application operating on a multimodal device supporting multiple modes of interaction that include a voice mode and one or more non-voice modes, the method further comprising:

identifying, by the multimodal application, a current background noise for a current operating environment in which the multimodal device operates; and

altering, by the multimodal application, flow of execution for the multimodal application in dependence upon the identified current background noise for the current operating environment in which the multimodal application operates.

18. The computer program product of claim 17 wherein altering, by the multimodal application, flow of execution for the multimodal application in dependence upon the identified current background noise for the current operating environment in which the multimodal application operates further comprises:

selecting, by the multimodal application, one of the plurality of grammars tested in dependence upon the current background noise and the evaluation of the speech recognition reliability of the plurality of grammars using the recorded background noises;

receiving, by the multimodal application, a voice utterance from a user; and

performing, by the multimodal application, speech recognition in dependence upon the selected grammar and the voice utterance.

19. The computer program product of claim 13 wherein the computer readable medium comprises a recordable medium.

20. The computer program product of claim 13 wherein the computer readable medium comprises a transmission medium.