Speech Recognition with Parallel Recognition Tasks

• US 20100004930A1
• Filed: 07/02/2008
• Published: 01/07/2010
• Est. Priority Date: 07/02/2008
• Status: Active Grant

• Current Assignee: Google LLC (Alphabet Inc.)
• Original Assignee: Google Inc. (Alphabet Inc.)
• Inventors: SIOHAN, OLIVIER, BEAUFAYS, FRANCOISE, STROPE, BRIAN
• Granted Patent: US 8,364,481 B2
• Time in Patent Office: Days
• US Class Current: 704/240
• CPC Class Codes: G10L 15/00 Speech recognition G10L17/00 takes precedence
• CPC Class Codes: G10L 15/01 Assessment or evaluation of speech recognition systems
• CPC Class Codes: G10L 15/26 Speech to text systems G10L15/08 takes precedence
• CPC Class Codes: G10L 15/32 Multiple recognisers used in sequence or in parallel; Score combination systems therefor, e.g. voting systems

Abstract

The subject matter of this specification can be embodied in, among other things, a method that includes receiving an audio signal and initiating speech recognition tasks by a plurality of speech recognition systems (SRS'"'"'s). Each SRS is configured to generate a recognition result specifying possible speech included in the audio signal and a confidence value indicating a confidence in a correctness of the speech result. The method also includes completing a portion of the speech recognition tasks including generating one or more recognition results and one or more confidence values for the one or more recognition results, determining whether the one or more confidence values meets a confidence threshold, aborting a remaining portion of the speech recognition tasks for SRS'"'"'s that have not completed generating a recognition result, and outputting a final recognition result based on at least one of the generated one or more speech results.

Claims

1. A computer-implemented method comprising:

receiving an audio signal;

initiating speech recognition tasks by a plurality of speech recognition systems (SRS'"'"'s), each configured to generate a recognition result specifying possible speech included in the audio signal and a confidence value indicating a confidence in a correctness of the recognition result;

completing a portion of the speech recognition tasks comprising generating one or more recognition results and one or more confidence values for the one or more recognition results;

determining whether the one or more confidence values meets a confidence threshold;

aborting a remaining portion of the speech recognition tasks for SRS'"'"'s that have not completed; and

outputting a final recognition result based on at least one of the generated one or more recognition results.

2. The method of claim 1, further comprising aborting a speech recognition task if the speech recognition task exceeds a latency threshold specifying a maximum period of time permitted for completion of the speech recognition task.

3. The method of claim 1, further comprising aborting an uncompleted speech recognition task if a completed speech recognition task is associated with a low confidence value in a generated recognition result.

4. The method of claim 1, wherein at least one of the SRS'"'"'s generates a plurality of recognition results.

5. The method of claim 4, further comprising ranking a top N recognition results of the plurality of recognition results based on confidence values associated with the top N recognition results, wherein N represents any positive integer.

6. The method of claim 5, further comprising determining a running average for confidence values associated with the top N recognition results based on confidence values generated by different SRS'"'"'s.

7. The method of claim 6, selecting the final recognition result from the top N results based on which recognition result is associated with the highest overall confidence value.

8. The method of claim 1, further comprising selecting the final recognition result based on which recognition result is associated with the greatest confidence value.

9. The method of claim 1, further comprising selecting the final recognition result based on the number of SRS'"'"'s that generate the final recognition result and the confidence values associated with the final recognition result.

10. The method of claim 1, further comprising selecting the final recognition result based on information about the reliability of one or more of the plurality of SRS'"'"'s and confidence values associated with the recognition results.

11. The method of claim 1, further comprising aborting all speech recognition tasks by the plurality of the SRS'"'"'s if a maximum time period is exceeded.

12. The method of claim 11, further comprising prompting of a user to repeat an utterance.

13. The method of claim 11, further comprising transmitting a request that a human operator receive the audio signal.

14. The method of claim 1, further comprising selecting the final recognition result based on a distribution of the confidence values on a normalized scale.

15. The method of claim 14, further comprising selecting a recognition result as the final recognition result if the recognition result is associated with a high confidence value and at least a portion of the other recognition results are clustered together within a relatively small range of low confidence values.

16. The method of claim 1, further comprising weighting a particular recognition result in the selection of the final recognition result based on a correlation between the SRS'"'"'s that generated the particular recognition result.

17. The method of claim 1, wherein at least a portion of the SRS'"'"'s comprise different language models or acoustic models.

18. The method of claim 1, further comprising pausing the remaining portion of the speech recognition tasks for SRS'"'"'s that have not completed if the one or more confidence values meets a confidence threshold.

19. The method of claim 18, further comprising resuming the paused remaining portion of speech recognition tasks if an indication is received that the final recognition result is incorrect.

20. A system comprising:

a plurality of speech recognition systems that initiate speech recognition tasks to identify possible speech encoded in a received audio signal, each speech recognition system (SRS) configured to generate a recognition result and a confidence value indicating a confidence in a correctness of the recognition result;

a recognition managing module to receive recognition results as the recognition results are generated by the SRS'"'"'s and to receive confidence values associated with the generated recognition results, wherein the recognition management module aborts uncompleted speech recognition tasks by SRS'"'"'s that have not completed generating recognition results if one or more of the received confidence values meets a confidence threshold; and

an interface for transmitting a final recognition result selected based on the confidence values of the generated recognition results.

21. A system comprising:

a plurality of speech recognition systems that initiate speech recognition tasks for a received audio signal, each speech recognition system (SRS) configured to generate a recognition result identifying possible speech within the audio signal and a confidence value indicating a confidence in a correctness of the recognition result;

means for receiving one or more recognition results and one or more corresponding confidence values from each SRS as the one or more recognition results are generated by the SRS, aborting uncompleted speech recognition tasks by SRS'"'"'s that have not completed generating recognition results if one or more of the received confidence values meets a confidence threshold, and selecting a final recognition result based on the confidence values of the generated recognition results; and

an interface for transmitting a final recognition result that represents possible speech within the audio signal.