System and method for supplemental speech recognition by identified idle resources

• US 9,431,005 B2
• Filed: 11/30/2012
• Issued: 08/30/2016
• Est. Priority Date: 12/04/2009
• Status: Active Grant

• Current Assignee: Microsoft Technology Licensing LLC (Microsoft Corporation)
• Original Assignee: AT&T Intellectual Property I LP (AT&T, Inc.)
• Inventors: Ljolje, Andrej, Gilbert, Mazin
• Primary Examiner(s): Lerner, Martin
• Application Number:
  US13/690,671
• Publication Number:
  US 20130090925A1
• Time in Patent Office: 1,369 Days
• Field of Search: 704/231, 704/236, 704/251, 704/270, 704/270.1
• US Class Current: 1/1
• CPC Class Codes: G10L 15/00 Speech recognition G10L17/00 takes precedence
• CPC Class Codes: G10L 15/285 Memory allocation or algorithm optimisation to reduce hardware requirements
• CPC Class Codes: G10L 15/32 Multiple recognisers used in sequence or in parallel; Score combination systems therefor, e.g. voting systems

Abstract

Disclosed herein are systems, methods, and computer-readable storage media for improving automatic speech recognition performance. A system practicing the method identifies idle speech recognition resources and establishes a supplemental speech recognizer on the idle resources based on overall speech recognition demand. The supplemental speech recognizer can differ from a main speech recognizer, and, along with the main speech recognizer, can be associated with a particular speaker. The system performs speech recognition on speech received from the particular speaker in parallel with the main speech recognizer and the supplemental speech recognizer and combines results from the main and supplemental speech recognizer. The system recognizes the received speech based on the combined results. The system can use beam adjustment in place of or in combination with a supplemental speech recognizer. A scheduling algorithm can tailor a particular combination of speech recognition resources and release the supplemental speech recognizer based on increased demand.

Claims

1. A method comprising:

projecting, via a processor, an expected demand for speech recognition resources;

identifying, via the processor and based on the expected demand for speech recognition, a main speech recognizer and a supplemental speech recognizer;

assigning the main speech recognizer to recognize a plurality of speech from a plurality of speakers;

beginning to process the plurality of speech using the main speech recognizer, to yield main results; and

upon determining that first speech recognition results of a first user in the plurality of speakers have a lower accuracy than alternative speech recognition results of a second user;

assigning, via the processor, the supplemental speech recognizer to recognize speech from the first user, wherein the assigning of the supplemental speech recognizer is a reallocation of the supplemental speech recognizer away from the second user;

continuing to process the speech from the first user using the supplemental speech recognizer and the main speech recognizer, to yield supplemental results; and

combining additional speech recognition results, produced by the main speech recognizer, and the supplemental results.

2. The method of claim 1, wherein processing the speech from the first user using the supplemental speech recognizer is done in parallel with processing of the speech from the first user using the main speech recognizer.

3. The method of claim 1, wherein the main speech recognizer and the supplemental speech recognizer belong to a plurality of speech recognizers comprising additional speech recognizers.

4. The method of claim 1, wherein the supplemental speech recognizer is assigned to idle based on decreased demand for speech recognition.

5. The method of claim 1, wherein the main results and the supplemental results comprise recognition strings.

6. The method of claim 5, further comprising recognizing the recognition strings using one of a word lattice and a confusion network.

7. The method of claim 1, wherein the main speech recognizer and the supplemental speech recognizer differ from each other in one of a spectral analysis in a front end, a pronouncing dictionary, and a training algorithm.

8. A system comprising:

a processor; and

a computer-readable storage medium having instructions stored which, when executed by the processor, cause the processor to perform a method comprising;

projecting an expected demand for speech recognition resources;

identifying, based on the expected demand for speech recognition, a main speech recognizer and a supplemental speech recognizer;

assigning the main speech recognizer to recognize a plurality of speech from a plurality of speakers;

beginning to process the plurality of speech using the main speech recognizer, to yield main results; and

upon determining that first speech recognition results of a first user in the plurality of speakers have a lower accuracy than alternative speech recognition results of a second user;

assigning the supplemental speech recognizer to recognize speech from the first user, wherein the assigning of the supplemental speech recognizer is a reallocation of the supplemental speech recognizer away from the second user;

continuing to process the speech from the first user using the supplemental speech recognizer and the main speech recognizer, to yield supplemental results; and

combining additional speech recognition results, produced by the main speech recognizer, and the supplemental results.

9. The system of claim 8, wherein processing the speech from the first user using the supplemental speech recognizer is done in parallel with processing of the speech from the first user using the main speech recognizer.

10. The system of claim 8, wherein the main speech recognizer and the supplemental speech recognizer belong to a plurality of speech recognizers comprising additional speech recognizers.

11. The system of claim 8, wherein the supplemental speech recognizer is assigned to idle based on decreased demand for speech recognition.

12. The system of claim 8, wherein the main results and the supplemental results comprise recognition strings.

13. The system of claim 12, the computer-readable storage medium having additional instructions stored which, when executed by the processor, result in operations comprising recognizing the recognition strings using one of a word lattice and a confusion network.

14. The system of claim 8, wherein the main speech recognizer and the supplemental speech recognizer differ from each other in one of a spectral analysis in a front end, a pronouncing dictionary, and a training algorithm.

15. A non-transitory computer-readable storage device having instructions stored which, when executed by a computing device, cause the computing device to perform operations comprising:

projecting, via the computing device, an expected demand for speech recognition resources;

identifying, via the computing device and based on the expected demand for speech recognition, a main speech recognizer and a supplemental speech recognizer;

assigning the main speech recognizer to recognize a plurality of speech from a plurality of speakers;

beginning to process the plurality of speech using the main speech recognizer, to yield main results;

and upon determining that first speech recognition results of a first user in the plurality of speakers have a lower accuracy than alternative speech recognition results of a second user;

assigning the supplemental speech recognizer to recognize speech from the first user, wherein the assigning of the supplemental speech recognizer is a reallocation of the supplemental speech recognizer away from the second user;

continuing to process the speech from the first user using the supplemental speech recognizer and the main speech recognizer, to yield supplemental results; and

combining additional speech recognition results, produced by the main speech recognizer, and the supplemental results.

16. The non-transitory computer-readable storage device of claim 15, wherein processing the speech from the first user using the supplemental speech recognizer is done in parallel with processing of the speech from the first user using the main speech recognizer.

17. The non-transitory computer-readable storage device of claim 15, wherein the main speech recognizer and the supplemental speech recognizer belong to a plurality of speech recognizers comprising additional speech recognizers.

18. The non-transitory computer-readable storage device of claim 15, wherein the supplemental speech recognizer is assigned to idle based on decreased demand for speech recognition.

19. The non-transitory computer-readable storage device of claim 15, wherein the main results and the supplemental results comprise recognition strings.

20. The non-transitory computer-readable storage device of claim 15, having additional instructions stored which, when executed by the computing device, result in operations comprising recognizing the additional speech recognition results using one of a word lattice and a confusion network.