System and Method for Latency Reduction for Automatic Speech Recognition Using Partial Multi-Pass Results

• US 20100094628A1
• Filed: 12/15/2009
• Published: 04/15/2010
• Est. Priority Date: 12/23/2003
• Status: Active Grant

• Current Assignee: Nuance Communications, Inc. (Microsoft Corporation)
• Original Assignee: AT&T Corporation (AT&T, Inc.)
• Inventors: Amento, Brian Scott, Bacchiani, Michiel Adriaan Unico
• Granted Patent: US 8,010,360 B2
• Time in Patent Office: Days
• US Class Current: 704/235
• CPC Class Codes: G10L 15/32 Multiple recognisers used in sequence or in parallel; Score combination systems therefor, e.g. voting systems

Abstract

A system and method is provided for reducing latency for automatic speech recognition. In one embodiment, intermediate results produced by multiple search passes are used to update a display of transcribed text.

Claims

1. (canceled)

2. A method for reducing latency in an automatic speech recognition (ASR) system, the method causing a computing device to perform steps comprising:

transcribing speech data using a first ASR pass, which operates at a first transcription rate near real time, to produce first transcription data;

displaying the first transcription data on a display unit;

determining how many additional ASR transcription passes are needed, wherein the additional transcription passes are slower and more accurate than the first ASR, to produce at least a second transcription data; and

displaying an indication of how many additional transcription passes are forthcoming that will update the displayed first transcription data.

3. The method of claim 2, wherein the first ASR pass operates at real time.

4. The method of claim 2, wherein the first ASR pass operates at greater than real time.

5. The method of claim 2, wherein displaying the first transcription data further comprises displaying an indicator that signifies that more accurate transcription data is being generated.

6. The method of claim 2, wherein the first transcription data is displayed in a different color or shade as compared to the indication.

7. The method of claim 2, wherein portions of displayed data having a relatively lower confidence score are distinctly displayed as compared to displayed data having a relatively higher confidence score.

8. The method of claim 7, wherein the displayed data having a relatively lower confidence score is displayed in a darker shade as compared to displayed data having a relatively higher confidence score.

9. The method of claim 7, wherein the portions of displayed data having a relatively lower confidence score enable a user to listen to the corresponding portions of speech data.

10. The method of claim 2, further comprising the additional transcription passes of the speech data being normalized ASR passes.

11. A method for reducing latency in an automatic speech recognition (ASR) system, the method causing a computing device to perform steps comprising:

transcribing speech data using a first ASR pass, which operates at a first transcription rate near real time to produce first transcription data and a first word graph;

adapting a model for a second ASR pass that uses the first word graph, wherein the second ASR pass produces a second transcription data and a second word graph and wherein the second ASR pass is slower and more accurate than the first ASR pass;

displaying a part of the first transcription data via a display unit, which corresponds to a portion of the speech data, prior to transcription of the portion of the speech data by the second ASR pass; and

updating the displayed part of the first transcription data with one or more of the second transcription data upon completion of the transcription.

12. The method of claim 11, wherein the first ASR pass operates at real time.

13. The method of claim 11, wherein the first ASR pass operates at greater than real time.

14. The method of claim 11, wherein the displaying comprises displaying an indicator that signifies that more accurate transcription data is being generated.

15. The method of claim 11, wherein the displayed data is in a different color or shade as compared to the updated displayed data.

16. The method of claim 11, wherein portions of displayed data having a relatively lower confidence score are distinctly displayed as compared to displayed data having a relatively higher confidence score.

17. The method of claim 16, wherein the displayed data having a relatively lower confidence score is displayed in a darker shade as compared to displayed data having a relatively higher confidence score.

18. The method of claim 16, wherein the portions of displayed data having a relatively lower confidence score enable a user to listen to the corresponding portions of speech data.

19. The method of claim 11, further comprising the additional transcription passes of the speech data being normalized ASR passes.

20. The method of claim 11, further comprising:

adapting a model for a third ASR pass that uses the second word graph, wherein the third ASR pass produces a third transcription data and a third word graph and wherein the third ASR pass is slower and more accurate than the second ASR pass; and

updating the displayed part of the second transcription data with one or more of the third transcription data upon completion of the transcription.

21. The method of claim 20, further comprising:

the adapted model performing the second ASR pass, constrained by the first word graph, using constrained model-space adaptation (CMA) to estimate a Maximum Likelihood Linear Regression (MLLR); and

transcribing the speech data using the third ASR pass based on the speech data and second transcription data from the second ASR pass to produce third transcription data, wherein the third ASR pass uses the CMA and the MLLR to adapt channel and speaker characteristics of the speaker.