System and method for latency reduction for automatic speech recognition using partial multi-pass results

• US 8,209,176 B2
• Filed: 08/27/2011
• Issued: 06/26/2012
• Est. Priority Date: 12/23/2003
• Status: Expired due to Fees

• Current Assignee: Microsoft Technology Licensing LLC (Microsoft Corporation)
• Original Assignee: AT&T Intellectual Property I LP (AT&T, Inc.)
• Inventors: Bacchiani, Michiel Adriaan Unico, Amento, Brian Scott
• Primary Examiner(s): Vo, Huyen X.
• Application Number:
  US13/219,640
• Publication Number:
  US 20110313764A1
• Time in Patent Office: 304 Days
• Field of Search: 704/231, 704/251, 704/255, 704/257, 704/270, 704/270.1, 704/235, 704/246, 704/249, 704/252, 704/254
• US Class Current: 704/252
• CPC Class Codes: G10L 15/32 Multiple recognisers used in sequence or in parallel; Score combination systems therefor, e.g. voting systems

Abstract

A system and method is provided for reducing latency for automatic speech recognition. In one embodiment, intermediate results produced by multiple search passes are used to update a display of transcribed text.

Claims

1. A method comprising:

transcribing, via a processor, speech data using a first automatic speech recognition pass, which operates at a first transcription rate, to produce a first transcription data and a first word graph;

displaying a displayed part comprising an indication of a second automatic speech recognition pass which is forthcoming and at least part of the first transcription data corresponding to a portion of the speech data;

after displaying the displayed part, transcribing the speech data using the second automatic speech recognition pass, wherein the second automatic speech recognition pass uses the first word graph to produce second transcription data and a second word graph, and wherein the second automatic speech recognition pass is slower than the first automatic speech recognition pass; and

upon completing the second automatic speech recognition pass, updating the displayed part based at least in part on the second transcription data.

2. The method of claim 1, wherein the first automatic speech recognition pass operates at real time.

3. The method of claim 1, wherein the first automatic speech recognition pass operates faster than real time.

4. The method of claim 1, wherein the indication signifies that more additional transcription data is being generated.

5. The method of claim 1, wherein the displayed part changes color upon updating.

6. The method of claim 1, wherein a low confidence portion of the displayed part is distinctly displayed as compared to a high confidence portion of the displayed part.

7. The method of claim 6, wherein the low confidence portion of the displayed part is displayed in a darker shade as compared to the high confidence portion of the displayed data.

8. The method of claim 1, further comprising:

adapting an additional model for a third automatic speech recognition pass that uses the second word graph, wherein the third automatic speech recognition pass produces a third transcription data and a third word graph and wherein the third automatic speech recognition pass is slower than the second automatic speech recognition pass; and

updating the displayed part with at least the third transcription data.

9. A system comprising:

a processor; and

a non-transitory computer-readable memory storing instructions which, when executed by the processor, cause the processor to perform a method comprising;

transcribing speech data using a first automatic speech recognition pass, which operates at a first transcription rate, to produce a first transcription data and a first word graph;

displaying a displayed part comprising an indication of a second automatic speech recognition pass which is forthcoming and at least part of the first transcription data corresponding to a portion of the speech data

after displaying the displayed part, transcribing the speech data using the second automatic speech recognition pass, wherein the second automatic speech recognition pass uses the first word graph, wherein the second automatic speech recognition pass produces a second transcription data and a second word graph, and wherein the second automatic speech recognition pass is slower than the first automatic speech recognition pass;

and

upon completing the second automatic speech recognition pass, updating the displayed part based at least in part on the second transcription data.

10. The system of claim 9, wherein the first automatic speech recognition pass operates at real time.

11. The system of claim 9, wherein the first automatic speech recognition pass operates faster than real time.

12. The system of claim 9, wherein the indication that signifies that more additional transcription data is being generated.

13. The system of claim 9, wherein the displayed part changes color upon updating.

14. The system of claim 9, wherein a low confidence portion of the displayed part is distinctly displayed as compared to a high confidence portion of the displayed part.

15. The system of claim 14, wherein the low confidence portion of the displayed part is displayed in a darker shade as compared to the high confidence portion of the displayed data.

16. The system of claim 9, the instructions further comprising:

adapting an additional model for a third automatic speech recognition pass that uses the second word graph, wherein the third automatic speech recognition pass produces a third transcription data and a third word graph and wherein the third automatic speech recognition pass is slower than the second automatic speech recognition pass; and

updating the displayed part with at least the third transcription data.

17. A non-transitory computer-readable storage medium storing instructions which, when executed by a computing device, cause the computing device to perform a method comprising:

transcribing speech data using a first automatic speech recognition pass, which operates at a first transcription rate, to produce a first transcription data and a first word graph;

displaying a displayed part comprising an indication of a second automatic speech recognition pass which is forthcoming and at least part of the first transcription data corresponding to a portion of the speech data

after displaying the displayed part, transcribing the speech data using the second automatic speech recognition pass, wherein the second automatic speech recognition pass uses the first word graph, wherein the second automatic speech recognition pass produces a second transcription data and a second word graph, and wherein the second automatic speech recognition pass is slower than the first automatic speech recognition pass;

and

upon completing the second automatic speech recognition pass, updating the displayed part based at least in part on the second transcription data.

18. The non-transitory computer-readable storage medium of claim 17, wherein the first automatic speech recognition pass operates at real time.

19. The non-transitory computer-readable storage medium of claim 17, wherein the first automatic speech recognition pass operates faster than real time.

20. The non-transitory computer-readable storage medium of claim 17, wherein the indication signifies that more additional transcription data is being generated.