Use of intermediate speech transcription results in editing final speech transcription results

• US 8,352,261 B2
• Filed: 03/09/2009
• Issued: 01/08/2013
• Est. Priority Date: 03/07/2008
• Status: Active Grant

• Current Assignee: Amazon Technologies, Inc. (Amazon.com, Inc.)
• Original Assignee: Canyon IP Holdings LLC (Intellectual Ventures LLC)
• Inventors: Terrell, James Richard II, White, Marc
• Primary Examiner(s): Smits, Talivaldis Ivars
• Application Number:
  US12/400,723
• Publication Number:
  US 20090228274A1
• Time in Patent Office: 1,401 Days
• Field of Search: None
• US Class Current: 704/235
• CPC Class Codes: G10L 15/22 Procedures used during a speech recognition process, e.g. man-machine dialogue
• CPC Class Codes: G10L 2015/221 Announcement of recognition results

Abstract

A communication system includes at least one transmitting device and at least one receiving device, one or more network systems for connecting the transmitting device to the receiving device, and an automatic speech recognition (“ASR”) system, including an ASR engine. A user speaks an utterance into the transmitting device, and the recorded speech audio is sent to the ASR engine. The ASR engine returns intermediate transcription results to the transmitting device, which displays the intermediate transcription results in real-time to the user. The intermediate transcription results are also correlated by utterance fragment to final transcription results and displayed to the user. The user may use the information thus presented to make decisions as to whether to edit the final transcription results or to speak the utterance again, thereby repeating the process. The intermediate transcription results may also be used by the user to edit the final transcription results.

Claims

1. A computer-implemented method comprising:

receiving, at a user device, data representing text, the text comprising final speech transcription results and intermediate speech transcription results generated from an audio stream comprising an utterance;

at least temporarily displaying, via the user device, all of the intermediate speech transcription results in a list, wherein each newly-received intermediate transcription result is added to the list as it is received at the user device; and

displaying, via the user device, the final speech transcription results for viewing by a user.

2. A computer-implemented method comprising:

receiving, at a user device, data representing text, the text comprising final speech transcription results and intermediate speech transcription results generated from an audio stream comprising an utterance;

at least temporarily displaying the intermediate speech transcription results via the user device; and

displaying the final speech transcription results via the user device;

wherein the intermediate speech transcription results are displayed via the user device substantially while the final speech transcription results are displayed.

3. The computer-implemented method of claim 2, wherein displaying the intermediate speech transcription results further comprises displaying fragments of the intermediate speech transcription results in association with corresponding fragments of the final speech transcription results.

4. The computer-implemented method of claim 3, wherein displaying the intermediate speech transcription results further comprises displaying one or more intermediate speech transcription results associated with a fragment in the final speech transcription results.

5. The computer-implemented method of claim 3, wherein displaying the intermediate speech transcription results further comprises displaying one or more intermediate speech transcription results only for a particular fragment in the final speech transcription results.

6. The computer-implemented method of claim 5, further comprising receiving user input representative of the particular fragment in the final speech transcription results for which associated intermediate speech transcription results are to be displayed.

7. The computer-implemented method of claim 3, wherein displaying the intermediate speech transcription results further comprises displaying one or more intermediate speech transcription results associated with a fragment in the final speech transcription results via a dropdown list.

8. The computer-implemented method of claim 7, wherein the dropdown list is ordered according to a confidence level associated with each of the respective intermediate speech transcription results.

9. The computer-implemented method of claim 3, wherein displaying the intermediate speech transcription results further comprises displaying one or more intermediate speech transcription results for each fragment in the final speech transcription results.

10. A non-transitory computer-readable medium comprising a computer-executable component configured to be executed in one or more processors of a user device, the computer-executable component being further configured to:

receive speech via the user device;

obtain one or more intermediate speech transcription results from the speech;

cause the user device to display each of the one or more intermediate speech transcription results as it is obtained;

obtain final transcription results from the speech; and

upon obtaining the final transcription results from the speech, cause the user device to display concurrently both the one or more intermediate transcription results and the final transcription results.

11. The non-transitory computer-readable medium of claim 10, wherein the computer-executable component is further configured to obtain the one or more intermediate speech transcription results from the speech by generating the one or more intermediate speech transcription results from the speech.

12. The non-transitory computer-readable medium of claim 10, wherein the computer-executable component is further configured to obtain the final transcription results from the speech by generating the final transcription results from the speech.

13. The non-transitory computer-readable medium of claim 10, wherein the computer-executable component is further configured to:

for a fragment of the final transcription results, cause the user device to display a list of the fragment'"'"'s corresponding intermediate speech transcription results.

14. The non-transitory computer-readable medium of claim 13, wherein the computer-executable component is further configured to receive, via the user device, a selection of an intermediate speech transcription result in the list.

15. The non-transitory computer-readable medium of claim 13, wherein:

the corresponding intermediate speech transcription results each have a confidence value; and

the corresponding intermediate speech transcription results are ordered in the list according to their confidence values.

16. A system comprising:

an electronic data store configured to store instructions, that when executed, implement an automatic speech recognition engine; and

a computing device in communication with the electronic data store, the computing device configured to;

receive speech;

obtain, using the automatic speech recognition engine, one or more intermediate speech transcription results from the speech;

display each of the one or more intermediate speech transcription results as it is obtained;

obtain, using the automatic speech recognition engine, final transcription results from the speech; and

upon obtaining the final transcription results from the speech, display concurrently both the one or more intermediate transcription results and the final transcription results.

17. The system of claim 16, wherein the computing device is further configured to obtain the one or more intermediate speech transcription results from the speech by generating the one or more intermediate speech transcription results from the speech.

18. The system of claim 16, wherein the computing device is further configured to obtain the final transcription results from the speech by generating the final transcription results from the speech.

19. The system of claim 16, wherein the computing device is further configured to:

for a fragment of the final transcription results, display a list of the fragment'"'"'s corresponding intermediate speech transcription results.

20. The system of claim 19, wherein the computing device is further configured to receive a selection of an intermediate speech transcription result in the list.

21. The system of claim 19, wherein:

the corresponding intermediate speech transcription results each have a confidence value; and

the corresponding intermediate speech transcription results are ordered in the list according to their confidence values.