Continuous speech transcription performance indication

• US 8,510,109 B2
• Filed: 08/22/2008
• Issued: 08/13/2013
• Est. Priority Date: 08/22/2007
• Status: Active Grant

• Current Assignee: Amazon Technologies, Inc. (Amazon.com, Inc.)
• Original Assignee: Canyon IP Holdings LLC (Intellectual Ventures LLC)
• Inventors: Terrell, James Richard II, White, Marc, Jablokov, Igor Roditis
• Primary Examiner(s): Abebe, Daniel D
• Application Number:
  US12/197,213
• Publication Number:
  US 20090055175A1
• Time in Patent Office: 1,817 Days
• Field of Search: 704/235, 704/251, 369/25.01
• US Class Current: 704/235
• CPC Class Codes: G10L 15/01 Assessment or evaluation of speech recognition systems
• CPC Class Codes: G10L 15/22 Procedures used during a speech recognition process, e.g. man-machine dialogue
• CPC Class Codes: G10L 15/26 Speech to text systems G10L15/08 takes precedence
• CPC Class Codes: G10L 2015/221 Announcement of recognition results

Abstract

A method of providing speech transcription performance indication includes receiving, at a user device data representing text transcribed from an audio stream by an ASR system, and data representing a metric associated with the audio stream; displaying, via the user device, said text; and via the user device, providing, in user-perceptible form, an indicator of said metric. Another method includes displaying, by a user device, text transcribed from an audio stream by an ASR system; and via the user device, providing, in user-perceptible form, an indicator of a level of background noise of the audio stream. Another method includes receiving data representing an audio stream; converting said data representing an audio stream to text via an ASR system; determining a metric associated with the audio stream; transmitting data representing said text to a user device; and transmitting data representing said metric to the user device.

Claims

1. A computer-implemented method of providing speech transcription performance indication, comprising:

displaying, by a user device, text transcribed from an audio stream by an Automatic speech recognition system; and

via the user device, providing, in user-perceptible form, an indicator of a level of background noise of the audio stream associated with a word or syllable of the displayed transcribed text in a manner that communicates correspondence of the indicator of the level of background noise with the associated word or syllable of the displayed transcribed text.

2. The computer-implemented method of claim 1, further comprising:

before displaying, by the user device, the text transcribed from an audio stream by an automatic speech recognition system, receiving, by the user device, the transcribed text from the automatic speech recognition system.

3. A computer-implemented method of providing speech transcription performance indication, comprising:

receiving, at a user device, text transcribed from an audio stream by an automatic speech recognition system and a metric associated with the audio stream;

displaying, via the user device, the text; and

via the user device, providing, in user-perceptible form, an indicator of the metric in a manner that communicates correspondence of the metric with a word or syllable of said text.

4. The computer-implemented method of claim 3, wherein:

the audio stream was transmitted to a server from a transmitting device; and

the audio stream was converted to the text by the server.

5. The computer-implemented method of claim 4, wherein the text was confirmed by a user of the transmitting device prior to receiving, at the user device, the test transcribed from the audio stream by the automatic speech recognition system and the metric associated with the audio stream.

6. The computer-implemented method of claim 3, wherein the text was confirmed prior to receiving, at the user device, the text transcribed from the audio stream by the automatic speech recognition system and the metric associated with the audio stream.

7. The computer-implemented method of claim 3, wherein the text comprises a word and the metric is a metric of the word.

8. The computer-implemented method of claim 3, wherein the text comprises a plurality of words and the metric is one of a plurality of metrics received at the user device and provided via the user device, each of the plurality of metrics being a metric of a distinct one of the plurality of words.

9. The computer-implemented method of claim 3, wherein the text comprises a syllable and the metric is a metric of the syllable.

10. The computer-implemented method of claim 3, wherein the text comprises a plurality of syllables and the metric is one of a plurality of metrics received at the user device and provided via the user device, each of the plurality of metrics being a metric of a distinct one of the plurality of syllables.

11. The computer-implemented method of claim 3, wherein the text comprises a phrase and the metric is a metric of the phrase.

12. The computer-implemented method of claim 3, wherein the text comprises a plurality of phrases and the metric is one of a plurality of metrics received at the user device and provided via the user device, each of the plurality of metrics being a metric of a distinct one of the plurality of phrases.

13. The computer-implemented method of claim 3, wherein the text comprises a sentence and the metric is a metric of the sentence.

14. The computer-implemented method of claim 3, wherein the text comprises a plurality of sentences and the metric is one of a plurality of metrics received at the user device and provided via the user device, each of the plurality of metrics being a metric of a distinct one of the plurality of sentences.

15. The computer-implemented method of claim 3, wherein:

the text comprises a plurality of units and the metric is one of a plurality of metrics received at the user device and provided via the user device;

each of the plurality of metrics comprise a metric of one of the plurality of units; and

each of the units comprise at least one of a word, a sentence, a phrase, or a syllable.

16. The computer-implemented method of claim 3, wherein the user device is a mobile phone.

17. The computer-implemented method of claim 16, wherein the text and the metric are received in the same manner as a text message.

18. The computer-implemented method of claim 16, wherein the text is displayed as a text message.

19. The computer-implemented method of claim 3, wherein the device is a computer.

20. The computer-implemented method of claim 19, wherein the text and the metric are received in the same manner as an instant message.

21. The computer-implemented method of claim 19, wherein the text is displayed as an instant message.

22. The computer-implemented method of claim 19, wherein the text and the metric are received in the same manner as an email.

23. The computer-implemented method of claim 19, wherein the text is displayed as an email.

24. The computer-implemented method of claim 3, wherein the metric associated with the audio stream comprises a volume of the audio stream.

25. The computer-implemented method of claim 3, wherein the metric associated with the audio stream comprises background noise of the audio stream.

26. The computer-implemented method of claim 3, wherein the metric associated with the audio stream comprises a confidence level of the audio stream.

27. The computer-implemented method of claim 3, wherein the indicator comprises a font color.

28. The computer-implemented method of claim 3, wherein the indicator comprises a font weight.

29. The computer-implemented method of claim 3, wherein the indicator comprises a font size.

30. The computer-implemented method of claim 3, wherein the indicator comprises underlining.

31. The computer-implemented method of claim 3, wherein the indicator comprises an audible indicator.

32. A computer-implemented method of providing speech transcription performance indication, comprising:

under control of one or more computing devices configured with specific computer executable instructions,

receiving an audio stream;

converting the audio stream to text via an automatic speech recognition system;

determining one or more metrics associated with the audio stream;

transmitting, to a user device, the text; and

transmitting, to the user device, the one or more metrics, each metric being linked to a word or syllable of the text.

33. A non-transitory computer-readable medium comprising a speech transcription performance indication module configured to execute in one or more processors of a user device, the speech transcription performance indication module being further configured to:

obtain, via the user device, text transcribed from an audio input by an automatic speech recognition engine;

obtain, via the user device, a metric associated with the audio input;

cause the user device to display the text; and

via the user device, provide, in user-perceptible form, an indicator of the metric in a manner that communicates correspondence of the metric with a word or syllable of the text.

34. The non-transitory computer-readable medium of claim 33, wherein the text comprises at least one of a syllable, a word, a phrase, or a sentence.

35. The non-transitory computer-readable medium of claim 33, wherein the metric associated with the audio input comprises at least one of a volume of the audio input, a background noise level of the audio input;

or a confidence level of the audio input.

36. The non-transitory computer-readable medium of claim 33, wherein the indicator comprises at least one of a font color, a font weight, a font size, an underlining, or an audible indicator.

37. The non-transitory computer-readable medium of claim 33, wherein the text is displayed as at least one of a text message, an instant message, or an email.

38. The non-transitory computer-readable medium of claim 33, wherein the automatic speech recognition engine is incorporated into the user device.

39. A system for providing speech transcription performance indication, the system comprising:

a data store configured to store instructions, that when executed, implement an automatic speech recognition engine; and

a first computing device in communication with the data store, the first computing device configured to;

receive an audio input from a second computing device;

determine one or more metrics associated with the audio input;

convert the audio input to text comprising one or more units using the automatic speech recognition engine; and

transmit the text and the one or more metrics to the second computing device for display of the text with a user-perceptible indicator for each of the one or more metrics,

wherein each metric is linked to a unit of the text; and

wherein the indicator communicates correspondence of the metric with the unit of the text to which it is linked.

40. The system of claim 39, wherein the each of the one or more metrics associated with the audio input comprises at least one of a volume of the audio input, a background noise level of the audio input;

or a confidence level of the audio input.

41. The system of claim 39, wherein the indicator comprises at least one of a font color, a font weight, a font size, an underlining, or an audible indicator.

42. The system of claim 39, wherein the first computing device is further configured to transmit an artifact to the second computing device for audio playback of the artifact.

43. The system of claim 42, wherein the artifact comprises at least one of a tone, an artificial noise, a noise derived from the audio input, or a pause.

44. The system of claim 39, wherein the one or more metrics comprise XML fragments.