Word-level correction of speech input

• US 9,263,048 B2
• Filed: 06/23/2015
• Issued: 02/16/2016
• Est. Priority Date: 01/05/2010
• Status: Active Grant

• Current Assignee: Google LLC (Alphabet Inc.)
• Original Assignee: Google Inc. (Alphabet Inc.)
• Inventors: LeBeau, Michael J., Byrne, William J., Jitkoff, John Nicholas, Ballinger, Brandon M., Kristjansson, Trausti T.
• Primary Examiner(s): GUERRA-ERAZO, EDGAR X
• Application Number:
  US14/747,306
• Publication Number:
  US 20150294668A1
• Time in Patent Office: 238 Days
• Field of Search: 704/235, 704/246, 704/240, 704/276, 704/270, 704/275, 704/9, 704/10, 704/251, 704/257
• US Class Current: 1/1
• CPC Class Codes: G06F 3/0482 Interaction with lists of selectable items, e.g. menus
• CPC Class Codes: G06F 3/04842 Selection of displayed objects or displayed text elements G06F3/0482 takes precedence
• CPC Class Codes: G06F 3/04886 by partitioning the display area of the touch-screen or the surface of the digitising tablet into independently controllable areas, e.g. virtual keyboards or menus
• CPC Class Codes: G06F 40/137 Hierarchical processing, e.g. outlines
• CPC Class Codes: G06F 40/166 Editing, e.g. inserting or deleting
• CPC Class Codes: G06F 40/232 Orthographic correction, e.g. spell checking or vowelisation
• CPC Class Codes: G06F 40/284 Lexical analysis, e.g. tokenisation or collocates
• CPC Class Codes: G10L 15/01 Assessment or evaluation of speech recognition systems
• CPC Class Codes: G10L 15/22 Procedures used during a speech recognition process, e.g. man-machine dialogue
• CPC Class Codes: G10L 15/26 Speech to text systems G10L15/08 takes precedence
• CPC Class Codes: G10L 15/30 Distributed recognition, e.g. in client-server systems, for mobile phones or network applications

Abstract

The subject matter of this specification can be implemented in, among other things, a computer-implemented method for correcting words in transcribed text including receiving speech audio data from a microphone. The method further includes sending the speech audio data to a transcription system. The method further includes receiving a word lattice transcribed from the speech audio data by the transcription system. The method further includes presenting one or more transcribed words from the word lattice. The method further includes receiving a user selection of at least one of the presented transcribed words. The method further includes presenting one or more alternate words from the word lattice for the selected transcribed word. The method further includes receiving a user selection of at least one of the alternate words. The method further includes replacing the selected transcribed word in the presented transcribed words with the selected alternate word.

Claims

1. A computer-implemented method comprising:

providing a first transcription of an utterance, wherein the first transcription of the utterance includes one or more words;

receiving data indicating a selection of a word from among the one or more words included in the first transcription of the utterance;

in response to receiving the data indicating the selection of the word, providing one or more alternate words for the selected word;

receiving data indicating a selection of a particular alternate word from among the one or more alternate words for the selected word;

selecting a second transcription of the utterance that includes the particular alternate word and that is identified as having a speech recognition confidence measure value that satisfies one or more criteria; and

replacing the first transcription of the utterance with the second transcription of the utterance.

2. The computer-implemented method of claim 1, wherein the one or more words are selected from a hierarchical word lattice.

3. The computer-implemented method of claim 2, wherein the hierarchical word lattice comprises nodes corresponding to the one or more words of the first transcription of the utterance and words of the second transcription of the utterance, and edges between the nodes that identify possible paths through the word lattice, wherein each path has an associated probability of being correct.

4. The computer-implemented method of claim 3, wherein providing the one or more alternate words for the selected word comprises:

identifying one or more alternate words for the selected word for which an edge exists to the other words of the first transcription in the hierarchical word lattice; and

providing the one or more alternate words for the selected word for which an edge exists to the other words of the first transcription as the one or more alternate words for the selected word, without providing other words from the hierarchical word lattice for which an edge does not exist to the other words of the first transcription as alternate words for the selected word.

5. The computer-implemented method of claim 1, wherein the first transcription of the utterance is a transcription of the utterance that is identified as having a highest speech recognizer confidence measure value among all transcriptions of the utterance.

6. The computer-implemented method of claim 1, wherein the second transcription of the utterance is a transcription of the utterance that includes the particular alternate word and that is identified as having a highest speech recognizer confidence measure value among all transcriptions of the utterance that include the particular alternate word.

7. The computer-implemented method of claim 1, wherein:

the first transcription of the utterance and the second transcription of the utterance are provided for output at a touchscreen display of a computing device; and

the data indicating the selection of the word from among the one or more words included in the first transcription and the data indicating the selection of the particular alternate word from among the one or more alternate words for the selected word are received in response to user input at the touchscreen display of the computing device.

8. A system for correcting words in transcribed text, the system comprising:

an automated speech recognizer operable to receive speech audio data and in response transcribe the speech audio data in a word lattice; and

a computing device comprising;

a microphone operable to receive speech audio and generate the speech audio data,

a network interface operable to send the speech audio data to the automated speech recognizer and in response receive the word lattice from the automated speech recognizer,

a display screen operable to present one or more transcribed words from the word lattice,

a user interface operable to receive a user selection of at least one of the transcribed words, and

one or more processors and a memory storing instructions that when executed by the processors cause the computing device to perform operations to;

provide a first transcription of an utterance, wherein the first transcription of the utterance includes one or more words;

receive data indicating a selection of a word from among the one or more words included in the first transcription of the utterance;

in response to receiving the data indicating the selection of the word, provide one or more alternate words for the selected word;

receive data indicating a selection of a particular alternate word from among the one or more alternate words for the selected word;

select a second transcription of the utterance that includes the particular alternate word and that is identified as having a speech recognition confidence measure value that satisfies one or more criteria; and

replace the first transcription of the utterance with the second transcription of the utterance.

9. The system of claim 8, wherein the one or more words are selected from a hierarchical word lattice.

10. The system of claim 9, wherein the hierarchical word lattice comprises nodes corresponding to the one or more words of the first transcription of the utterance and words of the second transcription of the utterance, and edges between the nodes that identify possible paths through the word lattice, wherein each path has an associated probability of being correct.

11. The system of claim 10, wherein providing the one or more alternate words for the selected word comprises:

identifying one or more alternate words for the selected word for which an edge exists to the other words of the first transcription in the hierarchical word lattice; and

providing the one or more alternate words for the selected word for which an edge exists to the other words of the first transcription as the one or more alternate words for the selected word, without providing other words from the hierarchical word lattice for which an edge does not exist to the other words of the first transcription as alternate words for the selected word.

12. The system of claim 8, wherein the first transcription of the utterance is a transcription of the utterance that is identified as having a highest speech recognizer confidence measure value among all transcriptions of the utterance.

13. The system of claim 8, wherein the second transcription of the utterance is a transcription of the utterance that includes the particular alternate word and that is identified as having a highest speech recognizer confidence measure value among all transcriptions of the utterance that include the particular alternate word.

14. The system of claim 8, wherein:

the first transcription of the utterance and the second transcription of the utterance are provided for output at a touchscreen display of a computing device; and

the data indicating the selection of the word from among the one or more words included in the first transcription and the data indicating the selection of the particular alternate word from among the one or more alternate words for the selected word are received in response to user input at the touchscreen display of the computing device.

15. A computer program product, encoded on a non-transitory computer-readable medium, operable to cause one or more processors to perform operations for correcting words in transcribed text, the operations comprising:

providing a first transcription of an utterance, wherein the first transcription of the utterance includes one or more words;

receiving data indicating a selection of a word from among the one or more words included in the first transcription of the utterance;

in response to receiving the data indicating the selection of the word, providing one or more alternate words for the selected word;

receiving data indicating a selection of a particular alternate word from among the one or more alternate words for the selected word;

selecting a second transcription of the utterance that includes the particular alternate word and that is identified as having a speech recognition confidence measure value that satisfies one or more criteria; and

replacing the first transcription of the utterance with the second transcription of the utterance.

16. The computer program product of claim 15, wherein the one or more words are selected from a hierarchical word lattice.

17. The computer program product of claim 16, wherein the hierarchical word lattice comprises nodes corresponding to the one or more words of the first transcription of the utterance and words of the second transcription of the utterance, and edges between the nodes that identify possible paths through the word lattice, wherein each path has an associated probability of being correct.

18. The computer program product of claim 17, wherein providing the one or more alternate words for the selected word comprises:

identifying one or more alternate words for the selected word for which an edge exists to the other words of the first transcription in the hierarchical word lattice; and

providing the one or more alternate words for the selected word for which an edge exists to the other words of the first transcription as the one or more alternate words for the selected word, without providing other words from the hierarchical word lattice for which an edge does not exist to the other words of the first transcription as alternate words for the selected word.

19. The computer program product of claim 15, wherein the first transcription of the utterance is a transcription of the utterance that is identified as having a highest speech recognizer confidence measure value among all transcriptions of the utterance.

20. The computer program product of claim 15, wherein the second transcription of the utterance is a transcription of the utterance that includes the particular alternate word and that is identified as having a highest speech recognizer confidence measure value among all transcriptions of the utterance that include the particular alternate word.