Speech Correction for Typed Input

• US 20110208507A1
• Filed: 02/19/2010
• Published: 08/25/2011
• Est. Priority Date: 02/19/2010
• Status: Active Grant

• Current Assignee: Google LLC (Alphabet Inc.)
• Original Assignee: Google Inc. (Alphabet Inc.)
• Inventors: HUGHES, Christopher Thaddeus
• Granted Patent: US 8,423,351 B2
• Time in Patent Office: Days
• US Class Current: 704/9
• CPC Class Codes: G06F 2203/0381 Multimodal input, i.e. interface arrangements enabling the user to issue commands by simultaneous use of input devices of different nature, e.g. voice plus gesture on digitizer
• CPC Class Codes: G06F 3/0237 using prediction or retrieval techniques
• CPC Class Codes: G06F 3/038 Control and interface arrangements therefor, e.g. drivers or device-embedded control circuitry

Abstract

A method, computer program product, and system are provided for correcting one or more typed words on an electronic device. The method can include receiving one or more typed words from a text input device and generating one or more candidate words for the one or more typed words. The method can also include receiving an audio stream at the electronic device that corresponds to the one or more typed words. The audio stream can then be translated into text using the one or more candidate words, where the translation includes assigning a confidence score to each of the one or more candidate words. Based on the confidence score associated with each of the one or more candidate words, a candidate word can be selected among the one or more candidate words to represent each portion of the text. A word from the one or more typed words can be replaced with the selected candidate word based on the value of the confidence score associated with the selected candidate word.

Claims

1. A method for correcting one or more typed words on an electronic device, comprising:

receiving one or more typed words from a text input device;

generating one or more candidate words for the one or more typed words;

receiving an audio stream at the electronic device that corresponds to the one or more typed words;

translating the audio stream into text using the one or more candidate words, wherein the translating comprises assigning a confidence score to each of the one or more candidate words and selecting a candidate word among the one or more candidate words to represent each portion of the text based on the confidence score of the selected candidate word; and

replacing a word from the one or more typed words with the corresponding selected candidate word when the confidence score of the selected candidate word is above a predetermined threshold value.

2. The method of claim 1, further comprising:

displaying the one or more typed words on the electronic device.

3. The method of claim 1, wherein generating the one or more candidate words comprises generating the one or more candidate words with a finite state transducer (FST), wherein the FST represents a language model that contains the one or more candidate words.

4. The method of claim 3, wherein generating the one or more candidate words with the FST is based on an edit distance between a typed word from the one or more typed words and a corresponding candidate word from the one or more candidate words.

5. The method of claim 1, wherein generating the one or more candidate words comprises generating the one or more candidate words based on a vocabulary set of words stored on the electronic device.

6. The method of claim 1, wherein generating the one or more candidate words comprises generating the one more candidate words based on a first vocabulary set of words stored on the electronic device and a second vocabulary set of words stored on a remote server device.

7. The method of claim 6, wherein generating the one or more candidate words based on the second vocabulary set of words comprises receiving a portion of the one or more candidate words from the remote server device.

8. The method of claim 1, wherein receiving the audio stream comprises activating a microphone on the electronic device when the one or more typed words are received by the text input device.

9. The method of claim 1, wherein translating the audio stream comprises:

selecting the candidate word from two or more candidate words that correspond to each of the one or more typed words, wherein the selected candidate word has the highest confidence score among the two or more candidate words.

10. The method of claim 1, wherein replacing the word from the one or more typed words comprises displaying a correction in the one or more typed words using the selected candidate word with the confidence score above the predetermined threshold value.

11. A computer program product comprising a computer-usable medium having computer program logic recorded thereon for enabling a processor to correct one or more typed words on an electronic device, the computer program logic comprising:

first computer readable program code that enables a processor to receive one or more typed words from a text input device;

second computer readable program code that enables a processor to generate one or more candidate words for the one or more typed words;

third computer readable program code that enables a processor to receive an audio stream at the electronic device that corresponds to the one or more typed words;

fourth computer readable program code that enables a processor to translate the audio stream into text using the one or more candidate words, wherein the translation of the audio stream comprises assignment of a confidence score to each of the one or more candidate words and selection of a candidate word among the one or more candidate words to represent each portion of the text based on the confidence score of the selected candidate word; and

fifth computer readable program code that enables a processor to replace a word from the one or more typed words with the corresponding selected candidate word when the confidence score of the selected candidate word is above a predetermined threshold value.

12. The computer program product of claim 11, further comprising:

sixth computer readable program code that enables a processor to display the one or more typed words on the electronic device.

13. The computer program product of claim 11, wherein the second computer readable program code comprises:

sixth computer readable program code that enables a processor to generate the one or more candidate words with a finite state transducer (FST), wherein the FST represents a language model that contains the one or more candidate words.

14. The computer program product of claim 13, wherein the sixth computer readable program code comprises:

seventh computer readable program code that enables a processor to generate the one or more candidate words with the FST based on an edit distance between a typed word from the one or more typed words and a corresponding candidate word from the one or more candidate words.

15. The computer program product of claim 11, wherein the second computer readable program code comprises:

sixth computer readable program code that enables a processor to generate the one or more candidate words based on a vocabulary set of words stored on the electronic device.

16. The computer program product of claim 11, wherein the second computer readable program code comprises:

sixth computer readable program code that enables a processor to generate the one or more candidate words based on a first vocabulary set of words stored on the electronic device and a second vocabulary set of words stored on a remote server device.

17. The computer program product of claim 16, wherein the sixth computer readable program code comprises:

seventh computer readable program code that enables a processor to receive a portion of the one or more candidate words from the remote server device.

18. The computer program product of claim 11, wherein the third computer readable program code comprises:

sixth computer readable program code that enables a processor to activate a microphone on the electronic device when the one or more typed words are received by the text input device.

19. The computer program product of claim 11, wherein the fourth computer readable program code comprises:

sixth computer readable program code that enables a processor to select the candidate word from two or more candidate words that correspond to each of the one or more typed words, wherein the selected candidate word has the highest confidence score among the two or more candidate words.

20. The computer program product of claim 11, wherein the fifth computer readable program code comprises:

sixth computer readable program code that enables a processor to display a correction in the one or more typed words using the selected candidate word with the confidence score above the predetermined threshold value.

21. A system for correcting one or more typed words on an electronic device, comprising:

a text input device configured to receive one or more typed words;

a language model builder configured to generate one or more candidate words for the one or more typed words;

a microphone configured to receive an audio stream that corresponds to the one or more typed words;

a speech recognizer configured to;

translate the audio stream into text using the one or more candidate words, wherein translation of the audio stream comprises assignment of a confidence score to each of the one or more candidate words and selection of a candidate word among the one or more candidate words to represent each portion of the text based on the confidence score of the selected candidate word; and

a text update manager configured to replace a word from the one or more typed words with the corresponding selected candidate word when the confidence score of the selected candidate word is above a predetermined threshold value.

22. The system of claim 21, further comprising:

a display configured to display the one or more typed words.

23. The system of claim 21, wherein the language model builder is configured to generate the one or more candidate words with a finite state transducer (FST), wherein the FST represents a language model that contains the one or more candidate words.

24. The system of claim 23, wherein the language model builder is configured to generate the one or more candidate words with the FST based on an edit distance between a typed word from the one or more typed words and a corresponding candidate word from the one or more candidate words.

25. The system of claim 21, wherein the language model builder is configured to generate the one or more candidate words based on a vocabulary set of words stored on the electronic device.

26. The system of claim 21, wherein the language model builder is configured to generate the one or more candidate words based on a first vocabulary set of words stored on the electronic device and a second vocabulary set of words stored on a remote server device.

27. The system of claim 26, wherein the language model builder is configured to receive a portion of the one or more candidate words from the remote server device.

28. The system of claim 21, wherein the microphone is configured to receive the audio stream when the one or more typed words are received by the text input device.

29. The system of claim 21, wherein the speech recognizer is configured to select the candidate word from two or more candidate words that correspond to each of the one or more typed words, wherein the selected candidate word has the highest confidence score among the two or more candidate words.

30. The system of claim 21, wherein the text update manager is configured to display a correction in the one or more typed words using the selected candidate word with the confidence score above the predetermined threshold value.