Biasing voice correction suggestions

• US 10,529,316 B1
• Filed: 02/06/2019
• Issued: 01/07/2020
• Est. Priority Date: 01/05/2016
• Status: Active Grant

• Current Assignee: Google LLC (Alphabet Inc.)
• Original Assignee: Google LLC (Alphabet Inc.)
• Inventors: Cherepanov, Evgeny A., Foerster, Jakob Nicolaus, Sridar, Vikram, Rabinovitz, Ishai, Tabach, Omer
• Primary Examiner(s): Patel, Shreyans A
• Application Number:
  US16/268,957
• Time in Patent Office: 335 Days
• CPC Class Codes: G10L 15/01 Assessment or evaluation of speech recognition systems
• CPC Class Codes: G10L 15/187 Phonemic context, e.g. pronunciation rules, phonotactical constraints or phoneme n-grams
• CPC Class Codes: G10L 15/22 Procedures used during a speech recognition process, e.g. man-machine dialogue
• CPC Class Codes: G10L 15/26 Speech to text systems G10L15/08 takes precedence
• CPC Class Codes: G10L 2015/221 Announcement of recognition results

Abstract

Methods, systems, and apparatus, including computer programs encoded on computer storage media, for natural language processing. One of the method includes receiving a voice input from a user device; generating a recognition output; receiving a user selection of one or more terms in the recognition output; receiving a user input of one or more letters replacing the user selected one or more terms; determining suggested correction candidates based in part on the user input and the voice input; and providing one or more suggested correction candidates to the user device as suggested corrected recognition outputs.

Claims

1. A method comprising:

receiving, at a user device, a voice input from a user of the user device;

displaying, by the user device, an initial recognition output in a graphical user interface based on the voice input, the initial recognition output comprising an initial transcription of the voice input, the initial transcription comprising a misrecognized term;

receiving, at the user device, a selection indication indicating a user selection in the graphical user interface of a portion of the initial recognition output to correct the misrecognized term; and

after receiving the selection indication, displaying, by the user device, a list of candidate transcriptions of the voice input in the graphical user interface, each candidate transcription comprising a respective phonetic similarity to the received voice input.

2. The method of claim 1, further comprising:

receiving, at the user device, another selection indication indicating another user selection in the graphical user interface of one of the candidate transcriptions displayed in the graphical user interface as a corrected transcription of the voice input; and

displaying, by the user device, one or more results corresponding to the corrected transcription of the voice input in the graphical user interface.

3. The method of claim 2, further comprising, when displaying the one or more results corresponding to the corrected transcription of the voice input, displaying, by the user device, the corrected transcription of the voice input in the graphical user interface of the user device.

4. The method of claim 3, wherein the one or more results corresponding to the corrected transcription of the voice input displayed in the graphical user interface are presented below the corrected transcription of the voice input displayed in the graphical user interface.

5. The method of claim 1, wherein the initial recognition output corresponds to an action to provide output text to an electronic message program executing on the user device.

6. The method of claim 1, wherein the initial recognition output corresponds to an action of inputting text in a text editor or an electronic message.

7. The method of claim 1, further comprising, after receiving the selection indication:

receiving, at the user device, one or more text characters inputted by the user through the graphical user interface, the one or more text characters replacing the misrecognized term of the initial transcription of the voice input to provide a corrected transcription of the voice input; and

displaying, by the user device, one or more results corresponding to the corrected transcription of the voice input in the graphical user interface.

8. The method of claim 7, wherein the one or more results corresponding to the corrected transcription of the voice input displayed in the graphical user interface comprise the one or more text characters inputted by the user through the graphical user interface.

9. The method of claim 7, wherein the one or more text characters comprise one or more letters inputted by the user through the graphical user interface.

10. The method of claim 1, further comprising, in response to receiving the selection indication, displaying, by the user device, a keyboard in the graphical user interface.

11. The method of claim 1, wherein the list of candidate transcriptions of the voice input displayed in the graphical user interface are ranked according to the respective phonetic similarity between each candidate transcription and the received voice input.

12. The method of claim 1, wherein the list of candidate transcriptions of the voice input displayed in the graphical user interface are based on a popularity of the candidate transcriptions.

13. The method of claim 1, further comprising, after receiving the selection indication, superimposing, by the user device, a graphical indicator in the graphical user interface over the misrecognized term of the initial transcription.

14. The method of claim 1, further comprising:

prior to receiving the selection indication indicating the user selection in the graphical user interface, displaying one or more initial results corresponding to the initial transcription of the voice input;

after displaying the list of candidate transcriptions of the voice input in the graphical user interface, receiving another selection indication indicating another user selection in the graphical user interface of one of the candidate transcriptions of the displayed list of candidate transcriptions as a corrected transcription of the voice input; and

updating the one or more initial results corresponding to the initial transcription of the voice input displayed in the graphical user interface with one or more corrected results corresponding to the corrected transcription of the voice input.

15. A user device comprising:

data processing hardware; and

memory hardware in communication with the data processing hardware and storing instructions that when executed on the data processing hardware cause the data processing hardware to perform operations comprising;

receiving a voice input from a user of the user device;

displaying an initial recognition output in a graphical user interface based on the voice input, the initial recognition output comprising an initial transcription of the voice input, the initial transcription comprising a misrecognized term;

receiving a selection indication indicating a user selection in the graphical user interface of a portion of the initial recognition output to correct the misrecognized term; and

after receiving the selection indication, displaying a list of candidate transcriptions of the voice input in the graphical user interface, each candidate transcription comprising a respective phonetic similarity to the received voice input.

16. The user device of claim 15, wherein the operations further comprise:

receiving another selection indication indicating another user selection in the graphical user interface of one of the candidate transcriptions displayed in the graphical user interface as a corrected transcription of the voice input; and

displaying one or more results corresponding to the corrected transcription of the voice input in the graphical user interface.

17. The user device of claim 16, wherein the operations further comprise, when displaying the one or more results corresponding to the corrected transcription of the voice input, displaying the corrected transcription of the voice input in the graphical user interface of the user device.

18. The user device of claim 17, wherein the one or more results corresponding to the corrected transcription of the voice input displayed in the graphical user interface are presented below the corrected transcription of the voice input displayed in the graphical user interface.

19. The user device of claim 15, wherein the initial recognition output corresponds to an action to provide output text to an electronic message program executing on the user device.

20. The user device of claim 15, wherein the initial recognition output corresponds to an action of inputting text in a text editor or an electronic message.

21. The user device of claim 15, wherein the operations further comprise, after receiving the selection indication:

receiving one or more text characters inputted by the user through the graphical user interface, the one or more text characters replacing the misrecognized term of the initial transcription of the voice input to provide a corrected transcription of the voice input; and

displaying one or more results corresponding to the corrected transcription of the voice input in the graphical user interface.

22. The user device of claim 21, wherein the one or more results corresponding to the corrected transcription of the voice input displayed in the graphical user interface comprise the one or more text characters inputted by the user through the graphical user interface.

23. The user device of claim 21, wherein the one or more text characters comprise one or more letters inputted by the user through the graphical user interface.

24. The user device of claim 15, wherein the operations further comprise, in response to receiving the selection indication, displaying a keyboard in the graphical user interface.

25. The user device of claim 15, wherein the list of candidate transcriptions of the voice input displayed in the graphical user interface are ranked according to the respective phonetic similarity between each candidate transcription and the received voice input.

26. The user device of claim 15, wherein the list of candidate transcriptions of the voice input displayed in the graphical user interface are based on a popularity of the candidate transcriptions.

27. The user device of claim 15, wherein the operations further comprise, after receiving the selection indication, superimposing a graphical indicator in the graphical user interface over the misrecognized term of the initial transcription.

28. The user device of claim 15, wherein the operations further comprise:

prior to receiving the selection indication indicating the user selection in the graphical user interface, displaying one or more initial results corresponding to the initial transcription of the voice input;

after displaying the list of candidate transcriptions of the voice input in the graphical user interface, receiving another selection indication indicating another user selection in the graphical user interface of one of the candidate transcriptions of the displayed list of candidate transcriptions as a corrected transcription of the voice input; and

updating the one or more initial results corresponding to the initial transcription of the voice input displayed in the graphical user interface with one or more corrected results corresponding to the corrected transcription of the voice input.