Allowing spelling of arbitrary words

• US 10,579,730 B1
• Filed: 01/25/2019
• Issued: 03/03/2020
• Est. Priority Date: 01/06/2016
• Status: Active Grant

• Current Assignee: Google LLC (Alphabet Inc.)
• Original Assignee: Google LLC (Alphabet Inc.)
• Inventors: Cherepanov, Evgeny A., Skobeltsyn, Gleb, Foerster, Jakob, Aleksic, Petar, Michaely, Assaf Hurwitz
• Primary Examiner(s): Patel, Shreyans A
• Application Number:
  US16/258,230
• Time in Patent Office: 403 Days
• CPC Class Codes: G06F 3/167 Audio in a user interface, e.g. using voice commands for navigating, audio feedback
• CPC Class Codes: G06F 40/232 Orthographic correction, e.g. spell checking or vowelisation
• CPC Class Codes: G10L 15/187 Phonemic context, e.g. pronunciation rules, phonotactical constraints or phoneme n-grams
• CPC Class Codes: G10L 15/19 Grammatical context, e.g. disambiguation of the recognition hypotheses based on word sequence rules
• CPC Class Codes: G10L 15/197 Probabilistic grammars, e.g. word n-grams
• CPC Class Codes: G10L 15/22 Procedures used during a speech recognition process, e.g. man-machine dialogue
• CPC Class Codes: G10L 15/26 Speech to text systems G10L15/08 takes precedence
• CPC Class Codes: G10L 15/30 Distributed recognition, e.g. in client-server systems, for mobile phones or network applications
• CPC Class Codes: G10L 15/32 Multiple recognisers used in sequence or in parallel; Score combination systems therefor, e.g. voting systems
• CPC Class Codes: G10L 2015/086 Recognition of spelled words
• CPC Class Codes: G10L 2015/223 Execution procedure of a spoken command

Abstract

Methods, systems, and apparatus, including computer programs encoded on computer storage media, for natural language processing. One of the methods includes receiving a first voice input from a user device; generating a first recognition output; receiving a user selection of one or more terms in the first recognition output; receiving a second voice input spelling a correction of the user selection; determining a corrected recognition output for the selected portion; and providing a second recognition output that merges the first recognition output and the corrected recognition output.

Claims

1. A method comprising:

transmitting, by data processing hardware, a transcription of a voice query over a network to a user device, the transcription of the voice query when received by the user device causing the user device to display the transcription of the voice query in a user interface of the user device;

receiving, at the data processing hardware, a correction input over the network from the user device, the correction input comprising;

a selection indication indicating a user selection in the user interface of a misrecognized term of the transcription of the voice query displayed in the user interface; and

a voice input from a user of the user device, the voice input including a series of letters that spells out a corrected term to replace the misrecognized term of the transcription of the voice query;

in response to receiving the correction input from the user device, generating, by the data processing hardware, an updated transcription of the voice query based on the voice input from the user of the user device; and

transmitting, by the data processing hardware, the updated transcription of the voice query over the network to the user device.

2. The method of claim 1, further comprising, in response to receiving the correction input from the user device:

obtaining, by the data processing hardware, one or more search results responsive to the updated transcription of the voice query; and

transmitting, by the data processing hardware, the one or more search results over the network to the user device.

3. The method of claim 2, wherein the one or more search results when received by the user device cause the user device to display the one or more obtained search results responsive to the updated transcription of the voice query in the user interface.

4. The method of claim 2, wherein the one or more search results when received by the user device cause the user device causing the user device to update one or more initial search results displayed in the user interface of the user device with the one or more obtained search results responsive to the updated transcription of the voice query.

5. The method of claim 1, wherein the updated transcription of the voice query when received by the user device causes the user device to display the updated transcription of the voice query in the user interface of the user device.

6. The method of claim 5, wherein the user device displays the updated transcription of the voice query by replacing the misrecognized term of the transcription of the voice query displayed in the user interface with the corrected term spelled out by the series of letters in the voice input from the user.

7. The method of claim 1, further comprising, performing, by the data processing hardware, a particular task responsive to the updated transcription of the voice query.

8. The method of claim 1, wherein the user device is configured to receive the selection indication indicating the user selection in the user interface by receiving a touch input by the user in the user interface via a finger or stylus that selects the misrecognized term of the transcription of the voice query displayed in the user interface.

9. The method of claim 1, wherein the user device is configured to superimpose a graphical indicator in the user interface that indicates the misrecognized term in response to the selection indication indicating the user selection in the user interface of the misrecognized term of the transcription of the voice query.

10. The method of claim 1, wherein the user device comprises a microphone configured to capture the voice input from the user.

20. The system of claim 1, wherein the user device comprises a microphone configured to capture the voice input from the user.

11. A system comprising:

data processing hardware; and

memory hardware in communication with the data processing hardware and storing instructions that when executed by the data processing hardware causes the data processing hardware to perform operations comprising;

transmitting a transcription of a voice query over a network to a user device, the transcription of the voice query when received by the user device causing the user device to display the transcription of the voice query in a user interface of the user device;

receiving a correction input over the network from the user device, the correction input comprising;

a selection indication indicating a user selection in the user interface of a misrecognized term of the transcription of the voice query displayed in the user interface; and

a voice input from a user of the user device, the voice input including a series of letters that spells out a corrected term to replace the misrecognized term of the transcription of the voice query;

in response to receiving the correction input from the user device, generating an updated transcription of the voice query based on the voice input from the user of the user device; and

transmitting the updated transcription of the voice query over the network to the user device.

12. The system of claim 11, wherein the operations further comprise, in response to receiving the correction input from the user device:

obtaining one or more search results responsive to the updated transcription of the voice query; and

transmitting the one or more search results over the network to the user device.

13. The system of claim 12, wherein the one or more search results when received by the user device cause the user device to display the one or more obtained search results responsive to the updated transcription of the voice query in the user interface.

14. The system of claim 12, wherein the one or more search results when received by the user device cause the user device causing the user device to update one or more initial search results displayed in the user interface of the user device with the one or more obtained search results responsive to the updated transcription of the voice query.

15. The system of claim 11, wherein the updated transcription of the voice query when received by the user device causes the user device to display the updated transcription of the voice query in the user interface of the user device.

16. The system of claim 15, wherein the user device displays the updated transcription of the voice query by replacing the misrecognized term of the transcription of the voice query displayed in the user interface with the corrected term spelled out by the series of letters in the voice input from the user.

17. The system of claim 11, wherein the operations further comprise, performing a particular task responsive to the updated transcription of the voice query.

18. The system of claim 11, wherein the user device is configured to receive the selection indication indicating the user selection in the user interface by receiving a touch input by the user in the user interface via a finger or stylus that selects the misrecognized term of the transcription of the voice query displayed in the user interface.

19. The system of claim 11, wherein the user device is configured to superimpose a graphical indicator in the user interface that indicates the misrecognized term in response to the selection indication indicating the user selection in the user interface of the misrecognized term of the transcription of the voice query.