Hosted voice recognition system for wireless devices

• US 9,542,944 B2
• Filed: 04/13/2015
• Issued: 01/10/2017
• Est. Priority Date: 04/05/2006
• Status: Active Grant

• Current Assignee: Amazon Technologies, Inc. (Amazon.com, Inc.)
• Original Assignee: Amazon Technologies, Inc. (Amazon.com, Inc.)
• Inventors: Jablokov, Victor R., Jablokov, Igor R., White, Marc
• Primary Examiner(s): Godbold, Douglas
• Application Number:
  US14/685,528
• Publication Number:
  US 20160217786A1
• Time in Patent Office: 638 Days
• US Class Current: 1/1
• CPC Class Codes: G06Q 30/0251 Targeted advertisements
• CPC Class Codes: G10L 13/00 Speech synthesis; Text to speech systems
• CPC Class Codes: G10L 15/26 Speech to text systems G10L15/08 takes precedence
• CPC Class Codes: G10L 15/30 Distributed recognition, e.g. in client-server systems, for mobile phones or network applications
• CPC Class Codes: H04L 51/066 Format adaptation, e.g. format conversion or compression
• CPC Class Codes: H04L 51/58 Message adaptation for wireless communication

Abstract

Methods, systems, and software for converting the audio input of a user of a handheld client device or mobile phone into a textual representation by means of a backend server accessed by the device through a communications network. The text is then inserted into or used by an application of the client device to send a text message, instant message, email, or to insert a request into a web-based application or service. In one embodiment, the method includes the steps of initializing or launching the application on the device; recording and transmitting the recorded audio message from the client device to the backend server through a client-server communication protocol; converting the transmitted audio message into the textual representation in the backend server; and sending the converted text message back to the client device or forwarding it on to an alternate destination directly from the server.

Claims

1. A computer-implemented method comprising:

under control of a first computing device executing specific computer-executable instructions,

receiving a first portion of audio input captured via a microphone;

in response to receiving the first portion of the audio input, transmitting to a second computing device, first data representing the first portion of the audio input;

receiving a next portion of the audio input, the next portion captured via the microphone directly following the first portion of the audio input;

in response to receiving the next portion of the audio input, transmitting to the second computing device, next data representing the next portion of the audio input;

receiving from the second computing device, first partial speech recognition results determined from the first data representing the first portion of the audio input,

wherein the first partial speech recognition results are received prior to the transmitting of the next data;

receiving from the second computing device, next partial speech recognition results determined from the next data representing the next portion of the audio input; and

initiating presentation, on a display of the first computing device, of the first partial speech recognition results,

wherein the presentation of the first partial speech recognition results on the display of the first computing device is initiated by the first computing device prior to the receiving of the next partial speech recognition results.

2. The computer-implemented method of claim 1, further comprising transmitting, to the second computing device, a first identifier associated with the audio input, wherein the first partial speech recognition results are determined from the first data representing the first portion of the audio input and are retrieved using the first identifier.

3. The computer-implemented method of claim 2, further comprising transmitting, to the second computing device, a second identifier associated with the audio input, wherein the next partial speech recognition results are determined from the next data representing the next portion of the audio input and are retrieved using the second identifier.

4. The computer-implemented method of claim 1, further comprising initiating presentation, on the display of the first computing device, of the next partial speech recognition results.

5. The computer-implemented method of claim 4, wherein the first partial speech recognition results are presented prior to the initiating presentation of the next partial speech recognition results.

6. The computer-implemented method of claim 5, wherein presentation of the first partial speech recognition results is completed prior to initiating presentation of the next partial speech recognition results.

7. The computer-implemented method of claim 1, wherein the next partial speech recognition results are determined from the first data representing the first portion of the audio input and the next data representing the next portion of the audio input.

8. A computer-readable, non-transitory storage medium storing computer executable instructions that, when executed by a first computing device, configure the first computing device to perform operations comprising:

receiving a first portion of audio input captured via a microphone of the first computing device;

in direct response to receiving the first portion of the audio input, transmitting to a second computing device, first data representing the first portion of the audio input;

receiving a next portion of the audio input that directly follows the first portion of the audio input, the next portion captured via the microphone;

in direct response to receiving the next portion of the audio input, transmitting to the second computing device, next data representing the next portion of the audio input;

receiving from the second computing device, first partial speech recognition results determined from the first data representing the first portion of the audio input,

wherein the first partial speech recognition results are received prior to the transmitting of the next data;

receiving from the second computing device, next partial speech recognition results determined from the next data representing the next portion of the audio input; and

initiating display of the first partial speech recognition results on a display of the first computing device,

wherein the display of the first partial speech recognition results on the display of the first computing device is initiated by the first computing device prior to the receiving of the next partial speech recognition results.

9. The computer-readable, non-transitory storage medium of claim 8, wherein the next partial speech recognition results are determined from the first data representing the first portion of the audio input and the next data representing the next portion of the audio input.

10. The computer-readable, non-transitory storage medium of claim 8, wherein the operations further comprise:

prior to receiving the first partial speech recognition results,

transmitting, to the second computing device, a first identifier associated with the audio input, wherein the first partial speech recognition results are determined from the first data representing the first portion of the audio input and are retrieved using the first identifier.

11. The computer-readable, non-transitory storage medium of claim 8, wherein the operations further comprise:

prior to receiving the next partial speech recognition results,

transmitting, to the second computing device, a second identifier associated with the audio input, wherein the next partial speech recognition results are determined from the next data representing the next portion of the audio input and are retrieved using the second identifier.

12. The computer-readable, non-transitory storage medium of claim 8, wherein the operations further comprise:

initiating capture of the audio input via the microphone when a button of the first computing device is activated; and

completing capture of the audio input via the microphone when the button is deactivated.

13. The computer-readable, non-transitory storage medium of claim 8, wherein:

the audio input comprises a plurality of spoken words, and

the first partial speech recognition results comprise at least one word of the plurality of spoken words represented as text.

14. The computer-readable, non-transitory storage medium of claim 8, wherein:

the audio input comprises a spoken word, and

the first partial speech recognition results comprise at least one letter of the spoken word represented as text.

15. A system comprising:

an electronic data store configured to at least store computer-executable instructions; and

a second computing device including at least one processor, the second computing device in communication with the electronic data store and configured to execute the computer-executable instructions to at least;

receive, from a first computing device, first data representing a first portion of an audio input;

in response to receiving the first data, determine first partial speech recognition results from the first data;

receive, from the first computing device, next data representing a next portion of the audio input that directly follows the first portion of the audio input;

in response to receiving the next data, determine next partial speech recognition results from the next data;

prior to determining the next partial speech recognition results from the next data, transmit the first partial speech recognition results to the first computing device, for display by the first computing device; and

transmit the next partial speech recognition results to the first computing device for display by the first computing device,

wherein, at the first computing device, the display of the first partial speech recognition results by the first computing device is initiated prior to the display of the next partial speech recognition results.

16. The system of claim 15, wherein the audio input is captured by a microphone of the first computing device.

17. The system of claim 15, wherein display by the first computing device of the first partial speech recognition results is completed prior to initiating display of the next partial speech recognition results.

18. The system of claim 15, wherein the second computing device is further configured to execute the computer-executable instructions to at least:

receive, from the first computing device, a first identifier associated with the audio input, wherein the first partial speech recognition results are determined from the first data and are retrieved using the first identifier.

19. The system of claim 15, wherein the second computing device is further configured to execute the computer-executable instructions to at least:

receive, from the first computing device, a second identifier associated with the audio input, wherein the next partial speech recognition results are determined from the next data and are retrieved using the second identifier.

20. The system of claim 15, wherein the next partial speech recognition results are determined from the first data and the next data.