Hosted voice recognition system for wireless devices

• US 8,117,268 B2
• Filed: 04/05/2007
• Issued: 02/14/2012
• Est. Priority Date: 04/05/2006
• Status: Expired due to Fees

• Current Assignee: Amazon Technologies, Inc. (Amazon.com, Inc.)
• Original Assignee: Canyon IP Holdings LLC (Intellectual Ventures LLC)
• Inventors: Jablokov, Victor R., Jablokov, Igor R., White, Marc
• Primary Examiner(s): Bates, Kevin
• Assistant Examiner(s): Survillo, Oleg
• Application Number:
  US11/697,074
• Publication Number:
  US 20070239837A1
• Time in Patent Office: 1,776 Days
• Field of Search: 709/206, 709/217, 709/219, 704/270.1, 704/235, 455/412.1, 455/412.2
• US Class Current: 709/206
• CPC Class Codes: G06Q 30/0251 Targeted advertisements
• CPC Class Codes: G10L 13/00 Speech synthesis; Text to speech systems
• CPC Class Codes: G10L 15/26 Speech to text systems G10L15/08 takes precedence
• CPC Class Codes: G10L 15/30 Distributed recognition, e.g. in client-server systems, for mobile phones or network applications
• CPC Class Codes: H04L 51/066 Format adaptation, e.g. format conversion or compression
• CPC Class Codes: H04L 51/58 Message adaptation for wireless communication

Abstract

Methods, systems, and software for converting the audio input of a user of a hand-held client device or mobile phone into a textual representation by means of a backend server accessed by the device through a communications network. The text is then inserted into or used by an application of the client device to send a text message, instant message, email, or to insert a request into a web-based application or service. In one embodiment, the method includes the steps of initializing or launching the application on the device; recording and transmitting the recorded audio message from the client device to the backend server through a client-server communication protocol; converting the transmitted audio message into the textual representation in the backend server; and sending the converted text message back to the client device or forwarding it on to an alternate destination directly from the server.

Claims

1. A method for converting an utterance, received by a hand-held mobile communication device, into a textual representation for display on the hand-held mobile communication device and delivery via text message to one or more recipients, comprising the steps of:

initializing a hand-held mobile communication device, via a client application loaded on the hand-held mobile communication device, so that the hand-held mobile communication device is capable of communicating with a backend server via a data channel of the hand-held mobile communication device;

upon receipt of the utterance by the hand-held mobile communication device, recording and storing an audio message, representative of the utterance, in the hand-held mobile communication device in the form of binary audio data;

transmitting, via the data channel, the recorded and stored binary audio data, representing the utterance, from the hand-held mobile communication device to a servlet of the backend server through a client-server communication protocol;

in response to transmitting the recorded and stored binary audio data, transmitting a receipt, identifying the binary audio data, from the server to the hand-held mobile communication device;

converting the transmitted binary audio data into a textual representation of the utterance in the backend server;

sending the receipt from the hand-held mobile communication device to a servlet of the backend server;

in response to sending the receipt from the hand-held mobile communication device to the server, sending the converted textual representation of the utterance from the backend server back to the hand-held mobile communication device; and

forwarding the converted textual representation from the hand-held mobile communication device to one or more recipients.

2. The method of claim 1, further comprising displaying the textual representation on the handheld mobile communication device.

3. The method of claim 1, wherein the textual representation comprises a transcription of the utterance that was represented by the binary audio data.

4. The method of claim 3, wherein the transcription comprises a filtered transcription.

5. The method of claim 3, wherein the transcription comprises an unfiltered transcription.

6. The method of claim 1, wherein the textual representation comprises both a filtered transcription of the utterance as represented by the binary audio data, and an unfiltered transcription of the utterance as represented by the binary audio data.

7. The method of claim 1, wherein the recorded and stored binary audio data, representing the utterance, is transmitted in the body of an HTTP request.

8. The method of claim 7, wherein the HTTP request is sent over the Internet.

9. The method of claim 1, wherein the sending of the receipt to the server comprises transmitting the receipt to the server in the header of an HTTP request.

10. The method of claim 9, wherein the HTTP request is sent over the Internet.

11. The method of claim 1, wherein the receipt is sent in the body of an HTTP response.

12. The method of claim 1, wherein the converted textual representation of the utterance is sent in the body of an HTTP response.

13. The method of claim 1, further comprising the step of initializing the handheld mobile communication device by sending an HTTP request to the server and receiving, in response, a session ID.

14. The method of claim 13, wherein the session ID is received in the body of an HTTP response.

15. The method of claim 13, further comprising the step of maintaining a session corresponding to the Session ID by sending, from the handheld mobile communication device, the session ID to the server in a Ping request.

16. The method of claim 15, wherein the session ID is located at the end of a URL of the Ping request.

17. The method of claim 1, wherein the step of converting the transmitted binary audio data into a textual representation of the utterance is performed using a speech recognition algorithm.

18. The method of claim 1, further comprising the steps of identifying the spoken audio input as an SMS message and filtering the transcription using an SMS filter.

19. The method of claim 1, further comprising the step of filtering the transcription using an obscenity filter.

20. The method of claim 1, further comprising the step of filtering the transcription using a number filter.

21. The method of claim 1, further comprising the step of filtering the transcription using a date filter.

22. The method of claim 1, further comprising the step of filtering the transcription using a currency filter.

23. The method of claim 1, further comprising the steps of applying an ad filter to the transcription and, based thereon, determining advertising for display on the handheld mobile communication device.

24. The method of claim 1, wherein the data that identifies the received binary audio data is transmitted in the body of an HTTP response.

25. The method of claim 1, wherein the data that represents the transcription of the spoken audio data is transmitted in the body of an HTTP response.

26. The method of claim 1, further comprising the step of receiving an HTTP request from the handheld mobile communication device and transmitting, in response thereto, a session ID.

27. The method of claim 26, wherein the session ID is included in the body of an HTTP response.

28. The method of claim 26, further comprising the step of maintaining a session corresponding to the session ID when the session ID is received in a Ping request from the handheld mobile communication device.

29. The method of claim 1, further comprising the steps of converting the data representing the transcription into binary audio data using a text-to-speech engine (TTS) and transmitting the resulting binary audio data to the handheld mobile communication device for playing.

30. A method for converting spoken audio input, received by a hand-held mobile communication device, into a textual representation for display on the hand-held mobile communication device and delivery via text message to one or more recipients, comprising the steps of:

initializing a hand-held mobile communication device, via a client application loaded on the hand-held mobile communication device, so that the hand-held mobile communication device is capable of communicating with a backend server via a data channel of the hand-held mobile communication device;

upon receipt of an utterance by the hand-held mobile communication device, recording and storing an audio message, representative of the utterance, in the hand-held mobile communication device in the form of binary audio data;

transmitting, via the data channel, the utterance, from the hand-held mobile communication device to a servlet of the backend server by transmitting the recorded and stored binary audio data from the handheld mobile communication device over a data channel using a client-server communication protocol;

receiving a receipt for the utterance sent to the server by receiving, at the handheld mobile communication device over a data channel using a client-server communication protocol, data that identifies the transmitted binary audio data;

sending the receipt from the hand-held mobile communication device to the a servlet of the backend server by transmitting, from the handheld mobile communication device over a data channel using a client-server communication protocol, data that identifies the transmitted binary audio data; and

receiving a textual representation of the utterance by receiving, at the handheld mobile communication device over a data channel using a client-server communication protocol, data that represents a transcription of the utterance; and

forwarding the textual representation from the hand-held mobile communication device to one or more recipients;

wherein the receipt is received in response to the sending of the utterance to the backend server; and

wherein the textual representation is received in response to the sending of the receipt to the backend server.

31. The method of claim 30, wherein the data channel over which the binary audio data is submitted is established with a wireless carrier network by the handheld mobile device.

32. The method of claim 31, wherein the handheld mobile device establishes the data channel after the utterance is converted to binary audio data.

33. The method of claim 31, wherein the handheld mobile device establishes the data channel before the utterance is converted to binary audio data.

34. The method of claim 30, wherein a hypertext transfer protocol is used in said steps of sending the utterance, receiving the receipt, sending the receipt, and receiving the textual representation.

35. The method of claim 30, further comprising displaying the textual representation on the handheld mobile communication device.

36. The method of claim 30, wherein the transcription comprises a transcription that is filtered using at least one of an SMS filter, an obscenity filter, a number filter, a date filter, and a currency filter.

37. The method of claim 30, wherein the transcription comprises an unfiltered transcription.

38. The method of claim 30, wherein the textual representation comprises both a filtered transcription and an unfiltered transcription.

39. The method of claim 30, wherein the sending of the utterance to the server comprises transmitting to the server the binary audio data in the body of an HTTP request.

40. The method of claim 39, wherein the HTTP request is sent over the Internet.

41. The method of claim 30, wherein the sending of the receipt to the server comprises transmitting to the server, in the header of an HTTP request, the data that identifies the transmitted binary audio data.

42. The method of claim 41, wherein the HTTP request is sent over the Internet.

43. The method of claim 30, wherein the receiving of a receipt for the utterance sent to the server comprises receiving the receipt in the body of an HTTP response.

44. The method of claim 30, wherein the receiving of a textual representation of the utterance comprises receiving the textual representation in the body of an HTTP response.

45. The method of claim 30, further comprising the step of initializing the handheld mobile communication device by sending an HTTP request to the server and receiving, in response thereto, a session ID.

46. The method of claim 45, wherein the session ID is received in the body of an HTTP response.

47. The method of claim 45, further comprising the step of maintaining a session corresponding to the session ID by sending, from the handheld mobile communication device, the session ID to the server in a Ping request.

48. The method of claim 47, wherein the session ID is located at the end of a URL of the Ping request.

49. The method of claim 30, further comprising the step of receiving binary audio data at the handheld mobile communication device, the binary audio data generated based on the textual representation using a text-to-speech engine (TTS).

50. The method of claim 49, further comprising the step of playing, at the handheld mobile communication device, the binary audio data, generated based on the textual representation, using a text-to-speech engine (TTS).

51. The method of claim 30, further comprising the steps displaying advertisements on the handheld mobile communication device that are selected based on keywords contained in the transcription.

52. The method of claim 30, further comprising the step of displaying a list of locations that are proximate to the position of the client device, which locations are determined to be related to a target of interest based on keywords contained in the transcription.

53. The method of claim 30, wherein the handheld mobile communication device comprises a mobile phone.