Facilitating presentation by mobile device of additional content for a word or phrase upon utterance thereof

• US 8,335,829 B1
• Filed: 11/09/2009
• Issued: 12/18/2012
• Est. Priority Date: 08/22/2007
• Status: Active Grant

• Current Assignee: Amazon Technologies, Inc. (Amazon.com, Inc.)
• Original Assignee: Canyon IP Holdings LLC (Intellectual Ventures LLC)
• Inventors: Jablokov, Victor Roditis, Jablokov, Igor Roditis, Terrell, James Richard II, Paden, Scott Edward
• Primary Examiner(s): Zhang, Shirley
• Assistant Examiner(s): Survillo, Oleg
• Application Number:
  US12/614,637
• Time in Patent Office: 1,135 Days
• Field of Search: 709/206, 709/217, 709/219, 704/270.1, 704/235, 707/705, 707/736
• US Class Current: 709/206
• CPC Class Codes: G06F 16/951 Indexing; Web crawling techniques
• CPC Class Codes: G06F 16/9535 Search customisation based on user profiles and personalisation
• CPC Class Codes: G06F 16/9538 Presentation of query results
• CPC Class Codes: G06F 16/957 Browsing optimisation, e.g. caching or content distillation
• CPC Class Codes: G06F 16/9577 Optimising the visualization of content, e.g. distillation of HTML documents
• CPC Class Codes: G06F 3/167 Audio in a user interface, e.g. using voice commands for navigating, audio feedback
• CPC Class Codes: G06Q 10/107 Computer-aided management of electronic mailing [e-mailing]
• CPC Class Codes: G06Q 30/02 Marketing; Price estimation or determination; Fundraising
• CPC Class Codes: G06Q 30/0256 User search
• CPC Class Codes: G06Q 30/0269 based on user profile or attribute
• CPC Class Codes: G06Q 30/0271 Personalized advertisement
• CPC Class Codes: G06Q 30/0631 Item recommendations
• CPC Class Codes: G10L 13/033 Voice editing, e.g. manipulating the voice of the synthesiser
• CPC Class Codes: G10L 13/04 Details of speech synthesis systems, e.g. synthesiser structure or memory management
• CPC Class Codes: G10L 15/26 Speech to text systems G10L15/08 takes precedence
• CPC Class Codes: G10L 2015/088 Word spotting
• CPC Class Codes: G10L 21/00 Speech or voice signal processing techniques to produce another audible or non-audible signal, e.g. visual or tactile, in order to modify its quality or its intelligibility G10L19/00 takes precedence
• CPC Class Codes: H04L 51/00 User-to-user messaging in packet-switching networks, transmitted according to store-and-forward or real-time protocols, e.g. e-mail
• CPC Class Codes: H04L 67/02 based on web technology, e.g. hypertext transfer protocol [HTTP]
• CPC Class Codes: H04L 69/329 in the application layer [OSI layer 7]
• CPC Class Codes: H04M 1/7243 : with interactive means for internal management of messages
• CPC Class Codes: H04M 2201/40 : using speech recognition speech recognition per se G10L15/00
• CPC Class Codes: H04M 3/4938 : comprising a voice browser which renders and interprets, e.g. VoiceXML

Abstract

A method for presenting additional content for a word that is part of a message, and that is presented by a mobile communication device, includes the steps of: presenting the message, including emphasizing one or more words for which respective additional content is available for presenting by the mobile communication device; receiving an utterance that includes an emphasized word for which additional content is available for presenting by the mobile communication device; and presenting the additional content for the emphasized word included in the utterance received by the mobile communication device. These steps are performed by the mobile communication device.

Claims

1. A computer-implemented method of facilitating presentation of additional content for a word that is part of a transcribed utterance, the computer-implemented method comprising:

under control of one or more computing devices configured with specific computer executable instructions,

(a) first,

(i) receiving first audio data communicated from a mobile communication device;

(ii) transcribing the first audio data to first text;

(iii) screening the first text to identify one or more words that are appropriate for searching;

(iv) searching for additional content in one or more databases based on the one or more words identified as being appropriate for searching;

(v) selecting a word for which respective additional content is found in the searching;

(vi) communicating the first text to the mobile communication device along with information identifying the selected word for which respective additional content is found in the searching; and

(vii) maintaining in a database an association between the selected word, the respective additional content for the selected word, and an identifier of the mobile communication device;

(b) thereafter,

(i) receiving second audio data communicated from the mobile communication device and the identifier of the mobile communication device, the second audio data representing an utterance that includes the selected word;

(ii) transcribing the second audio data to second text;

(iii) determining that the identifier of the mobile communication device is maintained in the database and, based at least in part upon the identifier of the mobile communication device being maintained in the database, comparing the second text to the selected word maintained in the database in association with the identifier of the mobile communication device; and

(iv) based at least in part on the selected word being present in the second text, communicating to the mobile communication device the respective additional content associated with the selected word, wherein the communicated respective additional content is for presentation by the mobile communication device.

2. The computer-implemented method of claim 1, wherein communicating the first text to the mobile communication device along with information identifying the selected word for which respective additional content is found in the searching comprises highlighting the selected word such that the selected word is emphasized when displayed on the mobile communication device.

3. The computer-implemented method of claim 1, wherein communicating the first text to the mobile communication device along with information identifying the selected word for which respective additional content is found in the searching comprises configuring an audio presentation of the first text to include pronunciation emphasis of the selected word in the first text for which respective additional content is found in the searching.

4. The computer-implemented method of claim 1, wherein communicating the first text to the mobile communication device along with information identifying the selected word for which respective additional content is found in the searching comprises configuring an audio presentation of the first text to include aural cueing for the selected word in the first text for which respective additional content is found in the searching.

5. The computer-implemented method of claim 1, wherein the first audio data that is received from the mobile communication device includes an indication that the first audio data is intended to be at least a portion of the text of a message.

6. The computer-implemented method of claim 5, wherein the indication comprises that the message is an instant message.

7. The computer-implemented method of claim 5, wherein the indication comprises that the message is to be communicated in accordance with a text messaging protocol.

8. The computer-implemented method of claim 5, further comprising filtering the first text that is transcribed from the first audio data based on the indication.

9. A non-transitory computer-readable medium whose contents configure one or more computing systems to perform a method of facilitating presentation of additional content for a word that is part of a transcribed utterance intended for a message, the method comprising:

(a) first,

(i) receiving first audio data communicated from a mobile communication device;

(ii) transcribing the first audio data to first text;

(iii) screening the first text to identify one or more words that are appropriate for searching;

(iv) searching for additional content in one or more databases based on the one or more words identified as being appropriate for searching;

(v) selecting a word for which respective additional content is found in the searching;

(vi) communicating the first text to the mobile communication device along with information identifying the selected word for which respective additional content is found in the searching; and

(vii) maintaining in a database an association between the selected word, the respective additional content for the selected word, and an identifier of the mobile communication device;

(b) thereafter,

(i) receiving second audio data communicated from the mobile communication device and the identifier of the mobile communication device, the second audio data representing an utterance that includes the selected word;

(ii) transcribing the second audio data to second text;

(iii) determining that the identifier of the mobile communication device is maintained in the database and, based at least in part upon the identifier of the mobile communication device being maintained in the database, comparing the second text to the selected word maintained in the database in association with the identifier of the mobile communication device; and

(iv) based at least in part on the selected word being present in the second text, communicating to the mobile communication device the respective additional content associated with the selected word, wherein the communicated respective additional content is for presentation by the mobile communication device.

10. The non-transitory computer readable medium of claim 9, wherein the searching for additional content includes searching based on the selected word using an Internet search engine and analyzing the results provided in response thereto.

11. The non-transitory computer readable medium of claim 9, wherein the respective additional content comprises a web address for passing to a web browser of the mobile communication device.

12. The non-transitory computer readable medium of claim 9, wherein the respective additional content comprises a URL for presentation by the mobile communication device.

13. The non-transitory computer readable medium of claim 9, wherein the searching includes sending a keyword to a third party SMS service and analyzing the results provided in response thereto.

14. The non-transitory computer readable medium of claim 9, wherein searching for additional content comprises communicating a keyword to a third party SMS service, in response to which the respective additional content found in the searching is provided.

15. The non-transitory computer readable medium of claim 9, wherein the utterance further includes a cue that the selected word is included in the utterance for communicating the respective additional content for the selected word.

16. The non-transitory computer readable medium of claim 9, wherein the association between the selected word, the respective additional content for the selected word, and the identifier of the mobile communication device is maintained in the database for an extended period of time comprising a plurality of days or weeks, whereby additional content may be provided for the selected word communicated to the mobile communication device even though a plurality of days have passed since the selected word was communicated to the mobile communication device.

17. A system for facilitating presentation of additional content for a word that is part of a transcribed utterance, the system comprising one or more processors configured to:

(a) first,

(i) receive first audio data communicated from a mobile communication device;

(ii) transcribe the first audio data to first text;

(iii) screen the first text to identify one or more words that are appropriate for searching;

(iv) search for additional content in one or more databases based on the one or more words identified as being appropriate for searching;

(v) select a word for which respective additional content is found in the searching;

(vi) communicate the first text to the mobile communication device along with information identifying the selected word for which respective additional content is found in the searching; and

(vii) maintain in a database an association between the selected word, the respective additional content for the selected word, and an identifier of the mobile communication device;

(b) thereafter,

(i) receive second audio data communicated from the mobile communication device and the identifier of the mobile communication device, the second audio data representing an utterance that includes the selected word;

(ii) transcribe the second audio data to second text;

(iii) determine that the identifier of the mobile communication device is maintained in the database and, based at least in part upon the identifier of the mobile communication device being maintained in the database, compare the second text to the selected word maintained in the database in association with the identifier of the mobile communication device; and

(iv) based at least in part on the selected word being present in the second text, communicate the respective additional content associated with the selected word to the mobile communication device, wherein the communicated respective additional content is for presentation by the mobile communication device.

18. The system of claim 17, wherein the respective additional content comprises a web address.

19. The system of claim 17, wherein the respective additional content comprises a response received after sending a keyword to a third party SMS service.