Facilitating presentation by mobile device of additional content for a word or phrase upon utterance thereof

• US 8,140,632 B1
• Filed: 11/09/2009
• Issued: 03/20/2012
• Est. Priority Date: 08/22/2007
• Status: Active Grant

• Current Assignee: Amazon Technologies, Inc. (Amazon.com, Inc.)
• Original Assignee: Canyon IP Holdings LLC (Intellectual Ventures LLC)
• Inventors: Jablokov, Victor Roditis, Jablokov, Igor Roditis, Terrell, James Richard II, Paden, Scott Edward
• Primary Examiner(s): Burgess, Glenton B
• Assistant Examiner(s): SURVILLO, OLEG
• Application Number:
  US12/614,652
• Time in Patent Office: 862 Days
• Field of Search: 709/206, 709/217, 709/219, 704/270.1, 704/235, 707/705, 707/736
• US Class Current: 709/206
• CPC Class Codes: G06F 16/951 Indexing; Web crawling techniques
• CPC Class Codes: G06F 16/9535 Search customisation based on user profiles and personalisation
• CPC Class Codes: G06F 16/9538 Presentation of query results
• CPC Class Codes: G06F 16/957 Browsing optimisation, e.g. caching or content distillation
• CPC Class Codes: G06F 16/9577 Optimising the visualization of content, e.g. distillation of HTML documents
• CPC Class Codes: G06F 3/167 Audio in a user interface, e.g. using voice commands for navigating, audio feedback
• CPC Class Codes: G06Q 10/107 Computer-aided management of electronic mailing [e-mailing]
• CPC Class Codes: G06Q 30/02 Marketing; Price estimation or determination; Fundraising
• CPC Class Codes: G06Q 30/0256 User search
• CPC Class Codes: G06Q 30/0269 based on user profile or attribute
• CPC Class Codes: G06Q 30/0271 Personalized advertisement
• CPC Class Codes: G06Q 30/0631 Item recommendations
• CPC Class Codes: G10L 13/033 Voice editing, e.g. manipulating the voice of the synthesiser
• CPC Class Codes: G10L 13/04 Details of speech synthesis systems, e.g. synthesiser structure or memory management
• CPC Class Codes: G10L 15/26 Speech to text systems G10L15/08 takes precedence
• CPC Class Codes: G10L 2015/088 Word spotting
• CPC Class Codes: G10L 21/00 Speech or voice signal processing techniques to produce another audible or non-audible signal, e.g. visual or tactile, in order to modify its quality or its intelligibility G10L19/00 takes precedence
• CPC Class Codes: H04L 51/00 User-to-user messaging in packet-switching networks, transmitted according to store-and-forward or real-time protocols, e.g. e-mail
• CPC Class Codes: H04L 67/02 based on web technology, e.g. hypertext transfer protocol [HTTP]
• CPC Class Codes: H04L 69/329 in the application layer [OSI layer 7]
• CPC Class Codes: H04M 1/7243 : with interactive means for internal management of messages
• CPC Class Codes: H04M 2201/40 : using speech recognition
• CPC Class Codes: H04M 3/4938 : comprising a voice browser which renders and interprets, e.g. VoiceXML

Abstract

A method for presenting additional content for a word that is part of a message, and that is presented by a mobile communication device, includes the steps of: presenting the message, including emphasizing one or more words for which respective additional content is available for presenting by the mobile communication device; receiving an utterance that includes an emphasized word for which additional content is available for presenting by the mobile communication device; and presenting the additional content for the emphasized word included in the utterance received by the mobile communication device. These steps are performed by the mobile communication device.

Claims

1. A method for presenting additional content for a word that is part of a message and that is presented by a mobile communication device, comprising the steps of:

(a) first, by a server,

(i) receiving an identifier of the mobile communication device and audio data communicated from the mobile communication device, the audio data representing an utterance that is intended to be at least a portion of the text of a message that is to be sent from the mobile communication device to a recipient;

(ii) transcribing the utterance to text based on the audio data;

(iii) screening the text to identify words that are appropriate for searching;

(iv) searching for additional content in one or more databases based on words identified as being appropriate for searching;

(v) identifying the words for which respective additional content is found in the searching;

(vi) communicating the transcribed text with the identified words to the mobile communication device such that the identified words are emphasized when the text is presented by the mobile communication device; and

(vii) maintaining in a database an association between the identified words, the respective additional content found for such words during searching, and an identifier of the mobile communication device;

(b) second, by the mobile communication device, presenting the message, including emphasizing the identified words for which the respective additional content is available for presenting by the mobile communication device;

(c) third, by the server,

(i) receiving an identifier of the mobile communication device and audio data communicated from the mobile communication device, the audio data representing an utterance received by the mobile communication device, the utterance including an identified word that is maintained in the database in association with the identifier of the mobile communication device;

(ii) transcribing the utterance to text based on the audio data;

(iii) comparing the transcribed text to the identified words maintained in the database in association with the identifier of the mobile communication device; and

(iv) communicating the respective additional content to the mobile communication device that is maintained in the database in association with an identified word found in the transcribed text based on the comparing; and

(d) fourth, by the mobile communication device, presenting the additional content received from the server.

2. The method of claim 1, wherein said step of communicating to the mobile communication device the respective additional content that is maintained in the database in association with an identified word included in the transcribed text includes communicating to the mobile communication device the respective additional content that is maintained in the database in association with each identified word that is found in the transcribed text based on the comparing, and wherein the step of presenting the additional content received from the server by the mobile communication device comprises presenting each respective additional content received from the server.

3. The method of claim 1, wherein the step of identifying the words for which respective additional content is found in the searching comprises highlighting the words such that the words are emphasized when displayed on the mobile communication device.

4. The method of claim 1, wherein the step of identifying the words for which respective additional content is found in the searching comprises configuring an audio presentation of the transcription to include pronunciation emphasis of the words for which respective additional content is found in the searching.

5. The method of claim 1, wherein the step of identifying the words for which respective additional content is found in the searching comprises configuring an audio presentation of the transcription to include aural cueing for the words for which respective additional content is found in the searching.

6. The method of claim 1, wherein the audio data that is received from the mobile communication device representing an utterance that is intended to be at least a portion of the text of a message is received with an indication of the type of message.

7. The method of claim 6, wherein the indication is that the message is an instant message.

8. The method of claim 6, wherein the indication is that the message is to be sent from the mobile communication device in accordance with a text messaging protocol.

9. The method of claim 6, further comprising filtering the transcribed text based on the indication.

10. The method of claim 1, wherein the searching includes searching based on an identified word using an Internet search engine and analyzing the results provided in response thereto.

11. The method of claim 1, wherein the additional content comprises a web address, and wherein said step of presenting the additional content based on the received additional content data comprises opening the web address in a web browser of the mobile communication device.

12. The method of claim 1, wherein the additional content comprises a URL for retrieving the additional content for presentation by the mobile communication device, and wherein said step of presenting the additional content based on the received additional content comprises passing the URL to an appropriate application of the mobile communication device.

13. The method of claim 1, wherein the searching includes sending a keyword to a third party SMS service and analyzing the results provided in response thereto.

14. The method of claim 1, wherein the additional content comprises a keyword for sending to a third party SMS service, and wherein said step of presenting the additional content based on the received additional content data comprises sending the keyword to the third party SMS service and receiving in response thereto the additional content data.

15. The method of claim 1, wherein said step of receiving audio data communicated from the mobile communication device, the audio data representing an utterance that includes an identified word that is maintained in the database in association with the identifier of the mobile communication device, comprises receiving audio data representing an utterance that further includes a cue that an identified word is included in the utterance for presenting the additional content for such identified word.

16. The method of claim 1, wherein the association between the identified words, respective additional content for such words, and an identifier of the mobile communication device is maintained in the database for an extended period of time comprising a plurality of days or weeks, whereby additional content may be provided for an identified word communicated to the mobile communication device even though a plurality of days have passed since the identified word was communicated to the mobile communication device.

17. The method of claim 1, wherein said step of presenting the message, including emphasizing the identified words for which respective additional content is available for presenting by the mobile communication device, comprises the step of playing audio of the message on the mobile communication device, wherein the audio includes pronunciation emphasis for the identified words for which respective additional content is available for presenting by the mobile communication device.

18. The method of claim 1, wherein said step of presenting the message, including emphasizing the identified words for which respective additional content is available for presenting by the mobile communication device, comprises the step of playing audio of the message on the mobile communication device, wherein the audio includes aural cueing for the identified words for which respective additional content is available for presenting by the mobile communication device.

19. The method of claim 1, wherein said step of presenting the message, including emphasizing the identified words for which respective additional content is available for presenting by the mobile communication device, comprises the step of displaying text of the message on the mobile communication device such that the identified words, for which respective additional content is available for presenting on the mobile communication device, are highlighted.