Facilitating presentation by mobile device of additional content for a word or phrase upon utterance thereof

• US 8,335,830 B2
• Filed: 11/09/2009
• Issued: 12/18/2012
• Est. Priority Date: 08/22/2007
• Status: Active Grant

• Current Assignee: Amazon Technologies, Inc. (Amazon.com, Inc.)
• Original Assignee: Canyon IP Holdings LLC (Intellectual Ventures LLC)
• Inventors: Jablokov, Victor Roditis, Jablokov, Igor Roditis, Terrell, James Richard II, Paden, Scott Edward
• Primary Examiner(s): Zhang, Shirley
• Assistant Examiner(s): Survillo, Oleg
• Application Number:
  US12/614,694
• Publication Number:
  US 20100058200A1
• Time in Patent Office: 1,135 Days
• Field of Search: 709/206, 709/217, 709/219, 704/270.1, 704/235, 707/705, 707/736
• US Class Current: 709/206
• CPC Class Codes: G06F 16/951 Indexing; Web crawling techniques
• CPC Class Codes: G06F 16/9535 Search customisation based on user profiles and personalisation
• CPC Class Codes: G06F 16/9538 Presentation of query results
• CPC Class Codes: G06F 16/957 Browsing optimisation, e.g. caching or content distillation
• CPC Class Codes: G06F 16/9577 Optimising the visualization of content, e.g. distillation of HTML documents
• CPC Class Codes: G06F 3/167 Audio in a user interface, e.g. using voice commands for navigating, audio feedback
• CPC Class Codes: G06Q 10/107 Computer-aided management of electronic mailing [e-mailing]
• CPC Class Codes: G06Q 30/02 Marketing; Price estimation or determination; Fundraising
• CPC Class Codes: G06Q 30/0256 User search
• CPC Class Codes: G06Q 30/0269 based on user profile or attribute
• CPC Class Codes: G06Q 30/0271 Personalized advertisement
• CPC Class Codes: G06Q 30/0631 Item recommendations
• CPC Class Codes: G10L 13/033 Voice editing, e.g. manipulating the voice of the synthesiser
• CPC Class Codes: G10L 13/04 Details of speech synthesis systems, e.g. synthesiser structure or memory management
• CPC Class Codes: G10L 15/26 Speech to text systems G10L15/08 takes precedence
• CPC Class Codes: G10L 2015/088 Word spotting
• CPC Class Codes: G10L 21/00 Speech or voice signal processing techniques to produce another audible or non-audible signal, e.g. visual or tactile, in order to modify its quality or its intelligibility G10L19/00 takes precedence
• CPC Class Codes: H04L 51/00 User-to-user messaging in packet-switching networks, transmitted according to store-and-forward or real-time protocols, e.g. e-mail
• CPC Class Codes: H04L 67/02 based on web technology, e.g. hypertext transfer protocol [HTTP]
• CPC Class Codes: H04L 69/329 in the application layer [OSI layer 7]
• CPC Class Codes: H04M 1/7243 : with interactive means for internal management of messages
• CPC Class Codes: H04M 2201/40 : using speech recognition speech recognition per se G10L15/00
• CPC Class Codes: H04M 3/4938 : comprising a voice browser which renders and interprets, e.g. VoiceXML

Abstract

A method for presenting additional content for a word that is part of a message, and that is presented by a mobile communication device, includes the steps of: presenting the message, including emphasizing one or more words for which respective additional content is available for presenting by the mobile communication device; receiving an utterance that includes an emphasized word for which additional content is available for presenting by the mobile communication device; and presenting the additional content for the emphasized word included in the utterance received by the mobile communication device. These steps are performed by the mobile communication device.

Claims

1. A computer-implemented method for facilitating presentation of additional content for a term that is part of a transcribed utterance intended for a message, the computer-implemented method comprising:

(a) first,

(i) receiving first audio data communicated from a first mobile communication device,

(ii) transcribing the first audio data to first text,

(iii) selecting one or more terms in the first text for which respective additional content is available,

(iv) communicating the first text with the selected one or more terms to a second mobile communication device along with information identifying the selected one or more terms, and

(v) maintaining in a database an association between the selected one or more terms, the respective additional content for the selected one or more terms, and an identifier of the second mobile communication device to which the selected one or more terms were communicated;

(b) thereafter,

(i) receiving second audio data communicated from the second mobile communication device and an identifier of the second mobile communication device, the second audio data representing an utterance that includes a selected term of the one or more selected terms,

(ii) transcribing the second audio data to second text,

(iii) determining that the identifier of the second mobile communication device is stored in the database and,

(iv) based at least in part upon the identifier of the second mobile communication device being stored in the database, comparing the second text to the selected one or more terms maintained in the database in association with the identifier of the second mobile communication device, and

(v) based at least in part on at least one of the selected one or more terms being present in the second text, communicating the respective additional content associated with the at least one of the selected one or more terms to the second mobile communication device, wherein the communicated respective additional content is for presentation by the second mobile communication device.

2. The computer-implemented method of claim 1, wherein communicating the first text with the selected one or more terms to the second mobile communication device along with information identifying the selected one or more terms comprises highlighting the terms such that the terms are emphasized when displayed on the second mobile communication device.

3. The computer-implemented method of claim 1, wherein communicating the first text with the selected one or more terms to the second mobile communication device along with information identifying the selected one or more terms comprises configuring an audio presentation of the first text to include pronunciation emphasis of the terms in the first text for which respective additional content is available.

4. The computer-implemented method of claim 1, wherein communicating the first text with the selected one or more terms to the second mobile communication device along with information identifying the selected one or more terms comprises configuring an audio presentation of the first text to include aural cueing for the terms in the first text for which respective additional content is available.

5. The computer-implemented method of claim 1, wherein the first audio data that is received from the first mobile communication device is received with an indication of a type of a message to be communicated to the second mobile communication device.

6. The computer-implemented method of claim 5, wherein the indication is that the message is an instant message.

7. The computer-implemented method of claim 5, wherein the indication is that the message is to be communicated to the second mobile communication device in accordance with a text messaging protocol.

8. The computer-implemented method of claim 5, further comprising filtering the first text that is transcribed from the first audio data based on the indication.

9. The computer-implemented method of claim 1, wherein the second mobile communication device is the first mobile communication device.

10. A non-transitory computer-readable medium whose contents configure one or more computing systems to perform a method for facilitating presentation of additional content for a term that is part of a transcribed utterance intended for a message, the method comprising:

(a) first,

(i) receiving first audio data communicated from a first mobile communication device,

(ii) transcribing the first-audio data to first text,

(iii) selecting one or more terms in the first text for which respective additional content is available,

(iv) communicating the first text with the selected one or more terms to a second mobile communication device along with information identifying the selected one or more terms, and

(v) maintaining in a database an association between the selected one or more terms, the respective additional content for the selected one or more terms, and an identifier of the second mobile communication device to which the selected one or more terms were communicated;

(b) thereafter,

(i) receiving second audio data communicated from the second mobile communication device and an identifier of the second mobile communication device, the second audio data representing an utterance that includes a selected term of the one or more selected terms,

(ii) transcribing the second audio data to second text,

(iii) determining that the identifier of the second mobile communication device is stored in the database and,

(iv) based at least in part upon the identifier of the second mobile communication device being stored in the database, comparing the second text to the selected one or more terms maintained in the database in association with the identifier of the second mobile communication device, and

(v) based at least in part on at least one of the selected one or more terms being present in the second text, communicating the respective additional content associated with the at least one of the selected one or more terms to the second mobile communication device, wherein the communicated respective additional content is for presentation by the second mobile communication device.

11. The non-transitory computer readable medium of claim 10, wherein communicating the first text with the selected one or more terms to the second mobile communication device along with information identifying the selected one or more terms comprises at least one of:

(a) highlighting the terms such that the terms are emphasized when displayed on the second mobile communication device, (b) configuring an audio presentation of the first text to include pronunciation emphasis of the terms in the first text for which respective additional content is available or (c) configuring an audio presentation of the first text to include aural cueing for the terms in the first text for which respective additional content is available.

12. The non-transitory computer readable medium of claim 10, wherein the first audio data that is received from the first mobile communication device is received with an indication of a type of a message to be communicated to the second mobile communication device.

13. The non-transitory computer readable medium of claim 12, wherein the indication is that the message is one of:

(1) an instant message or (2) to be communicated to the second mobile communication device in accordance with a text messaging protocol.

14. The non-transitory computer readable medium of claim 12, wherein the method further comprises filtering the first text that is transcribed from the first audio data based on the indication.

15. The non-transitory computer-readable medium of claim 10, wherein the second mobile communication device is the first mobile communication device.

16. A system for facilitating presentation of additional content for a term that is part of a transcribed utterance intended for a message, the system comprising one or more processors configured to:

(a) first,

(i) receive first audio data communicated from a first mobile communication device,

(ii) transcribe the first-audio data to first text,

(iii) select one or more terms in the first text for which respective additional content is available,

(iv) communicate the first text with the selected one or more terms to a second mobile communication device along with information identifying the selected one or more terms, and

(v) maintain in a database an association between the selected one or more terms, the respective additional content for the selected one or more terms, and an identifier of the second mobile communication device to which the selected one or more terms were communicated;

(b) thereafter,

(i) receive second audio data communicated from the second mobile communication device and an identifier of the second mobile communication device, the second audio data representing an utterance that includes a selected term of the one or more selected terms,

(ii) transcribe the second audio data to second text,

(iii) determine that the identifier of the second mobile communication device is stored in the database and,

(iv) based at least in part upon the identifier of the second mobile communication device being stored in the database, compare the second text to the selected one or more terms maintained in the database in association with the identifier of the second mobile communication device, and

(v) based at least in part on at least one of the selected one or more terms being present in the second text, communicate the respective additional content associated with the at least one of the selected one or more terms to the second mobile communication device, wherein the communicated respective additional content is for presentation by the second mobile communication device.

17. The system of claim 16, wherein the one or more processors are configured to communicate the first text with the selected one or more terms to the second mobile communication device along with information identifying the selected one or more terms by (a) highlighting the terms such that the terms are emphasized when displayed on the second mobile communication device, or (b) configuring an audio presentation of the first text to include pronunciation emphasis of the terms in the first text for which respective additional content is available, or (c) configuring an audio presentation of the first text to include aural cueing for the terms in the first text for which respective additional content is available.

18. The system of claim 16, wherein the first audio data that is received from the first mobile communication device is received with an indication of a type of a message to be communicated to the second mobile communication device.

19. The system of claim 18, wherein the indication is that the message is one of (1) an instant message or (2) to be communicated to the second mobile communication device in accordance with a text messaging protocol.

20. The system of claim 18, wherein the one or more processors are further configured to filter the first text that is transcribed from the first audio data based on the indication.

21. The system of claim 16, wherein the second mobile communication device is the first mobile communication device.