Facilitating presentation by mobile device of additional content for a word or phrase upon utterance thereof

• US 9,436,951 B1
• Filed: 08/25/2008
• Issued: 09/06/2016
• Est. Priority Date: 08/22/2007
• Status: Active Grant

• Current Assignee: Amazon Technologies, Inc. (Amazon.com, Inc.)
• Original Assignee: Amazon Technologies, Inc. (Amazon.com, Inc.)
• Inventors: Jablokov, Victor Roman, Jablokov, Igor Roditis, Terrell, James Richard II, Paden, Scott Edward
• Primary Examiner(s): Survillo, Oleg
• Application Number:
  US12/198,116
• Time in Patent Office: 2,934 Days
• Field of Search: 709/206, 709/217, 709/219, 704/270.1, 704/235, 707/705, 707/736
• US Class Current: 1/1
• CPC Class Codes: G06F 16/632 Query formulation
• CPC Class Codes: G06F 16/683 using metadata automatically derived from the content
• CPC Class Codes: G06F 16/951 Indexing; Web crawling techniques
• CPC Class Codes: G06F 16/9535 Search customisation based on user profiles and personalisation
• CPC Class Codes: G06F 16/9538 Presentation of query results
• CPC Class Codes: G06F 16/957 Browsing optimisation, e.g. caching or content distillation
• CPC Class Codes: G06F 16/9577 Optimising the visualization of content, e.g. distillation of HTML documents
• CPC Class Codes: G06F 3/167 Audio in a user interface, e.g. using voice commands for navigating, audio feedback
• CPC Class Codes: G06Q 10/107 Computer-aided management of electronic mailing [e-mailing]
• CPC Class Codes: G06Q 30/02 Marketing; Price estimation or determination; Fundraising
• CPC Class Codes: G06Q 30/0256 User search
• CPC Class Codes: G06Q 30/0269 based on user profile or attribute
• CPC Class Codes: G06Q 30/0271 Personalized advertisement
• CPC Class Codes: G06Q 30/0631 Item recommendations
• CPC Class Codes: G10L 15/22 Procedures used during a speech recognition process, e.g. man-machine dialogue
• CPC Class Codes: G10L 15/26 Speech to text systems G10L15/08 takes precedence
• CPC Class Codes: G10L 21/00 Speech or voice signal processing techniques to produce another audible or non-audible signal, e.g. visual or tactile, in order to modify its quality or its intelligibility G10L19/00 takes precedence
• CPC Class Codes: H04L 51/216 Handling conversation history, e.g. grouping of messages in sessions or threads
• CPC Class Codes: H04L 67/02 based on web technology, e.g. hypertext transfer protocol [HTTP]
• CPC Class Codes: H04L 69/329 in the application layer [OSI layer 7]
• CPC Class Codes: H04M 1/7243 : with interactive means for internal management of messages
• CPC Class Codes: H04M 2201/39 : using speech synthesis speech synthesis per se G10L13/00
• CPC Class Codes: H04M 2201/40 : using speech recognition speech recognition per se G10L15/00
• CPC Class Codes: H04M 2203/357 : Autocues for dialog assistance
• CPC Class Codes: H04M 3/42382 : Text-based messaging services in telephone networks such as PSTN/ISDN, e.g. User-to-User Signalling or Short Message Service for fixed networks
• CPC Class Codes: H04M 3/4878 : Advertisement messages
• CPC Class Codes: H04M 3/4938 : comprising a voice browser which renders and interprets, e.g. VoiceXML
• CPC Class Codes: H04M 7/0036 : where the data service is an information service
• CPC Class Codes: H04W 4/12 : Messaging; Mailboxes; Announcements
• CPC Class Codes: H04W 4/21 : for social networking applications

Abstract

A method for presenting additional content for a word that is part of a message, and that is presented by a mobile communication device, includes the steps of: presenting the message, including emphasizing one or more words for which respective additional content is available for presenting by the mobile communication device; receiving an utterance that includes an emphasized word for which additional content is available for presenting by the mobile communication device; and presenting the additional content for the emphasized word included in the utterance received by the mobile communication device. These steps are performed by the mobile communication device.

Claims

1. A computer-implemented method of presenting additional content for a term presented by a first mobile communication device, the computer-implemented method comprising:

receiving, by the first mobile communication device, a first utterance;

transmitting, by the first mobile communication device, a first identifier of the first mobile communication device and the first utterance to a server;

receiving, by the first mobile communication device from the server, text representing a transcription of the first utterance;

receiving, by the first mobile communication device from the server, an indicator that first additional content is available for a term identified within the text by the indicator, wherein the term is associated at the server with the first identifier of the first mobile communication device and a second identifier of a second mobile communication device, and wherein the first additional content for the term is associated with the first identifier and the second identifier;

presenting, on the first mobile communication device, the text with an emphasis on the term identified by the indicator;

after presenting the text on the first mobile communication device, receiving, by the first mobile communication device, a second utterance that includes the term;

transmitting, by the first mobile communication device, the first identifier and the second utterance to the server;

receiving, by the first mobile communication device from the server, in response to transmitting the second utterance and the first identifier, the first additional content;

presenting, on the first mobile communication device the first additional content for the term;

transmitting, by the first mobile communication device, the second identifier to the server, the server configured to send the text as well as the indicator that first additional content is available for the term to the second mobile communication device using the second identifier of the second mobile communication device; and

receiving, by the first mobile communication device from the server, a message including a transcribed third utterance received by the second communication device in response to the text.

2. The method of claim 1, wherein the first mobile communication device is a phone.

3. The method of claim 1, wherein the first additional content is presented by playing audio over a speaker of the first mobile communication device.

4. The method of claim 1, wherein presenting the text on the first mobile communication device, including emphasizing the term comprises playing audio of the text on the first mobile communication device, wherein the audio includes pronunciation emphasis for the term.

5. The method of claim 1, wherein presenting the text on the first mobile communication device with an emphasis on the term identified by the indicator comprises playing audio of the text on the first mobile communication device, wherein the audio includes aural cueing for the term.

6. The method of claim 1, wherein presenting the text on the first mobile communication device with an emphasis on the term identified by the indicator comprises displaying the text on the first mobile communication device such that the term is highlighted.

7. The method of claim 6, wherein the first additional content for the term is presented by expanding the displayed text to include the respective additional content in conjunction with the term.

8. The method of claim 1, wherein the first additional content comprises at least one of a web address, an email address, or geospatial information.

9. The method of claim 1, wherein the first additional content is presented by displaying a video on the first mobile communication device.

10. The method of claim 1, wherein the first additional content is presented in a web browser of the first mobile communication device.

11. The method of claim 1, wherein the first additional content is presented in a popup window of the first mobile communication device.

12. The method of claim 1, wherein the term consists of a single word.

13. The method of claim 1, wherein the term comprises a phrase.

14. The method of claim 1, wherein the term comprises an alphanumeric string.

15. The method of claim 1, wherein the second utterance is transmitted with a cue indicating that the second utterance includes the term.

16. The method of claim 1, wherein multiple alternative first additional content exists for the term and are available for presentation by the first mobile communication device, and wherein the method further comprises presenting, on the first mobile communication device, the first additional content of each alternative first additional content of the multiple alternative first additional content.

17. The method of claim 1, further comprising receiving non-verbal input by the first mobile communication device, which input represents a selection of term that is emphasized;

and presenting, on the first mobile communication device, the first additional content for the term of the selection, whereby the first additional content may be presented during times when the first mobile communication device is unable to successfully receive an utterance.

18. The method of claim 1, wherein multiple alternative first additional content exists for the term and are available for presentation by the first mobile communication device, and wherein the method further comprises selecting and presenting at least one alternative first additional content of the multiple alternative first additional content.

19. The method of claim 18, wherein selecting the at least one alternative first additional content of the multiple alternative first additional content is performed by a user of the first mobile communication device.

20. The method of claim 18, wherein selecting the at least one alternative first additional content of the multiple alternative first additional content is performed in accordance with a computer algorithm.

21. The method of claim 1, wherein receiving the text representing a transcription of the first utterance includes receiving a plurality of terms associated with the first identifier of the first mobile communication device.

22. A non-transitory computer-readable medium whose contents configure one or more computing systems to perform a method of presenting additional content for a term presented by a first mobile communication device, the method comprising:

receiving, by the first mobile communication device, a first utterance;

transmitting a first identifier of the first mobile communication device and the first utterance from the first mobile communication device to a computing device;

receiving, by the first mobile communication device from the computing device, text representing a transcription of the first utterance;

receiving, by the first mobile communication device from the computing device, an indicator that first additional content is available for a term identified within the text by the indicator, wherein the term is associated at the computing device with the first identifier of the first mobile communication device and a second identifier of a second mobile communication device, and wherein the first additional content for the term is associated with the first identifier and the second identifier;

presenting, on the first mobile communications device, the text with an emphasis on the term identified by the indicator;

after presenting the text on the first mobile communication device, receiving, by the first mobile communication device, a second utterance that includes the term;

transmitting, by the first mobile communication device, the first identifier and the second utterance to the computing device;

receiving, by the first mobile communication device from the computing device, in response to transmitting the second utterance and the first identifier, the first additional content;

presenting, on the first mobile communication device, the first additional content for the term;

transmitting, by the first mobile communication device, the second identifier to the computing device, the computing device configured to send the text as well as the indicator that first additional content is available for the term to the second mobile communication device using the second identifier; and

receiving, by the first mobile communication device from the computing device, a message including a transcribed third utterance received by the second communication device in response to the text.

23. The non-transitory computer-readable medium of claim 22, wherein the first mobile communication device is at least one of a phone, a PDS, a tablet notebook or a two-way pager.

24. The non-transitory computer-readable medium of claim 22, wherein presenting the text on the first mobile communication device with an emphasis on the term identified by the indicator comprises playing audio of the text on the first mobile communication device, wherein the audio includes one or more of pronunciation emphasis for the term or aural cueing for the term.

25. The non-transitory computer-readable medium of claim 22, wherein presenting the text on the first mobile communication device with an emphasis on the term identified by the indicator comprises displaying the text on the first mobile communication device such that the term is highlighted.

26. The non-transitory computer-readable medium of claim 22, wherein presenting the first additional content for the term on the first mobile communication device comprises one or more of playing audio over a speaker of the first mobile communication device, expanding the displayed text to include the respective first additional content in conjunction with the term, or displaying a video on the first mobile communication device.

27. The non-transitory computer-readable medium of claim 22, wherein the first additional content comprises one or more of a web address, an email address, or geospatial information.

28. The non-transitory computer-readable medium of claim 22, wherein the first additional content is presented in one or more of a web browser of the first mobile communication device or a popup window of the first mobile communication device.

29. The non-transitory computer-readable medium of claim 22, wherein the term comprises one of more of a single word, a phrase, or an alphanumeric string.

30. The non-transitory computer-readable medium of claim 22, wherein the second utterance is transmitted with a cue indicating that the second utterance includes an emphasized term.

31. The non-transitory computer-readable medium of claim 22, wherein multiple alternative first additional content exists for the term and are available for presentation by the first mobile communication device, and wherein the method further comprises presenting by the first mobile communication device the first additional content of each such alternative.

32. The non-transitory computer-readable medium of claim 22, wherein the method further comprises receiving non-verbal input by the first mobile communication device, which input represents a selection of an emphasized term;

and presenting, by the first mobile communication device, the first additional content for the selected term, whereby the first additional content may be presented during times when the first mobile communication device is unable to successfully receive an utterance.

33. The non-transitory computer-readable medium of claim 22, wherein multiple alternative first additional content exists for the term and are available for presentation by the first mobile communication device, and wherein the method further comprises presenting, on the first mobile communication device, the multiple alternative first additional content.

34. The non-transitory computer-readable medium of claim 33, wherein selecting the first additional content of one of the alternatives is performed by a user of the first mobile communication device or in accordance with a computer algorithm.

35. The non-transitory computer-readable medium of claim 22, wherein receiving the text from the first computing device includes receiving a plurality of terms that have been selected by the first computing device from the transcription and associated with the identifier of the first mobile communication device.

36. A system for presenting additional content for a term presented by a first mobile communication device, the system comprising one or more processors executing instructions configured to:

receive, by the first mobile communication device, a first utterance;

transmit, by the first mobile communication device, a first identifier of the first mobile communication device and the first utterance to a computing device;

receive, by the first mobile communication device from the computing device, text representing a transcription of the first utterance;

receive, by the first mobile communication device from the computing device, an indicator that first additional content is available for a term identified within the text by the indicator, wherein the term is associated at the computing device with the first identifier of the first mobile communication device and a second identifier of a second mobile communication device, and wherein the first additional content for the term is associated with the first identifier and the second identifier;

present, on the first mobile communication device, the text with an emphasis on the term identified by the indicator;

after presenting the text on the first mobile communication device, receive, by the first mobile communication device, a second utterance that includes the term;

transmit, by the first mobile communication device, the first identifier and the second utterance to the computing device;

receive, by the first mobile communication device from the computing device, in response to transmitting the second utterance and the first identifier, the first additional content;

present, on the first mobile communication device, the first additional content for the term;

transmit, by the first mobile communication device, the second identifier to the computing device, the computing device configured to send the text as well as the indicator that first additional content is available for the term to the second mobile communication device using the second identifier; and

receive, by the first mobile communication device from the computing device, a message including a transcribed third utterance received by the second communication device in response to the text.

37. The system of claim 36, wherein the one or more processors executing the instructions are configured to present the text on the first mobile communication device by playing audio of the text on the first mobile communication device, wherein the audio includes one or more of pronunciation emphasis for the term or aural cueing for the term.

38. The system of claim 36, wherein the one or more processors executing the instructions are configured to present the text representing the transcription on the first mobile communication device by displaying the text representing the transcription on the first mobile communication device such that within the text representing the transcription the term is highlighted.

39. The system of claim 36, wherein the one or more processors executing the instructions are configured to present the first additional content for the term on the first mobile communication device by at least one of playing audio over a speaker of the first mobile communication device, expanding the presented text representing the transcription to include the respective first additional content in conjunction with the term, or displaying a video on the first mobile communication device.

40. The system of claim 36, wherein the first additional content comprises one or more of a web address, an email address, or geospatial information.

41. The system of claim 36, wherein multiple alternative first additional content exists for the term and are available for presentation by the first mobile communication device, and wherein the one or more processors executing the instructions are further configured to present, on the first mobile communication device, the multiple alternative first.

42. The system of claim 36, wherein the one or more processors executing the instructions are further configured to receive non-verbal input by the first mobile communication device, which input represents a selection of the term that is emphasized;

and present, on the first mobile communication device, the first additional content for the term, whereby the first additional content may be presented during times when the first mobile communication device is unable to successfully receive an utterance.

43. The system of claim 36, wherein multiple alternative first additional content exists for the term and are available for presentation by the first mobile communication device, and wherein the one or more processors executing the instructions are configured to present at least one alternative first additional content of the multiple alternative first additional content.

44. The system of claim 43, wherein the one or more processors executing the instructions are configured to select the at least one alternative first additional content of the multiple alternative first additional content by input from a user of the first mobile communication device or in accordance with a computer algorithm.

45. The system of claim 36, wherein to receive the text representing a transcription of the first utterance, the one or more processors executing the instructions are configured to receive a plurality of terms associated with the identifier of the first mobile communication device.