Highlighting of Voice Message Transcripts

• US 20110013756A1
• Filed: 07/15/2010
• Published: 01/20/2011
• Est. Priority Date: 07/15/2009
• Status: Active Grant

• Current Assignee: Google LLC (Alphabet Inc.)
• Original Assignee: Google Inc. (Alphabet Inc.)
• Inventors: Brunschen, Christian, Davies, Benedict
• Granted Patent: US 8,588,378 B2
• Time in Patent Office: Days
• US Class Current: 379/88.14
• CPC Class Codes: G10L 15/22 Procedures used during a speech recognition process, e.g. man-machine dialogue
• CPC Class Codes: G10L 2015/223 Execution procedure of a spoken command

Abstract

A computer-implemented voice mail method includes obtaining an electronic audio file of a recorded user message directed to a telephone user, automatically generating a transcript of the recorded user message, and identifying locations in the transcript in coordination with timestamps in the recorded user message so that successive portions of the transcript can be highlighted in coordination with playing of the recorded user message. The method also include identifying one or characteristics of the message using meta data relating to the recorded user message, and storing the recorded user message and information about the identified locations of the recorded user message.

Claims

1. A computer-implemented voice mail method, comprising:

obtaining an electronic audio file of a recorded user message directed to a telephone user;

automatically generating a transcript of the recorded user message;

identifying locations in the transcript in coordination with timestamps in the recorded user message so that successive portions of the transcript can be highlighted in coordination with playing of the recorded user message;

identifying one or characteristics of the message using meta data relating to the recorded user message; and

storing the recorded user message and information about the identified locations of the recorded user message.

2. The method of claim 1, further comprising providing the recorded user message to the telephone user and providing the transcript and data for displaying the transcript with highlighting that shows the identified locations along with one or more areas adjacent to the identified locations, wherein the one or more areas are identified using the one or more characteristics of the message.

3. The method of claim 1, wherein obtaining the electronic audio file comprises receiving an incoming telephone call and routing the incoming telephone call to a voice mail message system.

4. The method of claim 1, wherein automatically generating the transcript of the recorded user message comprises submitting the recorded user message to a speech-to-text conversion system.

5. The method of claim 1, further comprising providing, in coordination, the recorded user message audibly to the telephone user and the transcript visually to the user.

6. The method of claim 5, wherein providing the transcript in coordination with the recorded user message comprises moving a highlighted area across the transcript in a manner that matches a current location in the message while it is played.

7. The method of claim 5, wherein identifying one or more characteristics of the message includes identifying a group of contiguous words in the transcript that make up a common topic.

8. The method of claim 7, further comprising receiving a user command and performing the user command on the group of contiguous words, without receiving an identification by the user of any one of the contiguous words.

9. The method of claim 7, wherein performing the user command comprises performing a web search on the group of contiguous words.

10. The method of claim 7, wherein the group of contiguous words defines an address, and performing the user command comprises generating a map for a geographic area around the address.

11. The method of claim 7, wherein the command comprises a command to send a communication, and further comprising automatically identifying a recipient for the communication from information corresponding to the message.

24. The method of claim 1, wherein the textual content comprises a transcript of a voicemail message received by the computing device.

25. The method of claim 1, wherein the content types for the one or more terms are selected from a group consisting of personal name, address, organization name, date, shipping tracking number, ISBN number, state, city, country, and venue name.

26. The method of claim 1, wherein receiving a user command that does not explicitly identify any of the one or more terms comprises receiving a voice input on the computing device and identifying a command represented by the voice input.

27. The method of claim 4, wherein the voice input takes the form of “

[command] that,”

wherein the command represents an action the user wishes to take with respect to one of the one or more terms.

28. The method of claim 1, wherein the identified one or more terms are highlighted compared to other displayed textual content so that the user can see which displayed terms are subject to commands.

29. The method of claim 1, further comprising identifying commands that can be performed on the one or more terms, and displaying the identified one or more commands with the displayed textual content so that the user can see commands that can be executed on the displayed content.

30. The method of claim 1, further comprising determining that more than one of the terms corresponds to a content type that also corresponds to the received user command, and providing a control for the user to select one of the more than one terms to be the first of the one or more terms.

31. The method of claim 1, further comprising invoking an application that corresponds to the user command, and displaying content with the invoked application, wherein the content is responsive to the first of the one or more terms.

33. The system of claim 10, wherein the text document comprises a transcript of a voicemail message received by the computing device, wherein the transcript is time-correlated to an audio file of the voice message.

34. The system of claim 10, wherein the content types for the one or more terms are selected from a group consisting of personal name, address, organization name, date, shipping tracking number, ISBN number, state, city, country, and venue name.

35. The system of claim 10, wherein the interface is programmed to convert a voice input to one or more predefined commands.

37. The system of claim 10, wherein the one or more terms are highlighted in a displayed document compared to other displayed textual content so that the user can see which displayed terms are subject to receiving commands.

38. The system of claim 10, wherein the system is programmed to identify commands that can be performed on the one or more terms, and to display the identified one or more commands with the displayed textual content so that the user can see commands that can be executed on the displayed content.

39. The system of claim 10, wherein the system is further programmed to determine that more than one of the terms corresponds to a content type that also corresponds to a received user command, and to provide a control for the user to select one of the more than one terms to be the first of the one or more terms.

40. The system of claim 10, wherein the system is further programmed to invoke an application that corresponds to the user command, and to display content with the invoked application, wherein the content is responsive to a selected first of the one or more terms.

12. A computer-implemented voice mail system, comprising:

an audio file data store storing a plurality of digitized voice mail message directed to one or more users of the voice mail system;

a transcript data store storing a plurality of voice mail transcripts, each corresponding to a voice mail message stored in the audio file data store; and

a voice mail player including an audio player and a transcript manager programmed to present a transcript while playing an audio file of a message and to show highlighting of transcript at a location corresponding to a current location of the playing audio file, wherein the a visual form of the highlighting depends on one or more characteristics of the message that are determined using meta data relating to the recorded user message.

13. The system of claim 12, further comprising a voice mail recorder to terminate an incoming telephone call, provide instructions to a caller of the call, and receive and store a message from the caller in the audio file data store.

14. The system of claim 13, further comprising a text-to-speech converter programmed to generate a transcript that exhibits the message from the caller, and to store the transcript in the transcript data store.

15. The system of claim 12, wherein the audio file data store and the transcript data store share a common data structure.

16. The system of claim 12, wherein the voice mail player is programmed to provide the transcript in coordination with the recorded user message by moving the highlighting across the transcript in a manner that matches a current location in the message while it is played.

17. The system of claim 12, wherein the system is programmed to identify one or more characteristics of the message by identifying a group of contiguous words in a transcript that make up a common topic.

18. The system of claim 17, further comprising a user device that is programmed to receive a user command and perform the user command on the group of contiguous words, without receiving an identification by the user of any one of the contiguous words.

19. The system of claim 18, further comprising a user device that is programmed to perform the user by comprises performing a web search on the group of contiguous words.

20. The system of claim 18, wherein the group of contiguous words defines an address, and wherein the system further comprises a user device that is programmed to generate a map for a geographic area around the address.

21. The system of claim 18, further comprising a user device that is programmed to respond to the command by sending a communication, and to automatically identify a recipient for the communication from information corresponding to the message.

36. The system of claim 13, wherein the voice input takes the form of “

[command] that,”

wherein the command represents an action the user wishes to take with respect to one of the one or more terms.

22. A computer-implemented voice mail system, comprising:

an audio file data store storing a plurality of digitized voice mail message directed to one or more users of the voice mail system;

a transcript data store storing a plurality of voice mail transcripts, each corresponding to a voice mail message stored in the audio file data store; and

means for presenting an audio file from the audio file data store in coordination with a corresponding transcript with information generated by analyzing meta data relating to a message in the audio file.

23. A computer-implemented method, comprising:

displaying textual content on a computing device, the textual content containing one or more terms that each have associated with them a content type that corresponds to a category to which the particular term is directed;

receiving a user command that does not explicitly identify any of the one or more terms;

identifying a content type that corresponds to the user command; and

causing the user command to be executed with a first of the one or more terms, by matching the identified content type that corresponds to the user command with the content types that correspond to the associated with the one or more terms.

32. A computer-implemented system, comprising:

a text document data store storing one or more textual documents, the textual documents including one or more terms that each have associated with them a content type that corresponds to a category to which the particular term is directed;

an interface programmed to receive user commands that do not include the one or more terms; and

a command correlator to identify a content type that corresponds to a command received by the interface, to select one of the one or more terms by correlating the identified content type to a content type that corresponds to one of the terms, and to cause the received command to be executed with he selected term as an argument.