Highlighting of voice message transcripts

• US 8,588,378 B2
• Filed: 07/15/2010
• Issued: 11/19/2013
• Est. Priority Date: 07/15/2009
• Status: Active Grant

• Current Assignee: Google LLC (Alphabet Inc.)
• Original Assignee: Google Inc. (Alphabet Inc.)
• Inventors: Davies, Benedict, Brunschen, Christian
• Primary Examiner(s): Patel, Hemant
• Application Number:
  US12/837,322
• Publication Number:
  US 20110013756A1
• Time in Patent Office: 1,223 Days
• Field of Search: 709201-207, 709217-248, 348 1401- 1416, 370259-271, 370351-357, 379 671- 8828, 4554121-4262, 4554561-466, 4555501-560, 4555751-5759, 455 901- 903, 704270-278
• US Class Current: 379/88.14
• CPC Class Codes: G10L 15/22 Procedures used during a speech recognition process, e.g. man-machine dialogue
• CPC Class Codes: G10L 2015/223 Execution procedure of a spoken command

Abstract

A computer-implemented voice mail method includes obtaining an electronic audio file of a recorded user message directed to a telephone user, automatically generating a transcript of the recorded user message, and identifying locations in the transcript in coordination with timestamps in the recorded user message so that successive portions of the transcript can be highlighted in coordination with playing of the recorded user message. The method also include identifying one or characteristics of the message using meta data relating to the recorded user message, and storing the recorded user message and information about the identified locations of the recorded user message.

Claims

1. A computer-implemented voice mail method, comprising:

obtaining an electronic audio file of a recorded user message directed to a telephone user;

automatically generating a transcript of the recorded user message;

in response to receipt of a first spoken phrase from a user of a computing device on which the electronic audio file is played, identifying (a) a first category to which the first spoken phrase is directed, (b) first one or more words in the transcript that correspond to the first category, and (c) a first command that corresponds to the first category; and

providing data to cause the identified first command to be performed automatically using the identified first one or more words as parameters for the identified command;

wherein the first spoken phrase does not identify the one or more words in the transcript.

2. The method of claim 1, wherein obtaining the electronic audio file comprises receiving an incoming telephone call and routing the incoming telephone call to a voice mail message system.

3. The method of claim 1, wherein automatically generating the transcript of the recorded user message comprises submitting the recorded user message to a speech-to-text conversion system.

4. The method of claim 1, further comprising providing, in coordination, the recorded user message audibly to the telephone user and the transcript visually to the user.

5. The method of claim 4, wherein providing the transcript in coordination with the recorded user message comprises moving a highlighted area across the transcript in a manner that matches a current location in the user message while it is played.

6. The method of claim 1, wherein the first command comprises a web search, the first category is a group of contiguous words category, and the first one or more words in the transcript comprise a group of contiguous words in the transcript.

7. The method of claim 1, wherein the first category is an address category, the one or more words in the transcript comprise an address, and the first command is a command to generate a map for a geographic area around the address.

8. The method of claim 1, wherein the first command comprises a command to send a communication, the first category is a recipient category, and the one or more words in the transcript comprise a recipient.

36. The method of claim 1, further comprising:

in response to receipt of a second spoken phrase from a user of a computing device on which the electronic audio file is played, identifying (a) a second category to which the second spoken phrase is directed, (b) second one or more words in the transcript that correspond to the second category, and (c) a second command that corresponds to the second category; and

wherein the first spoken phrase comprises the first command and an additional term, and the second spoken phrase comprises the second command and the additional term, and wherein the first one or more words in the transcript are different than the second one or more words in the transcript.

37. The method of claim 1, wherein the first spoken phrase comprises the command and one or more words to which a system that performs the identifying receives in association with different commands, and for which the system selects different words in the transcript, the selection depending identifying which of the plurality of commands is in the spoken input.

9. A computer-implemented voice mail system, comprising:

an audio file data store storing a plurality of digitized voice mail message directed to one or more users of the voice mail system;

a transcript data store storing a plurality of voice mail transcripts, each corresponding to a voice mail message stored in the audio file data store; and

a voice mail player including an audio player and a transcript manager programmed to;

in response to receipt of a first spoken phrase from a user, identify (a) a first category to which the first spoken phrase is directed, (b) first one or more words in the transcript that correspond to the first category, and (c) a first command that corresponds to the first category; and

provide data to cause the identified first command to be performed automatically using the identified first one or more words as parameters for the identified command;

wherein the first spoken phrase does not identify the one or more words in the transcript.

10. The system of claim 9, further comprising a voice mail recorder to terminate an incoming telephone call, provide instructions to a caller of the call, and receive and store a message from the caller in the audio file data store.

11. The system of claim 10, further comprising a speech-to-text converter programmed to generate a transcript that exhibits the voice mail message from the caller, and to store the transcript in the transcript data store.

12. The system of claim 9, wherein the audio file data store and the transcript data store share a common data structure.

13. The system of claim 9, wherein the voice mail player is programmed to provide the transcript in coordination with the recorded user message by moving a highlighting across the transcript in a manner that matches a current location in the voice mail message while it is played.

14. The system of claim 9, further comprising a user device that is programmed to receive the first spoken phrase.

15. The system of claim 9, wherein the first command comprises a web search, the first category is a group of contiguous words category, and the first one or more words in the transcript comprise a group of contiguous words in the transcript.

16. The system of claim 9, wherein the first category is an address category, the one or more words in the transcript comprise an address, and the first command is a command to generate a map for a geographic area around the address.

17. The system of claim 9, wherein the first command is a command to send a communication, the first category is a recipient category, and the one or more words in the transcript comprise a recipient.

18. A computer-implemented voice mail system, comprising:

an audio file data store storing a plurality of digitized voice mail message directed to one or more users of the voice mail system;

a transcript data store storing a plurality of voice mail transcripts, each corresponding to a voice mail message stored in the audio file data store; and

means for,

in response to receipt of a first spoken phrase from a user, identifying (a) a first category to which the first spoken phrase is directed, (b) first one or more words in the transcript that correspond to the first category, and (c) a first command that corresponds to the first category; and

providing data to cause the identified first command to be performed automatically using the identified first one or more words as parameters for the identified command;

wherein the first spoken phrase does not identify the one or more words in the transcript.

19. A computer-implemented method, comprising:

displaying textual content on a computing device, the textual content containing one or more terms that each have associated with them a content type that corresponds to a category to which the particular term is directed;

receiving a first spoken phrase from a user of a computing device;

identifying, in response to receiving the first spoken phrase, (a) a first category to which the first spoken phrase is directed, (b) first one or more words in the textual content that correspond to the first category, and (c) a first command that corresponds to the first category; and

causing the identified first command to be performed automatically using the identified first one or more words as parameters for the identified command;

wherein the first spoken phrase does not identify the one or more words in the textual content.

20. The method of claim 19, wherein the textual content comprises a transcript of a voicemail message received by the computing device.

21. The method of claim 19, wherein the first category is selected from a group consisting of personal name, address, organization name, date, shipping tracking number, ISBN number, state, city, country, and venue name.

22. The method of claim 19, wherein receiving a first spoken phrase comprises receiving a voice input on the computing device.

23. The method of claim 19, wherein the first spoken phrase takes the form of “

[command] that,”

.

24. The method of claim 19, wherein the first one or more words in the textual content are highlighted compared to other displayed textual content so that the user can see which displayed terms are subject to commands.

25. The method of claim 19, further comprising displaying the first command with the displayed textual content so that the user can see the first command that can be executed on the displayed content.

26. The method of claim 19, further comprising determining that a plurality of one or more words correspond to the first category, and providing a control for the user to select one of the plurality of one or more words to be the first one or more words.

27. The method of claim 19, further comprising invoking an application that corresponds to the first command, and displaying content with the invoked application, wherein the content is responsive to the first one or more words.

28. A computer-implemented system, comprising:

a text document data store storing one or more textual documents, the textual documents including one or more terms that each have associated with them a content type that corresponds to a category to which the particular term is directed;

an interface programmed to display a textual document and to receive a first spoken phrase from a user; and

a command correlator to;

identify, in response to receiving the first spoken phrase, (a) a first category to which the first spoken phrase is directed, (b) first one or more words in the textual document that correspond to the first category, and (c) a first command that corresponds to the first category; and

cause the identified first command to be performed automatically using the identified first one or more words as parameters for the identified command;

wherein the first spoken phrase does not identify the one or more words in the textual document.

29. The system of claim 28, wherein the textual document comprises a transcript of a voicemail message, wherein the transcript is time-correlated to an audio file of the voice message.

30. The system of claim 28, wherein the first category is selected from a group consisting of personal name, address, organization name, date, shipping tracking number, ISBN number, state, city, country, and venue name.

31. The system of claim 28, wherein the first spoken phrase takes the form of “

[command] that,”

.

32. The system of claim 28, wherein the first one or more words in the textual document are highlighted compared to other displayed textual content so that the user can see which displayed terms are subject to receiving commands.

33. The system of claim 28, wherein the system is programmed to display the first command with the displayed textual content so that the user can see the first command that can be executed on the displayed content.

34. The system of claim 28, wherein the system is further programmed to determine that a plurality of one or more words correspond to the first category, and to provide a control for the user to select one of the plurality of one or more words to be the first one or more words.

35. The system of claim 28, wherein the system is further programmed to invoke an application that corresponds to the first command, and to display content with the invoked application, wherein the content is responsive to a selected first one or more words.