Method and apparatus for modifying digital messages containing at least audio

• US 9,185,225 B1
• Filed: 06/08/2011
• Issued: 11/10/2015
• Est. Priority Date: 06/08/2011
• Status: Active Grant

• Current Assignee: Cellco Partnership, Inc. (Verizon Communications Inc.)
• Original Assignee: Cellco Partnership, Inc. (Verizon Communications Inc.)
• Inventors: Vance, Charles Terry
• Primary Examiner(s): Bezuayehu, Solomon
• Application Number:
  US13/156,095
• Time in Patent Office: 1,616 Days
• Field of Search: 379/93.15, 379/100.13, 379/142.14, 379 8801- 8819, 704201-219
• US Class Current: 1/1
• CPC Class Codes: H04M 11/10 with dictation recording and playback systems
• CPC Class Codes: H04M 3/5307 for recording messages comprising any combination of audio and non-audio components
• CPC Class Codes: H04M 3/53333 Message receiving aspects

Abstract

A voice and/or video message for a user in the form of a voicemail or a video mail is edited in accordance with editing of text corresponding to audio of the message. Speech contained in the audio of the voice or video message is automatically converted to corresponding text and presented to the user via a graphical user interface. In response to various user inputs in relation to the presentation of the corresponding text, the corresponding text is modified and the digital representation of the voice or video message is modified in a manner corresponding to the modification of the corresponding text. New versions of the modified voice or video message including the modified digital representation of the voice or video message can be created and saved.

Claims

1. A computer-implemented method comprising steps of:

obtaining a digital representation of a message having at least audio from a mailbox of a user in a server;

automatically converting, by a processor, speech contained in the audio of the message to corresponding text;

automatically converting, by the processor, the audio of the message to a corresponding spectral diagram, the spectral diagram being a frequency domain representation of the audio of the message;

presenting a unified display on a computing device screen containing both the text and the spectral diagram to the user;

automatically editing the text in response to one or more user inputs in relation to the presentation of the text;

using the spectral diagram to remove sound in the audio that does not correspond to the text;

automatically editing, by the processor, the digital representation of the message, including the audio of the message, in a manner corresponding to the editing of the text;

annotating the text in response to one or more user inputs in relation to the presentation of the text;

annotating the digital representation of the message, including the audio of the message, in a manner corresponding to the annotating of the text; and

storing, in the server, the edited and annotated digital representation of the message.

2. The method of claim 1, further comprising synchronizing the corresponding text to the audio of the message.

3. The method of claim 2, wherein the synchronization of the corresponding text to the audio of the message is based on timestamp information.

4. The method of claim 1, wherein the server is a mail server, the message is a voicemail message and the mailbox is a voicemail mailbox.

5. The method of claim 1, wherein:

the message further has video and the mailbox is for video mail; and

the step of editing the digital representation of the message further includes editing the video of the message in a manner corresponding to the editing of the text.

6. The method of claim 1, wherein the annotation includes at least one of text inputted by the user or voice inputted by the user.

19. The method of claim 1, further comprising synchronizing the spectral diagram to the corresponding text.

20. The method of claim 19, further comprising:

adding an annotation to at least one of the spectral diagram or the text in response to one or more user inputs in relation to the presentation of the spectral diagram; and

annotating the digital representation of the message, including the audio of the message, in a manner corresponding to and in response to the annotation added to the at least one of the spectral diagram or the text.

7. A server, comprising:

an interface for network communication;

a processor coupled to the interface;

a program for the processor; and

storage for the program and for a mailbox of a user,

wherein execution of the program by the processor causes the server to perform functions, including functions to;

process a digital representation of a message in the mailbox of the user to convert speech contained in audio of the message to corresponding text;

process the digital representation of the message to convert the audio of the message to a corresponding spectral diagram, the spectral diagram being a frequency domain representation of the audio of the message;

present a unified display on a computing device screen containing both the text and the spectral diagram to the user;

automatically edit the text in response to one or more user inputs in relation to the presentation of the text;

use the spectral diagram to remove sound in the audio that does not correspond to the text;

automatically edit, by the processor, the digital representation of the message including the audio of the message, in a manner corresponding to the editing the text;

annotate the text in response to one or more user inputs in relation to the presentation of the text;

annotate the digital representation of the message, including the audio of the message, in a manner corresponding to the annotating of the text; and

store, in the server, the edited and annotated digital representation of the message.

8. The server of claim 7, wherein the execution of the program by the processor causes the server to perform further a function to synchronize the corresponding text to the audio of the message.

9. The server of claim 7, wherein the synchronization of the corresponding text to the audio of the message is based on timestamp information.

10. The server of claim 7, wherein the message is a voicemail message and the mailbox is a voicemail mailbox.

11. The server of claim 7, wherein:

the message further has video and the mailbox is for video mail; and

the function to edit the digital representation of the message further includes editing the video of the message in a manner corresponding to the editing of the text.

12. The server of claim 7, wherein the function to annotate the text adds at least one of text received from the user or voice received from the user.

13. The server of claim 12, wherein the function to annotate the text converts the text into corresponding audio.

21. The server of claim 7, wherein the execution of the program by the processor causes the server to perform further a function to synchronize the spectral diagram to the corresponding text.

22. The server of claim 21, wherein the server is caused to perform further functions to:

add an annotation to at least one of the spectral diagram or the text in response to one or more user inputs in relation to the presentation of the spectral diagram; and

annotate the digital representation of the message, including the audio of the message, in a manner corresponding to the annotation added to the at least one of the spectral diagram or the text.

14. A user terminal device, comprising:

an interface for network communication;

a processor coupled to the interface;

a program for the processor; and

storage for the program and for a digital representation of a message having at least audio, obtained from a mailbox of a user in a server,

wherein execution of the program by the processor causes the user terminal device to perform functions, including functions to;

process the digital representation of the message to convert speech contained in the audio of the message to corresponding text;

process the digital representation of the message to convert the audio of the message to a corresponding spectral diagram, the spectral diagram being a frequency domain representation of the audio of the message;

present a unified display on a computing device screen containing both the text and the spectral diagram to the user;

automatically edit the text in response to one or more user inputs in relation to the presentation of the text;

use the spectral diagram to remove sound in the audio that does not correspond to the text;

automatically edit, by the processor, the digital representation of the message including the audio of the message, in a manner corresponding to the editing of the text;

annotate the text in response to one or more user inputs in relation to the presentation of the text;

annotate the digital representation of the message, including the audio of the message, in a manner corresponding to the annotating of the text; and

store, in the server, the edited and annotated digital representation of the message.

15. The user terminal device of claim 14, wherein the execution of the program by the processor causes the user terminal device to perform further a function to synchronize the corresponding text to the audio of the message.

16. The user terminal device of claim 15, wherein the synchronization of the corresponding text to the audio of the message is based on timestamp information.

17. The user terminal device of claim 14, wherein the message is a voicemail message.

18. The user terminal device of claim 14, wherein:

the message further has video; and

the function to edit the digital representation of the message further includes editing the video of the message in a manner corresponding to the editing of the text.

23. The user terminal device of claim 14, wherein the execution of the program by the processor causes the user terminal device to perform further a function to synchronize the spectral diagram to the corresponding text.

24. The user terminal device of claim 23, wherein the execution of the program by the processor causes the user terminal device to perform further functions to:

add an annotation to at least one of the spectral diagram or the text in response to one or more user inputs in relation to the presentation of the spectral diagram; and

annotate the digital representation of the message, including the audio of the message, in a manner corresponding to and in response to the annotation added to the at least one of the spectral diagram or the text.