Electronic mail method and system using associated audio and visual techniques

• US 6,975,988 B1
• Filed: 11/10/2000
• Issued: 12/13/2005
• Est. Priority Date: 11/10/2000
• Status: Expired due to Fees

• Current Assignee: Gabmail IP Holdings, LLC
• Original Assignee: Adam Roth, Barclay A. Dunn, Geoffrey O'Sullivan
• Inventors: O'Sullivan, Geoffrey, Dunn, Barclay A., Roth, Adam
• Primary Examiner(s): Chawan, Vijay B.
• Application Number:
  US09/709,659
• Time in Patent Office: 1,859 Days
• Field of Search: 704/200, 704/2, 704/235, 704/270, 704/275, 704/270.1, 704/201, 379/68, 345/184, 345/848, 455/556, 455/557, 455/456.4, 370/401
• US Class Current: 704/260
• CPC Class Codes: G06Q 10/107 Computer-aided management of electronic mailing [e-mailing]
• CPC Class Codes: G10L 15/26 Speech to text systems G10L15/08 takes precedence

Abstract

A multi-mail system and method is disclosed in which a sender may convey and a recipient can realize emotional aspects associated with substantive content of a multi-mail message by receiving a message that is more than textual in nature. Voice recognition technology and programmatic relation of sound and graphics may be used to produce a talking image. In one embodiment, the image may include the user'"'"'s own visual and/or audio likeness. In an alternate embodiment, the image may comprise any available visual and/or audio display selected by the user. The multi-mail message may be inputted by a user in a text format and transposed into a format including the selected image and/or voice. In an alternate embodiment, a spoken message may be converted into a format including the selected image and/or voice. The formatted messages are then stored and/or transmitted via an email system or some other electronic network.

Claims

1. A method for preparing a multi-mail message for transmission over a network, comprising:

verifying identifier information of the sender of the multi-mail message;

allowing access to stored multimedia information of the sender;

receiving data comprising textual content of said message;

creating one or more multimedia components associated with said message, wherein said multimedia component is a likeness of the sender of the multi-mail message based on stored multimedia information of the sender;

allowing the sender to review and manipulate the one or more multimedia components; and

synthesizing said multimedia components with said textual content.

2. The method according to claim 1, wherein the multimedia component comprises audio information.

3. The method according to claim 2, wherein the audio component comprises voice data that enables the generation of sounds similar to the user'"'"'s voice speaking the words of the textual content of the message.

4. The method according to claim 2, wherein the audio component comprises voice data that enables the generation of sounds similar to a generic voice sample.

5. The method according to claim 2, wherein the audio component comprises voice data that enables the generation of any stored sound.

6. The method according to claim 2 wherein the synthesis of the multimedia component containing audio information further comprises:

parsing the audio information into sentences and for voice modulation controls;

assigning voice modulation to audio information;

sequencing phoneme and modulation information; and

translating said phoneme sequence into a sound component sequence.

7. The method according to claim 1, wherein the multimedia component comprises image information.

8. The method according to claim 7, wherein the image information may be static or dynamic.

9. The method according to claim 1, wherein the synthesis of the image multimedia component further comprises:

identifying speech movement image feature; and

generating frames representing movement of said image features.

10. The method of claim 1, wherein the multimedia component comprises audio information and image information.

11. The method according to claim 10, wherein the synthesis of the audio and image multimedia components further comprises:

composing a phoneme sequence;

composing a mouth frame time sequence which matches the phoneme time sequence;

composing speech movement image frame sequence; and

combining the image and phoneme sequences.

12. The method according to claim 10, further comprising:

changing one or more of said components to convey one or more senses of said message content.

13. The method according to claim 12, wherein the senses of said message content correspond to one or more sender emotions associated with said message.

14. The method according to claim 13, wherein the sender emotions are conveyed by manipulating one or more said image components.

15. The method according to claim 13, wherein the sender emotions are conveyed by manipulating one or more said audio components.

16. A system for preparing a multi-mail message for transmission over a network, comprising:

means for verifying identifier information of the sender of the multi-mail message;

means for allowing access to stored multimedia information of the sender;

means for receiving data comprising textual content of said message;

means for creating one or more multimedia components associated with said message, wherein said multimedia component is a likeness of the sender of the multi-mail message based on stored multimedia information of the sender;

means for allowing the sender to review and manipulate the one or more multimedia components; and

means for synthesizing said multimedia components with said textual content.

17. The system according to claim 16, wherein the multimedia component comprises audio information.

18. The system according to claim 17, wherein the audio component comprises voice data that enables the generation of sounds similar to the user'"'"'s voice.

19. The system according to claim 17, wherein the audio component comprises voice data that enables the generation of sounds similar to a generic voice sample.

20. The system according to claim 17, wherein the audio component comprises voice data that enables the generation of any stored sound.

21. The system according to claim 17 wherein the synthesis of the multimedia component containing audio information further comprises:

means for parsing the audio information into sentences and for voice modulation controls;

means for assigning voice modulation to audio information;

means for sequencing phoneme and modulation information; and

means for translating said phoneme sequence into a sound component sequence.

22. The system according to claim 16, wherein the multimedia component comprises image information.

23. The system according to claim 22, wherein the image information may be static or dynamic.

24. The system according to claim 22, wherein the synthesis of the image multimedia component further comprises:

means for identifying speech movement image feature; and

means for generating frames representing movement of said image features.

25. The system of claim 16, wherein the multimedia component comprises audio information and image information.

26. The system according to claim 25, further comprising:

means for changing one or more of said components to convey one or more senses of said message content.

27. The system according to claim 26, wherein the senses of said message content correspond to one or more sender emotions associated with said message.

28. The system according to claim 27, wherein the sender emotions are conveyed by manipulating one or more said image components.

29. The system according to claim 27, wherein the sender emotions are conveyed by manipulating one or more said audio components.

30. The system according to claim 25, wherein the synthesis of the audio and image multimedia components further comprises:

means for composing a phoneme sequence;

means for composing a mouth frame time sequence which matches the phoneme time sequence;

means for composing speech movement image frame sequence; and

means for combining the image and phoneme sequences.

31. A system for preparing a multi-mail message for transmission over a network, comprising:

a database for receiving data comprising textual content of said message; and

a central processing unit configured for

verifying identifier information of the sender of the multi-mail message;

allowing access to stored multimedia information of the sender;

creating one or more multimedia components associated with said message, wherein said multimedia component is a likeness of the sender of the multi-mail message based on stored multimedia information of the sender;

allowing the sender to review and manipulate the one or more multimedia components; and

synthesizing said multimedia components with said textual content.

32. The system according to claim 31, wherein the multimedia component comprises audio information.

33. The system according to claim 32, wherein the audio component comprises voice data that enables the generation of sounds similar to the user'"'"'s voice.

34. The system according to claim 32, wherein the audio component comprises voice data that enables the generation of sounds similar to a generic voice sample.

35. The system according to claim 32, wherein the audio component comprises voice data that enables the generation of any stored sound.

36. The system according to claim 32 wherein the central processing unit is further configured for parsing the audio information into sentences and for voice modulation controls, for assigning voice modulation to audio information, for sequencing phoneme and modulation information, and for translating said phoneme sequence into a sound component sequence.

37. The system according to claim 31, wherein the multimedia component comprises image information.

38. The system according to claim 37, wherein the image information may be static or dynamic.

39. The system according to claim 37, wherein the central processing unit is further configured for identifying speech movement image feature, and for generating frames representing movement of said image features.

40. The system of claim 31, wherein the multimedia component comprises audio information and image information.

41. The system according to claim 40, wherein the central processing unit is further configured for composing a phoneme sequence, for composing a mouth frame time sequence which matches the phoneme time sequence, for composing speech movement image frame sequence, and for combining the image and phoneme sequences.

42. The system according to claim 41, wherein the central processing unit is further configured for changing one or more of said components to convey one or more senses of said message content.

43. The system according to claim 42, wherein the senses of said message content correspond to one or more sender emotions associated with said message.

44. The system according to claim 43, wherein the central processing unit is further configured for manipulating one or more said image components to convey said sender emotions.

45. The system according to claim 43, wherein the central processing unit is further configured for manipulating one or more said audio components to convey said sender emotions.

46. Computer executable software code stored on a computer readable medium, the code for preparing a multi-mail message for transmission over a network, comprising:

code to verify identifier information of the sender of the multi-mail message;

code to allow access to stored multimedia information of the sender;

code to receive data comprising textual content of said message;

code to create one or more multimedia components associated with said message, wherein said multimedia component is a likeness of the sender of the multi-mail message based on stored multimedia information of the sender;

code to allow the sender to review and manipulate the one or more multimedia components; and

code to synthesize said multimedia components with said textual content.

47. A programmed computer for preparing a multi-mail message for transmission over a network, comprising:

a memory having at least one region for storing computer executable program code; and

a processor for executing the program code stored in memory, wherein the program code includes;

code to verify identifier information of the sender of the multi-mail message;

code to allow access to stored multimedia information of the sender;

code to receive data comprising textual content of said message;

code to create one or more multimedia components associated with said message, wherein said multimedia component is a likeness of the sender of the multi-mail message based on stored multimedia information of the sender;

code to allow the sender to review and manipulate the one or more multimedia components; and

code to synthesize said multimedia components with said textual content.

48. A computer readable medium having computer executable software code stored thereon, the code for preparing a multi-mail message for transmission over a network, comprising:

code to verify identifier information of the sender of the multi-mail message;

code to allow access to stored multimedia information of the sender;

code to receive data comprising textual content of said message;

code to create one or more multimedia components associated with said message, wherein said multimedia component is a likeness of the sender of the multi-mail message based on stored multimedia information of the sender;

code to allow the sender to review and manipulate the one or more multimedia components; and

code to synthesize said multimedia components with said textual content.