Text to visual speech system and method incorporating facial emotions

• US 20020194006A1
• Filed: 03/29/2001
• Published: 12/19/2002
• Est. Priority Date: 03/29/2001
• Status: Abandoned Application

• Current Assignee: Koninklijke Philips Electronics N.V. (Koninklijke Philips N.V.)
• Original Assignee: Koninklijke Philips Electronics N.V. (Koninklijke Philips N.V.)
• Inventors: Challapali, Kiran
• Application Number:
  US09/821,138
• Publication Number:
  US 20020194006A1
• Time in Patent Office: Days
• US Class Current: 704/276
• CPC Class Codes: G06T 13/40 of characters, e.g. humans, animals or virtual beings

Abstract

A visual speech system for converting emoticons into facial expressions on a displayable animated facial image. The system comprises: (1) a data import system for receiving text data that includes at least one emoticon string, wherein the at least one emoticon string is associated with a predetermined facial expression; and (2) a text-to-animation system for generating a displayable animated face image that can simulate at least one facial movement corresponding to the predetermined facial expression. The system is preferably implemented remotely over a network, such as in an on-line chat environment.

Claims

1. A visual speech system, wherein the visual speech system comprises:

a data import system for receiving text data that includes word strings and emoticon strings; and

a text-to-animation system for generating a displayable animated face image that can reproduce facial movements corresponding to the received word strings and the received emoticon strings.

2. The visual speech system of claim 1, further comprising a keyboard for typing in text data.

3. The visual speech system of claim 1, further comprising a text-to-audio system that can generate an audio speech broadcast corresponding the received word strings.

4. The visual speech system of claim 3, further comprising an audio-visual interface for displaying the display able animated face image along with the audio speech broadcast.

5. The visual speech system of claim 1, wherein the text-to-animation system associates each emoticon string with an expressed emotion, and wherein the expressed emotion is reproduced on the animated face image with at least one facial movement.

6. The visual speech system of claim 5, wherein the text-to-animation system associates each word string with a spoken word, and wherein the spoken word is reproduced on the animated face image with at least one mouth movement.

7. The visual speech system of claim 6, wherein the at least one facial movement is morphed with the at least one mouth movement.

8. The visual speech system of claim 1, further comprising an input/output system for receiving and sending text data over a network.

9. A program product stored on a recordable medium, which when executed provides a visual speech system, comprising:

a data import system for receiving text data that includes word strings and emoticon strings; and

a text-to-animation system for generating a displayable animated face image that can reproduce facial movements corresponding to the received word strings and the received emoticon strings.

10. The program product of claim 9, wherein an inputted emoticon string is reproduced on the animated face image as an expressed emotion.

11. The program product of claim 10, wherein an inputted word string is reproduced on the animated face image by mouth movements.

12. The program product of claim 11, wherein the expressed emotion is morphed with the mouth movements.

14. The online chat system of claim 13, wherein each emoticon string is reproduced on the animated face image as an expressed emotion.

15. The online chat system of claim 14, wherein each word string is reproduced on the animated face image by mouth movements.

16. The online chat system of claim 15, wherein the expressed emotion is morphed with the mouth movements.

18. The method of claim 17, wherein the mouth movements and facial movements are morphed together.

19. The method of claim 17, wherein the displaying of the animated face image along with the broadcast of the audio speech is done remotely over a network.

13. An online chat system having visual speech capabilities, comprising:

a first networked client having;

a first data import system for receiving text data that includes word strings and emoticon strings; and

a data ex(port system for sending the text data to a network; and

a second networked client having;

a second data import system for receiving the text data from the network; and

a text-to-animation system for generating a displayable animated face image that reproduces facial movements corresponding to the received word strings and the received emoticon strings contained in the text data.

17. A method of performing visual speech on a system having a displayable animated face image, comprising the steps of:

entering text data into a keyboard, wherein the text data includes word strings and emoticon strings;

converting the word strings to audio speech;

converting the word strings to mouth movements on the displayable animated face image, such that the mouth movements correspond with the audio speech;

converting the emoticon strings to facial movements on the displayable animated face image, such that the facial movements correspond with expressed emotions associated with the entered emoticon strings; and

displaying the animated face image along with a broadcast of the audio speech.

20. A visual speech system, comprising:

a data import system for receiving text data that includes at least one emoticon string, wherein the at least one emoticon string is associated with a predetermined facial expression; and

a text-to-animation system for generating a displayable animated face image that can simulate at least one facial movement corresponding to the predetermined facial expression.