Facial animation system and production method

• US 8,648,866 B2
• Filed: 07/07/2010
• Issued: 02/11/2014
• Est. Priority Date: 12/16/2009
• Status: Active Grant

• Current Assignee: Industrial Technology Research Institute
• Original Assignee: Industrial Technology Research Institute
• Inventors: Ting, Wen-Hung, Yen, Chen-Lan, Liao, Duan-Li, Chi, Wen-Liang
• Primary Examiner(s): Tung, Kee M
• Assistant Examiner(s): Du, Haixia
• Application Number:
  US12/831,666
• Publication Number:
  US 20110141105A1
• Time in Patent Office: 1,315 Days
• Field of Search: None
• US Class Current: 345/473
• CPC Class Codes: G06T 13/40 of characters, e.g. humans, animals or virtual beings

Abstract

A facial animation production method for producing 3-dimensional (3D) facial animation data in response to input video data includes the following steps. First, data positioning and character sorting processes are performed on the input video data to acquire first-layer character data, for indicating multiple first-layer character points, and first-layer model data. Next, first-layer model outline data and first-layer character outline data are respectively obtained according to the first-layer model data and the first-layer character data. Then, the first-layer character outline data is compared with the first-layer model outline data to judge whether a judgment condition is satisfied. If not, output character data are produced according to the first-layer character data, and fundamental facial-mesh transformation data are thus produced. Thereafter, the 3D facial animation data are displayed according to the fundamental facial-mesh transformation data.

Claims

1. A facial animation system for producing 3-dimensional (3D) facial animation data in response to input video data, the input video data comprising a first frame, the facial animation system comprising:

a display module;

a transformation calculating module;

a character acquiring device for performing data positioning and character sorting operations on the input video data to acquire first-layer character data according to the peak value of the variance of pixel data gray level of the input video data, for indicating a plurality of first-layer character points, and performing data positioning and character sorting operations on the first frame of the input video data to acquire first-layer model data; and

a character verifying device for obtaining first-layer model outline data according to the first-layer model data, and obtaining first-layer character outline data according to the first-layer character data, the character verifying device further comparing the first-layer character outline data with the first-layer model outline data to judge whether the first-layer character outline data and the first-layer model outline data satisfy a judgment condition;

wherein when the first-layer character outline data and the first-layer model outline data do not satisfy the judgment condition, the character verifying device outputs the first-layer character data, as output character data, to the transformation calculating module;

wherein the transformation calculating module produces fundamental facial-mesh transformation data according to the output character data, and thus drives the display module to display the 3D facial animation data.

2. The system according to claim 1, wherein the transformation calculating module comprises:

a simulating device for defining a first-layer virtual muscle group on a fundamental facial mesh model based on facial muscle distribution information; and

a transformation calculating device for calculating displacement quantities and displacement directions corresponding to the first-layer character points according to the output character data, and thus producing first-layer driving information and driving the first-layer virtual muscle group to control the fundamental facial mesh model to produce corresponding transformation and thus to produce the fundamental facial-mesh transformation data.

3. The system according to claim 2, wherein a spring model is applied between two nodes on the fundamental facial mesh model, so that the first-layer virtual muscle group can control the fundamental facial mesh model to produce the corresponding transformation in response to the first-layer driving information to produce the fundamental facial-mesh transformation data.

4. The system according to claim 1, wherein the display module comprises:

a setting device for mapping the fundamental facial-mesh transformation data to actual output facial-mesh transformation data; and

an output device for displaying the 3D facial animation data according to the actual output facial-mesh transformation data.

5. The system according to claim 1, wherein when the first-layer character outline data and the first-layer model outline data satisfy the judgment condition, the character verifying device drives the character acquiring device to acquire second-layer model data and second-layer character data, for indicating a plurality of second-layer character points, from the input video data;

wherein the character verifying device further obtains second-layer character outline data and second-layer model outline data according to the second-layer character data and the second-layer model data, respectively, and compares the second-layer character outline data with the second-layer model outline data to judge whether the second-layer character outline data and the second-layer model outline data satisfy the judgment condition.

6. The system according to claim 5, wherein when the second-layer character outline data and the second-layer model outline data do not satisfy the judgment condition, the character verifying device outputs the first and second-layer character data, as the output character data, to the transformation calculating module, and the transformation calculating module produces the fundamental facial-mesh transformation data according to the output character data, and thus drives the display module to display the 3D facial animation data.

7. The system according to claim 6, wherein the transformation calculating module comprises:

a simulating device for defining a second-layer virtual muscle group on a fundamental facial mesh model based on facial muscle distribution information; and

a transformation calculating device for calculating displacement quantities and displacement directions corresponding to the second-layer character points according to the output character data, and thus producing second-layer driving information to drive the second-layer virtual muscle group to control the fundamental facial mesh model to produce corresponding transformation and thus to produce the fundamental facial-mesh transformation data.

8. The system according to claim 7, wherein a spring model is applied between two nodes on the fundamental facial mesh model, so that the second-layer virtual muscle group can control the fundamental facial mesh model to produce the corresponding transformation in response to the second-layer driving information.

9. The system according to claim 6, wherein the display module comprises:

a setting device for mapping the fundamental facial-mesh transformation data to actual output facial-mesh transformation data; and

an output device for displaying the 3D facial animation data according to the actual output facial-mesh transformation data.

10. The system according to claim 5, wherein when the second-layer character outline data and the second-layer model outline data satisfy the judgment condition, the character verifying device drives the character acquiring device to acquire third-layer model data and third-layer character data, for indicating a plurality of third-layer character points, from the input video data;

wherein the character verifying device further obtains third-layer character outline data and third-layer model outline data according to the third-layer character data and the third-layer model data, respectively, and compares the third-layer character outline data with the third-layer model outline data to judge whether the third-layer character outline data and the third-layer model outline data satisfy the judgment condition.

11. The system according to claim 10, wherein when the third-layer character outline data and the third-layer model outline data do not satisfy the judgment condition, the character verifying device outputs the third-layer character data, as the output character data, to the transformation calculating module, and the transformation calculating module produces the fundamental facial-mesh transformation data according to the output character data, and thus drives the display module to display the 3D facial animation data.

12. The system according to claim 11, wherein the transformation calculating module comprises:

a simulating device for defining a third-layer virtual muscle group on a fundamental facial mesh model based on facial muscle distribution information; and

a transformation calculating device for calculating displacement quantities and displacement directions corresponding to the third-layer character points according to the output character data, and thus producing third-layer driving information to drive the third-layer virtual muscle group to control the fundamental facial mesh model to produce corresponding transformation and thus to produce the fundamental facial-mesh transformation data.

13. The system according to claim 12, wherein a spring model is applied between two nodes of the fundamental facial mesh model, so that the third-layer virtual muscle group can control the fundamental facial mesh model to produce the corresponding transformation in response to the third-layer driving information.

14. The system according to claim 11, wherein the display module comprises:

a setting device for mapping the fundamental facial-mesh transformation data to actual output facial-mesh transformation data; and

an output device for displaying the 3D facial animation data according to the actual output facial-mesh transformation data.

15. The system according to claim 1, wherein the character acquiring device and the character verifying device are disposed at a remote end of the display module and the transformation calculating module, and the facial animation system further comprises:

a first communication device for receiving the output character data and outputting the output character data through a communication link; and

a second communication device for receiving the output character data through the communication link, and providing the output character data to the transformation calculating module and the display module.

16. The system according to claim 1, wherein the character acquiring device, the character verifying device and the transformation calculating module are disposed at a remote end of the display module, and the facial animation system further comprises:

a first communication device for receiving the fundamental facial-mesh transformation data and outputting the fundamental facial-mesh transformation data through a communication link; and

a second communication device for receiving the fundamental facial-mesh transformation data through the communication link and providing the fundamental facial-mesh transformation data to the display module.

17. A facial animation production method for producing 3-dimensional (3D) facial animation data in response to input video data, the input video data comprising a first frame, the facial animation production method comprising the steps of:

performing data positioning and character sorting processes on the input video data to acquire first-layer character data according to the peak value of the variance of pixel data gray level of the input video data, for indicating a plurality of first-layer character points, and performing data positioning and character sorting operations on the first frame of the input video data to acquire first-layer model data;

obtaining first-layer model outline data according to the first-layer model data, and obtaining first-layer character outline data according to the first-layer character data;

comparing the first-layer character outline data with the first-layer model outline data to judge whether the first-layer character outline data and the first-layer model outline data satisfy a judgment condition;

producing output character data according to the first-layer character data and thus producing fundamental facial-mesh transformation data when the first-layer character outline data and the first-layer model outline data do not satisfy the judgment condition; and

displaying the 3D facial animation data according to the fundamental facial-mesh transformation data.

18. The method according to claim 17, wherein the step of producing the fundamental facial-mesh transformation data comprises:

defining a first-layer virtual muscle group on a fundamental facial mesh model based on facial muscle distribution information; and

calculating displacement quantities and displacement directions corresponding to the first-layer character points according to the output character data, and thus producing first-layer driving information to drive the first-layer virtual muscle group to control the fundamental facial mesh model to produce corresponding transformation and thus to produce the fundamental facial-mesh transformation data.

19. The method according to claim 18, wherein in the step of producing the fundamental facial-mesh transformation data, a spring model is applied between two nodes on the fundamental facial mesh model.

20. The method according to claim 17, wherein the step of displaying the 3D facial animation data comprises:

mapping the fundamental facial-mesh transformation data to actual output facial-mesh transformation data; and

displaying the 3D facial animation data according to the actual output facial-mesh transformation data.

21. The method according to claim 17, further comprising the steps of:

acquiring second-layer character data, for indicating a plurality of second-layer character points, and second-layer model data from the input video data when the first-layer character outline data and the first-layer model outline data satisfy the judgment condition;

obtaining second-layer model outline data according to the second-layer model data, and obtaining second-layer character outline data according to the second-layer character data; and

comparing the second-layer character outline data with the second-layer model outline data to judge whether the second-layer character outline data and the second-layer model outline data satisfy the judgment condition.

22. The method according to claim 21, further comprising the step of:

producing the fundamental facial-mesh transformation data according to the second-layer character data when the second-layer character outline data and the second-layer model outline data do not satisfy the judgment condition.

23. The method according to claim 21, further comprising the steps of:

acquiring third-layer character data, for indicating a plurality of third-layer character points, and third-layer model data from the input video data when the second-layer character outline data and the second-layer model outline data satisfy the judgment condition;

obtaining third-layer model outline data according to the third-layer model data, and obtaining third-layer character outline data according to the third-layer character data; and

comparing the third-layer character outline data with the third-layer model outline data to judge whether the third-layer character outline data and the third-layer model outline data satisfy the judgment condition.

24. The method according to claim 23, further comprising the step of:

producing the fundamental facial-mesh transformation data according to the third-layer character data when the third-layer character outline data and the third-layer model outline data do not satisfy the judgment condition.