System and method for animating a digital facial model

• US 7,068,277 B2
• Filed: 05/23/2003
• Issued: 06/27/2006
• Est. Priority Date: 03/13/2003
• Status: Expired due to Term

• Current Assignee: Sony Corporation (Sony Group Corp.), Sony Pictures Entertainment Incorporated (Sony Group Corp.)
• Original Assignee: Sony Corporation (Sony Group Corp.), Sony Pictures Entertainment Incorporated (Sony Group Corp.)
• Inventors: Menache, Alberto
• Primary Examiner(s): Nguyen, Kimbinh T.
• Application Number:
  US10/444,278
• Publication Number:
  US 20040179013A1
• Time in Patent Office: 1,131 Days
• Field of Search: 345/473, 345/474, 345/660, 345/952, 345/957
• US Class Current: 345/473
• CPC Class Codes: G06T 11/00 2D [Two Dimensional] image generation
• CPC Class Codes: G06T 13/40 of characters, e.g. humans, animals or virtual beings
• CPC Class Codes: Y10S 345/952 Simulation
• CPC Class Codes: Y10S 345/957 Actor

Abstract

A system and method for animating facial motion comprises an animation processor adapted to generate three-dimensional graphical images and having a user interface and a facial performance processing system operative with the animation processor to generate a three-dimensional digital model of an actor'"'"'s face and overlay a virtual muscle structure onto the digital model. The virtual muscle structure includes plural muscle vectors that each respectively define a plurality of vertices along a surface of the digital model in a direction corresponding to actual facial muscles. The facial performance processing system is responsive to an input reflecting selective actuation of at least one of the plural muscle vectors to thereby reposition corresponding ones of the plurality of vertices and re-generate the digital model in a manner that simulates facial motion. The muscle vectors further include an origin point defining a rigid connection of the muscle vector with an underlying structure corresponding to actual cranial tissue, an insertion point defining a connection of the muscle vector with an overlying surface corresponding to actual skin, and interconnection points with other ones of the plural muscle vectors.

Claims

1. A method for animating facial motion, comprising:

generating a three-dimensional digital model of an animated face;

defining a virtual muscle structure for the digital model, said virtual muscle structure including plural muscle vectors each respectively defining a plurality of vertices lying along a surface of said digital model in a direction corresponding to that of actual facial muscles;

receiving motion capture data reflecting physical movement of an actor'"'"'s face; and

re-generating said digital model in direct response to said motion capture data to produce an actuation of at least one of said plural muscle vectors that repositions corresponding ones of said plurality of vertices and thereby simulates motion of said digital model that corresponds to said physical movement of the actor'"'"'s face.

2. The method for animating facial motion of claim 1, wherein at least one of said plural muscle vectors further includes an origin point defining a rigid connection of said muscle vector with an underlying structure corresponding to actual cranial tissue.

3. The method for animating facial motion of claim 1, wherein at least one of said plural muscle vectors further includes an insertion point defining a connection of said muscle vector with an overlying surface corresponding to actual skin.

4. The method for animating facial motion of claim 1, wherein at least one of said plural muscle vectors further includes interconnection points with other ones of said plural muscle vectors.

5. The method for animating facial motion of claim 1, further comprising defining facial marker locations for the actor'"'"'s face corresponding to said plurality of vertices.

6. The method for animating facial motion of claim 5, further comprising generating a template having holes corresponding to said defined facial marker locations for use in marking said locations on the actor'"'"'s face.

7. The method for animating facial motion of claim 1, receiving an input signal defining said actuation in addition to said motion capture data.

8. The method for animating facial motion of claim 7, wherein said input signal further comprises a user selection of at least one of said plural muscle vectors and a compression value to be applied to said selected one of said plural muscle vectors.

9. The method for animating facial motion of claim 7, wherein said input signal further comprises a user selection of pose comprised of plural ones of said plural muscle vectors and a compression value to be applied to said plural ones of said plural muscle vectors.

10. The method for animating facial motion of claim 1, further comprising calibrating said motion capture data with respect to a default data set.

11. The method for animating facial motion of claim 1, further comprising re-targeting said motion capture data for a different digital model.

12. The method for animating facial motion of claim 1, further comprising determining directional orientation of the actor'"'"'s eyes from the motion capture data and animating eyes for said digital model consistent with said determined directional orientation.

13. A system for animating facial motion, comprising:

an animation processor adapted to generate three-dimensional graphical images and having a user interface;

a motion capture processor adapted to produce motion capture data reflecting physical movement of an actor'"'"'s face; and

a facial performance processing system operative with said animation processor and said motion capture processor to generate a three-dimensional digital model of an animated face and provide a virtual muscle structure for the digital model including plural muscle vectors each respectively defining a plurality of vertices along a surface of said digital model in a direction corresponding to that of actual facial muscles, said facial performance processing system being responsive to said motion capture data to produce selective actuation of at least one of said plural muscle vectors to thereby reposition corresponding ones of said plurality of vertices and re-generate said digital model in a manner that simulates motion of said digital model in correspondence with said physical movement of the actor'"'"'s face.

14. The system of claim 13, wherein at least one of said plural muscle vectors further includes an origin point defining a rigid connection of said muscle vector with an underlying structure corresponding to actual cranial tissue.

15. The system of claim 13, wherein at least one of said plural muscle vectors further includes at least one insertion point defining a connection of said muscle vector with an overlying surface corresponding to actual skin.

16. The system of claim 13, wherein at least one of said plural muscle vectors further includes at least one interconnection point with other ones of said plural muscle vectors.

17. The system of claim 13, wherein said facial performance processing system is further operative to define facial marker locations corresponding to said plurality of vertices.

18. The system of claim 17, wherein said facial performance processing system is further operative to generate a template having holes corresponding to said defined facial marker locations for use in marking said locations onto the actor'"'"'s face.

19. The system of claim 13, wherein said input further comprises a user selection of at least one of said plural muscle vectors and a compression value to be applied to said selected one of said plural muscle vectors.

20. The system of claim 13, wherein said input further comprises a user selection of a pose comprising a combination of plural ones of said plural muscle vectors and at least one associated compression value to be applied to said plural ones of said plural muscle vectors.

21. The system of claim 13, wherein said facial performance processing system is further operative to calibrate said motion capture data with respect to a default data set.

22. The system of claim 13, wherein said facial performance processing system is further operative to re-target said motion capture data for a different digital model.

23. The system of claim 13, wherein said facial performance processing system is further operative to determine directional orientation of the actor'"'"'s eyes from the motion capture data and animate eyes for said digital model in accordance with said determined directional orientation.