Animation control retargeting

• US 9,342,912 B1
• Filed: 06/06/2007
• Issued: 05/17/2016
• Est. Priority Date: 06/06/2007
• Status: Active Grant

• Current Assignee: Lucasfilm Limited (The Walt Disney Company)
• Original Assignee: Lucasfilm Limited (The Walt Disney Company)
• Inventors: Pighin, Frederic P., Phillips, Cary, Sullivan, Steve
• Primary Examiner(s): McDowell, Jr., Maurice L
• Assistant Examiner(s): Ricks, Donna J
• Application Number:
  US11/759,140
• Time in Patent Office: 3,268 Days
• Field of Search: None
• US Class Current: 1/1
• CPC Class Codes: G06T 13/00 Animation
• CPC Class Codes: G06T 13/40 of characters, e.g. humans, animals or virtual beings

Abstract

The subject matter of this specification can be embodied in, among other things, a method that includes determining a transform of a portion of constituent components of a source shape. The transform includes one or more states for the portion of constituent components of the source shape. The method also includes accessing a mapping function that associates the one or more states with one or more controls for a target shape, where the one or more controls configured for access by a user for manipulating constituent components of the target shape. The method includes outputting a transform for the target shape based on the one or more controls associated with the transformed constituent components of the source shape.

Claims

1. A system comprising:

a display;

an input device;

one or more processors operatively coupled to the display and the input device; and

a memory coupled with and readable by the one or more processors and storing therein a set of instructions which, when executed by the one or more processors, cause the one or more processors to;

receive a training set comprising a plurality of source-to-target mappings, wherein each source-to-target mapping corresponds to a respective state of a source object and a respective state of a target object, and wherein each source-to-target mapping comprises a unique transformation function that determines how movement information of the source object relative to a baseline position of the source object is transformed to produce movement information of the target object from a baseline position of the target object;

receive specified movement information for the source object from the baseline position of the source object;

generate, from the training set and the specified movement information for the source object, a corresponding transformation function that determines how the specified movement information of the source object is transformed to obtain a corresponding movement information specifying movement of the target object from the baseline position of the target object;

apply the generated transformation function to the specified movement information for the source object to obtain the corresponding movement information for the target object; and

create a transformed target object comprising the baseline position of the target object displaced by the corresponding movement information.

2. The system set forth in claim 1 wherein:

the source object comprises a source mesh;

the target object comprises a target mesh; and

the movement information comprises vectors relative to components of a baseline mesh representing the baseline position of the source object.

3. The system set forth in claim 2 wherein the transformation function maps displacements of the source mesh onto displacements of the target mesh at a given time.

4. The system of claim 2, wherein at least one of the vectors is specified with respect to a local coordinate system for a component of the source object.

5. The system set forth in claim 1 wherein the instructions further cause the one or more processors to:

display the transformed target object on the display along with at least one control associated with a state of the source object that enables a user to modify the transformed target object using the input device;

in response to input received from the input device that adjusts the at least one control, modify the transformed target object to create a modified target object;

display the modified target object on the display; and

modify the training set based at least in part on the modified target object.

6. The system set forth in claim 5 wherein a first control of the at least one control comprises an abstracted component of the transformed target object that permits a user to manipulate underlying constituent components of the transformed target object.

7. A non-transitory computer-readable memory storing a plurality of instructions executable by one or more processors, the plurality of instructions comprising:

instructions to receive a training set comprising a plurality of source-to-target mappings, wherein each source-to-target mapping corresponds to a respective state of a source object and a respective state of a target object, and wherein each source-to-target mapping comprises a unique transformation function that determines how movement information of the source object relative to a baseline position of the source object is transformed to produce movement information of the target object from a baseline position of the target object;

instructions to receive specified movement information for the source object from the baseline position of the source object;

instructions to generate, from the training set and the specified movement information for the source object, a corresponding transformation function that determines how the specified movement information of the source object is transformed to obtain a corresponding movement information specifying movement of the target object from the baseline position of the target object;

instructions to apply the generated transformation function to the specified movement information for the source object to obtain the corresponding movement information for the target object; and

instructions to create a transformed target object comprising the baseline position of the target object displaced by the corresponding movement information.

8. The non-transitory computer-readable memory set forth in claim 7 wherein:

the source object comprises a source mesh;

the target object comprises a target mesh; and

the movement information comprises vectors relative to components of a baseline mesh representing the baseline position of the source object.

9. The non-transitory computer-readable memory set forth in claim 8 wherein the transformation function maps displacements of the source mesh onto displacements of the target mesh at a given time.

10. The non-transitory computer-readable memory of claim 8, wherein at least one of the vectors is specified with respect to a local coordinate system for a component of the source object.

11. The non-transitory computer-readable memory of claim 7, the plurality of instructions further comprising:

instructions to display the transformed target object on the display along with at least one control associated with a state of the source object that enables a user to modify the transformed target object using the input device;

instructions that, in response to input received from the input device that adjusts the at least one control, modify the transformed target object to create a modified target object;

instructions to display the modified target object on the display; and

instructions to modify the training set based at least in part on the modified target object.

12. The non-transitory computer-readable memory set forth in claim 11 wherein a first control of the at least one control comprises an abstracted component of the transformed target object that permits a user to manipulate underlying constituent components of the transformed target object.

13. A method comprising:

receiving, by one or more processors, a training set comprising a plurality of source-to-target mappings, wherein each source-to-target mapping corresponds to a respective state of a source object and a respective state of a target object, and wherein each source-to-target mapping comprises a unique transformation function that determines how movement information of the source object relative to a baseline position of the source object is transformed to produce movement information of the target object from a baseline position of the target object;

receiving, by the one or more processors, specified movement information for the source object from the baseline position of the source object;

generating, by the one or more processors and from the training set and the specified movement information for the source object, a corresponding transformation function that determines how the specified movement information of the source object is transformed to obtain a corresponding movement information specifying movement of the target object from the baseline position of the target object;

applying, by the one or more processors, the generated transformation function to the specified movement information for the source object to obtain the corresponding movement information for the target object; and

creating, by the one or more processors, a transformed target object comprising the baseline position of the target object displaced by the corresponding movement information.

14. The method set forth in claim 13 wherein:

the source object comprises a source mesh;

the target object comprises a target mesh; and

the movement information comprises vectors relative to components of a baseline mesh representing the baseline position of the source object.

15. The method set forth in claim 14 wherein the transformation function maps displacements of the source mesh onto displacements of the target mesh at a given time.

16. The method of claim 14, wherein at least one of the vectors is specified with respect to a local coordinate system for a component of the source object.

17. The method set forth in claim 13 wherein the transformation function is determined by interpolating between multiple states of the source object in the training set.

18. The method of claim 13, further comprising:

displaying the transformed target object on the display along with at least one control associated with a state of the source object that enables a user to modify the transformed target object using the input device;

in response to input received from the input device that adjusts the at least one control, modifying the transformed target object to create a modified target object;

displaying the modified target object on the display; and

modifying the training set based at least in part on the modified target object.

19. The method set forth in claim 18 wherein a first control of the at least one control comprises an abstracted component of the transformed target object that permits an animation artist to manipulate underlying constituent components of the transformed target object.

20. The method set forth in claim 18 wherein a first control of the at least one control adjusts a percentage contribution of an expression of a face of the modified target object.

21. The method set forth in claim 18 wherein a first control of the at least one control manipulates an individual muscle or group of muscles of a face of the modified target object.

22. The method set forth in claim 13 wherein the target object comprises a face of a computer-generated character.

23. The method set forth in claim 13 wherein the movement information is generated from motion capture data.