System and method for augmenting hand animation with three-dimensional secondary motion

• US 8,988,422 B1
• Filed: 12/17/2010
• Issued: 03/24/2015
• Est. Priority Date: 12/17/2010
• Status: Active Grant

• Current Assignee: Disney Enterprises Incorporated (The Walt Disney Company)
• Original Assignee: Disney Enterprises Incorporated (The Walt Disney Company)
• Inventors: Hodgins, Jessica Kate, Sheikh, Yaser, Jain, Eakta
• Primary Examiner(s): Nguyen, Phu K
• Application Number:
  US12/972,126
• Time in Patent Office: 1,558 Days
• Field of Search: 345/419, 345/473, 345/474, 345/475
• US Class Current: 345/419
• CPC Class Codes: G06T 13/40 of characters, e.g. humans, animals or virtual beings
• CPC Class Codes: G06T 13/80 2D [Two Dimensional] animation, e.g. using sprites

Abstract

Techniques are disclosed for augmenting hand-drawn animation of human characters with three-dimensional (3D) physical effects to create secondary motion. Secondary motion, or the motion of objects in response to that of the primary character, is widely used to amplify the audience'"'"'s response to the character'"'"'s motion and to provide a connection to the environment. These 3D effects are largely passive and tend to be time consuming to animate by hand, yet most are very effectively simulated in current animation software. The techniques enable hand-drawn characters to interact with simulated objects such as cloth and clothing, balls and particles, and fluids. The driving points or volumes for the secondary motion are tracked in two dimensions, reconstructed into three dimensions, and used to drive and collide with the simulated objects.

Claims

1. A method of constructing a three-dimensional animation, comprising:

receiving a human-generated two-dimensional animation depicting an object in one or more scenes;

receiving existing three-dimensional motion data that is generated from a separate source than the two-dimensional animation and stored in a library of three-dimensional motion data;

constructing a collision volume associated with the object that is configured to interact with a simulated three-dimensional object, wherein the collision volume is formed by rotating a bounding surface about an axis that passes through at least two virtual markers associated with the object; and

constructing a three-dimensional animation using image plane coordinates from the two-dimensional animation and depth coordinates from the three-dimensional motion data.

2. The method of claim 1, further comprising:

receiving annotations including (x,y) positions of the virtual markers associated with the two-dimensional animation; and

associating the virtual markers in the two-dimensional animation with (x,y,z) positions of the three-dimensional motion data.

3. The method of claim 2, further comprising determining a z coordinate of a first (x,y) position of a first virtual marker based on a first (x,y,z) position of the three-dimensional motion data that is associated with the first virtual marker as a depth value of the first virtual marker.

4. The method of claim 2, further comprising:

rendering the simulated three-dimensional object that interacts with the collision volume to generate a rendered three-dimensional object with secondary motion.

5. The method of claim 2, combining the two-dimensional animation with the rendered three-dimensional object to generate a two-dimensional animation augmented with secondary motion.

6. The method of claim 2, further comprising:

determining z coordinates of (x,y) positions of the virtual markers based on (x,y,z) positions of the three-dimensional motion data to generate depth values for the respective virtual markers; and

performing depth compositing using the depth values to combine the two-dimensional animation with a rendered three-dimensional object.

7. The method of claim 2, wherein a first virtual marker is located on the object at a collision point between the object and the simulated three-dimensional object.

8. The method of claim 2, wherein a first virtual marker is located on the object at an attachment point between the object and the simulated three-dimensional object.

9. The method of claim 2, further comprising:

determining z coordinates of (x,y) positions of the virtual markers based on (x,y,z) positions of the three-dimensional motion data to generate depth values for the respective virtual markers and to generate the two-dimensional animation back-projected into three-dimensional space; and

rendering the two-dimensional animation projected into three-dimensional space as a first image of a stereo image pair from a first camera view; and

rendering the two-dimensional animation back-projected into three-dimensional space as a second image of the stereo image pair from a second camera view.

10. The method of claim 2, further comprising processing or filtering the a z position of the three-dimensional motion data before determining a z coordinate of a first (x,y) position of a first virtual marker.

11. The method of claim 1, wherein the object is a character and (x,Y,z) positions of the three-dimensional motion data are associated with virtual markers on the character in the two-dimensional animation.

12. The method of claim 1, wherein the three-dimensional motion data comprises motion of a camera.

13. The method of claim 1, wherein the three-dimensional motion data is motion capture data.

14. The method of claim 1, wherein the three-dimensional motion data is keyframe data.

15. The method of claim 1, further comprising:

rendering the two-dimensional animation projected into three-dimensional space as a first image of a stereo image pair from a first camera view; and

rendering the two-dimensional animation back-projected into three-dimensional space as a second image of the stereo image pair from a second camera view.

16. The method of claim 1, wherein the three-dimensional motion data is selected by a user.

17. The method of claim 1, wherein the two-dimensional animation is a hand-drawn animation.

18. The method of claim 1, wherein the three-dimensional motion data comprises a combination of motion capture segments.

19. A computer-readable storage medium storing instructions that, when executed by a processor, cause a computer system to perform an operation for constructing a three-dimensional animation, the operation comprising:

receiving a human-generated two-dimensional animation depicting an object in one or more scenes;

receiving existing three-dimensional motion data that is generated from a separate source than the two-dimensional animation and stored in a library of three-dimensional motion data;

constructing a collision volume associated with the object that is configured to interact with a simulated three-dimensional object, wherein the collision volume is formed by rotating a bounding surface about an axis that passes through at least two virtual markers associated with the object; and

constructing a three-dimensional animation using image plane coordinates from the two-dimensional animation and depth coordinates from the three-dimensional motion data.

20. A method of constructing a two-dimensional animation augmented with secondary motion, comprising:

receiving a human-generated two-dimensional animation;

receiving depth values from computer-generated three-dimensional motion data;

constructing a collision volume of an object in the two-dimensional animation, wherein the collision volume is formed by rotating a bounding surface about an axis that passes through at least two virtual markers associated with the object;

rendering a simulated three-dimensional object that interacts with the collision volume to generate a rendered three-dimensional object with secondary motion; and

combining the two-dimensional animation with the rendered three-dimensional object to generate the two-dimensional animation augmented with secondary motion.

21. The method of claim 20, further comprising receiving annotations including (x,y) positions of the virtual markers located on the object that are associated with respective (x,y,z) positions of the three-dimensional motion data.

22. The method of claim 21, further comprising determining a z coordinates of the (x,y) positions of the virtual markers based on the respective (x,y,z) positions of the three-dimensional motion data to generate depth values for the virtual markers.

23. The method of claim 21, wherein a first virtual marker is located on the object at a collision point between the object and the simulated three-dimensional object.

24. The method of claim 21, wherein a first virtual marker is located on the object at an attachment point between the object and the simulated three-dimensional object.

25. The method of claim 21, further comprising:

determining z coordinates of (x,y) positions of the virtual markers based on (x,y,z) positions of the three-dimensional motion data to generate depth values for the respective virtual markers; and

performing depth compositing using the depth values to combine the two-dimensional animation with the rendered three-dimensional object.

26. The method of claim 20, wherein the object is a character and (x,y,z) positions of the three-dimensional motion data are associated with virtual markers on the character in the two-dimensional animation.

27. The method of claim 20, wherein the two-dimensional animation is a hand-drawn animation.

28. The method of claim 20, further comprising:

rendering the two-dimensional animation projected into three-dimensional space as a first image of a stereo image pair from a first camera view; and

rendering the two-dimensional animation back-projected into three-dimensional space as a second image of the stereo image pair from a second camera view.

29. The method of claim 20, wherein the three-dimensional motion data comprises motion of a camera.

30. A computer-readable storage medium storing instructions that, when executed by a processor, cause a computer system to perform an operation for constructing two-dimensional animation augmented with secondary motion, the operation comprising:

receiving a human-generated two-dimensional animation;

receiving computer-generated three-dimensional motion data;

constructing a collision volume of an object in the two-dimensional animation, wherein the collision volume is formed by rotating a bounding surface about an axis that passes through at least two virtual markers associated with the object;

rendering a simulated three-dimensional object that interacts with the collision volume to generate a rendered three-dimensional object with secondary motion; and

combining the two-dimensional animation with the rendered three-dimensional object to generate the two-dimensional animation augmented with secondary motion.