Apparatus and method for capturing the expression of a performer

• US 20060055699A1
• Filed: 09/15/2004
• Published: 03/16/2006
• Est. Priority Date: 09/15/2004
• Status: Active Grant

• Current Assignee: Rearden Mova, LLC
• Original Assignee: OL2 Incorporated (Sony Group Corp.)
• Inventors: Perlman, Stephen G., Cotter, Tim S., Speck, John, LaSalle, Greg, Pearce, Kenneth A.
• Granted Patent: US 8,194,093 B2
• Time in Patent Office: Days
• US Class Current: 345/473
• CPC Class Codes: B33Y 50/00 Data acquisition or data processing for additive manufacturing
• CPC Class Codes: B33Y 80/00 Products made by additive manufacturing
• CPC Class Codes: G06T 17/10 Constructive solid geometry [CSG] using solid primitives, e.g. cylinders, cubes
• CPC Class Codes: G06T 7/246 using feature-based methods, e.g. the tracking of corners or segments
• CPC Class Codes: G06V 10/245 by locating a pattern; Special marks for positioning
• CPC Class Codes: G06V 40/103 Static body considered as a whole, e.g. static pedestrian or occupant recognition
• CPC Class Codes: G06V 40/166 using acquisition arrangements

Abstract

A method is described comprising: applying a series of curves on specified regions of a performer'"'"'s face; tracking the movement of the series of curves during a motion capture session; and generating motion data representing the movement of the performer'"'"'s face using the tracked movement of the series of curves.

Claims

1. A method comprising:

applying a series of curves on specified regions of a performer'"'"'s face;

tracking the movement of the series of curves during a motion capture session; and

generating motion data representing the movement of the performer'"'"'s face using the tracked movement of the series of curves.

2. The method as in claim 1 wherein the curves are comprised of a retro-reflective material.

3. The method as in claim 1 wherein two or more different colors are used for different curves applied to different portions of the performer'"'"'s face.

4. The method as in claim 3 further comprising:

identifying one or more of the curves based on the color of the curves.

5. The method as in claim 1 wherein applying further comprises:

creating a mask having slits corresponding to the curves;

placing the mask over the performer'"'"'s face; and

applying the curves through the slits in the mask.

6. The method as in claim 1 wherein tracking comprises capturing a video of the curves from two or more different angles and wherein generating motion data comprises:

generating two-dimensional (“

2-D”

) data representing the motion of the curves in two dimensions from each of the two or more different angles; and

using the 2-D data from the two or more different angles to generate three-dimensional (“

3-D”

) data representing the motion of the curves in three dimensions.

7. The method as in claim 6 further comprising:

storing the 2-D data and 3-D data in a .OBJ file format.

8. A method comprising:

capturing video of a plurality of curves painted on a performer'"'"'s face during a motion capture session;

identifying each of the curves and intersection points of the curves within frames of the captured video; and

generating motion data describing the motion of each of the curves and/or intersection points over time during the motion capture session.

9. The method as in claim 8 wherein capturing further comprises:

generating two or more video streams of the plurality of curves, the two or more video streams captured from two or more different angles; and

storing the two or more streams to a mass storage device.

10. The method as in claim 9 wherein the video streams are stored in the Audio Video Interleaved (“

AVI”

) format.

11. The method as in claim 8 wherein identifying further comprises:

cleaning the video frames to increase contrast between the curves and other image data within the video frames.

12. The method as in claim 8 wherein identifying further comprises:

correlating the curve images from the captured frames with curve data provided in a curve data file; and

identifying curves having the highest correlation to corresponding curves in the curve data file.

13. The method as in claim 12 wherein correlating further comprises:

comparing a color of each of the curves in the captured video frames to a known color of each of the curves stored within the curve data file.

14. The method as in claim 8 further comprising:

quantizing the identified curves into a plurality of line segments based on a specified resolution;

using the intersection points of the line segments as the vertices of planar triangles to create a geometric mesh; and

generating motion data describing the geometric mesh.

15. The method as in claim 9 further comprising:

storing two-dimensional (“

2-D”

) data for each of the two or more video streams; and

analyzing the 2-D data from each of the two or more video streams to generate 3-D data describing the motion of the curves in three dimensions during the performance.

16. The method as in claim 15 wherein the 2-D data and 3-D data are stored in a .OBJ file format.

18. The machine-readable medium as in claim 8 wherein capturing further comprises:

generating two or more video streams of the plurality of curves, the two or more video streams captured from two or more different angles; and

storing the two or more streams to a mass storage device.

19. The machine-readable medium as in claim 9 wherein the video streams are stored in the Audio Video Interleaved (“

AVI”

) format.

20. The machine-readable medium as in claim 8 wherein identifying further comprises:

cleaning the video frames to increase contrast between the curves and other image data within the video frames.

21. The machine-readable medium as in claim 8 wherein identifying further comprises:

correlating the curve images from the captured frames with curve data provided in a curve data file; and

identifying curves having the highest correlation to corresponding curves in the curve data file.

22. The machine-readable medium as in claim 21 wherein correlating further comprises:

comparing a color of each of the curves in the captured video frames to a known color of each of the curves stored within the curve data file.

23. The machine-readable medium as in claim 8 further comprising:

quantizing the identified curves into a plurality of line segments based on a specified resolution;

using the intersection points of the line segments as the vertices of planar triangles to create a geometric mesh; and

generating motion data describing the geometric mesh.

24. The machine-readable medium as in claim 9 further comprising:

storing two-dimensional (“

2-D”

) data for each of the two or more video streams; and

analyzing the 2-D data from each of the two or more video streams to generate 3-D data describing the motion of the curves in three dimensions during the performance.

25. The machine-readable medium as in claim 15 wherein the 2-D data and 3-D data are stored in a .OBJ file format.

17. A machine-readable medium having program code stored thereon which, when executed by a machine, causes the machine to perform the operations of:

capturing video of a plurality of curves painted on a performer'"'"'s face during a motion capture session;

identifying each of the curves and intersection points of the curves within frames of the captured video; and

generating motion data describing the motion of each of the curves and/or intersection points over time during the motion capture session.

26. A system comprising:

a plurality of camera controllers to capture video of a plurality of curves painted on a performer'"'"'s face during a motion capture session;

curve identification logic identifying each of the curves and intersection points of the curves within frames of the captured video; and

motion capture logic generating motion data describing the motion of each of the curves and/or intersection points over time during the motion capture session.