System and method for capturing facial and body motion

• US 7,573,480 B2
• Filed: 12/03/2004
• Issued: 08/11/2009
• Est. Priority Date: 05/01/2003
• Status: Expired due to Term

• Current Assignee: Sony Corporation (Sony Group Corp.), Sony Pictures Entertainment Incorporated (Sony Group Corp.)
• Original Assignee: Sony Corporation (Sony Group Corp.), Sony Pictures Entertainment Incorporated (Sony Group Corp.)
• Inventors: Gordon, Demian
• Primary Examiner(s): Nguyen; Phu K
• Application Number:
  US11/004,320
• Publication Number:
  US 20050083333A1
• Time in Patent Office: 1,712 Days
• Field of Search: 345473-475, 345 7- 8, 382/107, 348/14.01, 348/14.08, 348/14.16
• US Class Current: 345/475
• CPC Class Codes: G06T 7/246 using feature-based methods, e.g. the tracking of corners or segments
• CPC Class Codes: G06T 7/596 from three or more stereo images
• CPC Class Codes: G06T 7/80 Analysis of captured images to determine intrinsic or extrinsic camera parameters, i.e. camera calibration
• CPC Class Codes: G06V 10/245 by locating a pattern; Special marks for positioning
• CPC Class Codes: G06V 40/166 using acquisition arrangements
• CPC Class Codes: H04N 5/222 Studio circuitry; Studio devices; Studio equipment cameras or camera modules comprising electronic image sensors, or control thereof H04N23/00

Abstract

A system and method for capturing motion comprises a motion capture volume adapted to contain at least one actor having body markers defining plural body points and facial markers defining plural facial points. A plurality of motion cameras are arranged around a periphery of the motion capture volume. The motion cameras are arranged such that all laterally exposed surfaces of the actor while in motion within the motion capture volume are within the field of view of at least one of the plurality of the motion cameras at substantially all times. A motion capture processor is coupled to the plurality of motion cameras to produce a digital model reflecting combined body and facial motion of the actor. At least one microphone may be oriented to pick up audio from the motion capture volume.

Claims

1. A system for capturing motion, comprising:

a motion capture volume adapted to contain at least one actor having body markers defining plural body points and facial markers defining plural facial points, said body markers being larger in size than said facial markers;

a plurality of motion cameras arranged around said periphery of said motion capture volume, said motion cameras being arranged such that all laterally exposed surfaces of said at least one actor while in motion within said motion capture volume are within a field of view of at least one of said plurality of motion cameras at substantially all times; and

a motion capture data processing system coupled to said plurality of motion cameras to simultaneously receive motion capture data from said plurality of motion cameras and to produce a digital representation reflecting combined body and facial motion of said at least one actor;

wherein, the body motion further comprises relatively large-scale muscle movements of an actor'"'"'s head, torso, arms and legs, and the facial motion further comprises relatively small-scale muscle movements of facial muscles in forming a facial expression.

2. The system for capturing motion of claim 1, wherein said motion capture volume further comprises a rectangular area subdivided into a plurality of sections.

3. The system for capturing motion of claim 2, wherein each of said plurality of sections comprise an area approximately 10 ft by 10 ft.

4. The system for capturing motion of claim 2, wherein at least forty of said motion cameras are focused onto each one of said plurality of sections.

5. The system for capturing motion of claim 1, wherein said plurality of motion cameras include a first group of cameras oriented most distant from said motion capture volume at a first height.

6. The system for capturing motion of claim 5, wherein said plurality of motion cameras include a second group of cameras oriented closer to said motion capture volume than said first group of cameras and at a second height higher than said first height.

7. The system for capturing motion of claim 6, wherein said plurality of motion cameras include a third group of cameras oriented closer to said motion capture volume than said second group of cameras and at a third height higher than said second height.

8. The system for capturing motion of claim 1, wherein said motion capture volume further comprises an area of approximately four hundred square feet.

9. The system for capturing motion of claim 1, wherein said motion capture data processing system further comprises a data network coupled to said plurality of motion cameras and adapted to generate two-dimensional (2D) data files based on signals received from said plurality of motion cameras.

10. The system for capturing motion of claim 9, wherein said motion capture data processing system further comprises a master server coupled to said data network and adapted to store said 2D data files.

11. The system for capturing motion of claim 10, wherein said motion capture data processing system further comprises a reconstruction render network coupled to said master server for performing automated reconstruction of said 2D data files and producing three-dimensional (3D) animation files that are subsequently stored by said master server.

12. The system for capturing motion of claim 10, wherein said motion capture data processing system further comprises an artists network coupled to said master server for checking said 2D data files to verify accurate data capture.

13. A method for capturing motion, comprising:

defining a motion capture volume adapted to contain at least one actor having body markers defining plural body points and facial markers defining plural facial points, said body markers being larger in size than said facial markers;

arranging a plurality of motion cameras around a periphery of said motion capture volume, said motion cameras each being arranged such that all laterally exposed surfaces of said at least one actor while in motion within said motion capture volume are within said field of view of at least one of said plurality of motion cameras at substantially all times; and

simultaneously receiving motion capture data from said plurality of motion cameras and processing the motion capture data to produce a digital representation reflecting combined body and facial motion of said at least one actor;

wherein, the body motion further comprises relatively large-scale muscle movements of an actor'"'"'s head, torso, arms and legs, and the facial motion further comprises relatively small-scale muscle movements of facial muscles in forming a facial expression.

14. The method of claim 13, wherein said defining step further comprises defining said motion capture volume as a rectangular area subdivided into a plurality of sections.

15. The method of claim 14, wherein said arranging step comprises orienting a first group of said plurality of motion cameras in a peripheral region of said motion capture volume most distant from said motion capture volume at a first height.

16. The method of claim 15, wherein said arranging step comprises orienting a second group of said plurality of motion cameras in said peripheral region closer to said motion capture volume than said first group of cameras and at a second height higher than said first height.

17. The method of claim 16, wherein said arranging step comprises orienting a third group of said plurality of motion cameras in said peripheral region closer to said motion capture volume than said second group of cameras and at a third height higher than said second height.

18. The method of claim 13, wherein said processing step further comprises generating two-dimensional (2D) data files based on signals received from said plurality of motion cameras.

19. The method of claim 18, wherein said processing step further comprises storing said 2D data files in a central data store.

20. The method of claim 18, wherein said processing step further comprises performing automated reconstruction of said 2D data files and producing three-dimensional (3D) animation files therefrom.

21. The method of claim 18, wherein said processing step further comprises verifying accuracy of said 2D data files.

22. The method of claim 13, further comprising recording audio from within said motion capture volume.