System and method for capturing facial and body motion

• US 7,358,972 B2
• Filed: 11/06/2006
• Issued: 04/15/2008
• Est. Priority Date: 05/01/2003
• Status: Expired due to Term

• Current Assignee: Sony Corporation (Sony Group Corp.), Sony Pictures Entertainment Incorporated (Sony Group Corp.)
• Original Assignee: Sony Corporation (Sony Group Corp.), Sony Pictures Entertainment Incorporated (Sony Group Corp.)
• Inventors: Chen, Jerome, Hastings, Albert Robert, Echegaray, Jody, Gordon, Demian
• Primary Examiner(s): Nguyen; Kimbinh T.
• Application Number:
  US11/557,081
• Publication Number:
  US 20070058839A1
• Time in Patent Office: 526 Days
• Field of Search: 345/473, 345/474, 345/475, 345/660, 345/952, 345/957, 382/103, 382/107
• US Class Current: 345/473
• CPC Class Codes: G06T 7/246 using feature-based methods, e.g. the tracking of corners or segments
• CPC Class Codes: G06V 10/245 by locating a pattern; Special marks for positioning
• CPC Class Codes: Y10S 345/952 Simulation
• CPC Class Codes: Y10S 345/957 Actor

Abstract

A system and method for capturing motion comprises a motion capture volume adapted to contain at least one actor having body markers defining plural body points and facial markers defining plural facial points. A plurality of body motion cameras and a plurality of facial motion cameras are arranged around a periphery of the motion capture volume. The facial motion cameras each have a respective field of view narrower than a corresponding field of view of the body motion cameras. The facial motion cameras are arranged such that all laterally exposed surfaces of the actor while in motion within the motion capture volume are within the field of view of at least one of the plurality of facial motion cameras at substantially all times. A motion capture processor is coupled to the plurality of facial motion cameras and the plurality of body motion cameras to produce a digital model reflecting combined body and facial motion of the actor. At least one microphone may be oriented to pick up audio from the motion capture volume.

Claims

1. A system for capturing motion, comprising:

a motion capture volume adapted to contain at least one actor having body markers defining plural body points and facial markers defining plural facial points, the body markers being substantially larger than the facial markers;

a plurality of motion capture cameras arranged around a periphery of said motion capture volume, said motion capture cameras being arranged such that all laterally exposed surfaces of said at least one actor while in motion within said motion capture volume are within a field of view of at least one of said plurality of motion capture cameras at substantially all times; and

a motion capture processor coupled to said plurality of motion capture cameras to produce a digital representation reflecting combined facial and body motion of said at least one actor;

wherein, the body motion further comprises relatively large-scale movements of an actor'"'"'s head, torso, arms and legs, and the facial motion further comprises relatively small-scale movements of facial muscles in forming a facial expression.

2. The system for capturing motion of claim 1, wherein the plurality of motion capture cameras further comprises a plurality of body motion cameras arranged around said periphery of said motion capture volume and a plurality of facial motion cameras arranged around said periphery of said motion capture volume, wherein said facial motion cameras each have a respective field of view narrower than a corresponding field of view of said body motion cameras.

3. The system for capturing motion of claim 1, wherein said motion capture volume further comprises a rectangular area subdivided into a plurality of quadrants, said plurality of quadrants each further comprising plural edges coincident with said periphery of said motion capture volume, said plurality of motion capture cameras including first subgroups of cameras oriented toward respective ones of said edges.

4. The system for capturing motion of claim 3, wherein said plurality of motion capture cameras include second subgroups of cameras oriented toward respective ones of said edges and spatially separated from said first subgroups of cameras.

5. The system for capturing motion of claim 4, wherein said first subgroups of cameras are oriented with respect to said second subgroups of cameras so that their respective fields of view substantially overlap.

6. The system for capturing motion of claim 3, wherein said first subgroups of cameras and said second subgroups of cameras each further comprise at least three cameras.

7. The system for capturing motion of claim 3, wherein said plurality of motion capture cameras further comprise third subgroups of cameras disposed respectively at corners of said motion capture volume and oriented generally toward a center of said motion capture volume.

8. The system for capturing motion of claim 1, wherein at least a first portion of said plurality of motion capture cameras are disposed at a first height above ground, and at least a second portion of said plurality of motion capture cameras are disposed at a second height above ground greater than said first height.

9. The system for capturing motion of claim 8, wherein said first portion of said plurality of motion capture cameras are oriented slightly upward to not encompass within respective fields of view other ones of said first portion of said plurality of motion capture cameras disposed substantially across said motion capture volume.

10. The system for capturing motion of claim 2, wherein said plurality of facial motion cameras and said plurality of body motion cameras are fixed.

11. The system for capturing motion of claim 2, wherein a portion of said plurality of facial motion cameras and a portion of said plurality of body motion cameras are movable to track motion within said motion capture volume.

12. The system for capturing motion of claim 1, wherein said motion capture volume further comprises an area of approximately one hundred square feet.

13. The system for capturing motion of claim 2, wherein said plurality of facial motion cameras and said plurality of body motion cameras each further comprise polarized light sources oriented to illuminate said motion capture volume and polarized filter to block polarized light from other cameras disposed across said motion capture volume.

14. The system for capturing motion of claim 1, further comprising at least one microphone oriented to pick up audio from said motion capture volume.