System and method for capturing facial and body motion

• US 7,218,320 B2
• Filed: 05/01/2003
• Issued: 05/15/2007
• Est. Priority Date: 03/13/2003
• Status: Expired due to Term

• Current Assignee: Sony Corporation (Sony Group Corp.), Sony Pictures Entertainment Incorporated (Sony Group Corp.)
• Original Assignee: Sony Corporation (Sony Group Corp.), Sony Pictures Entertainment Incorporated (Sony Group Corp.)
• Inventors: Gordon, Demian, Chen, Jerome, Echegaray, Jody, Hastings, Albert Robert
• Primary Examiner(s): Nguyen; Kimbinh T.
• Application Number:
  US10/427,114
• Publication Number:
  US 20040179008A1
• Time in Patent Office: 1,475 Days
• Field of Search: 345/419, 345/473, 345/475, 348/13, 382/107, 382/236, 382/276, 382/103, 382/237
• US Class Current: 345/419
• CPC Class Codes: G06F 3/011 Arrangements for interaction with the human body, e.g. for user immersion in virtual reality blind teaching G09B21/00
• CPC Class Codes: G06T 2207/10016 Video; Image sequence
• CPC Class Codes: G06T 2207/30201 Face
• CPC Class Codes: G06T 2207/30204 Marker
• CPC Class Codes: G06T 7/246 using feature-based methods, e.g. the tracking of corners or segments
• CPC Class Codes: G06V 10/10 Image acquisition document image scanning and transmission H04N1/00; control of digital cameras H04N23/60

Abstract

A system and method for capturing motion comprises a motion capture volume adapted to contain at least one actor having body markers defining plural body points and facial markers defining plural facial points. A plurality of body motion cameras and a plurality of facial motion cameras are arranged around a periphery of the motion capture volume. The facial motion cameras each have a respective field of view narrower than a corresponding field of view of the body motion cameras. The facial motion cameras are arranged such that all laterally exposed surfaces of the actor while in motion within the motion capture volume are within the field of view of at least one of the plurality of facial motion cameras at substantially all times. A motion capture processor is coupled to the plurality of facial motion cameras and the plurality of body motion cameras to produce a digital model reflecting combined body and facial motion of the actor. At least one microphone may be oriented to pick up audio from the motion capture volume.

Claims

1. A system for capturing combined facial and body motion, comprising:

a motion capture volume adapted to contain at least one actor having body markers defining plural body points and facial markers defining plural facial points;

a plurality of body motion cameras arranged around a periphery of said motion capture volume;

a plurality of facial motion cameras arranged around said periphery of said motion capture volume, said facial motion cameras each having a respective field of view narrower than a corresponding field of view of said body motion cameras;

said facial motion cameras being arranged such that all laterally exposed surfaces of said at least one actor while in motion within said motion capture volume are within said field of view of at least one of said plurality of facial motion cameras at substantially all times; and

a motion capture processor coupled to said plurality of body motion cameras and said plurality of facial motion cameras to simultaneously receive motion capture data from said plurality of body motion cameras and said plurality of facial motion cameras, and produce a digital representation reflecting combined facial and body motion of said at least one actor.

2. The system for capturing motion of claim 1, wherein said plurality of facial motion cameras and said plurality of body motion cameras are fixed.

3. The system for capturing motion of claim 1, wherein a portion of said plurality of facial motion cameras and a portion of said plurality of body motion cameras are movable to track motion within said motion capture volume.

4. The system for capturing motion of claim 1, wherein said plurality of facial motion cameras and said plurality of body motion cameras each further comprise polarized light sources oriented to illuminate said motion capture volume and polarized filter to block polarized light from other cameras disposed across said motion capture volume.

5. A method for capturing motion, comprising:

defining a motion capture volume adapted to contain at least one actor having body markers defining plural body points and facial markers defining plural facial points;

arranging a plurality of body motion cameras around a periphery of said motion capture volume;

arranging a plurality of facial motion cameras around said periphery of said motion capture volume, said facial motion cameras each having a respective field of view narrower than a corresponding field of view of said body motion cameras, said facial motion cameras being arranged such that all laterally exposed surfaces of said at least one actor while in motion within said motion capture volume are within said field of view of at least one of said plurality of facial motion cameras at substantially all times; and

simultaneously receiving motion capture data from said plurality of facial motion cameras and said plurality of body motion cameras and producing a digital representation reflecting combined body and facial motion of said at least one actor.

6. The method for capturing motion of claim 5, further comprising subdividing said motion capture volume into a plurality of quadrants, said plurality of quadrants each further comprising plural edges coincident with said periphery of said motion capture volume, and orienting first subgroups of said plurality of facial motion cameras toward respective ones of said edges.

7. The method for capturing motion of claim 6, further comprising orienting second subgroups of said plurality of facial motion cameras toward respective ones of said edges and spatially separated from said first subgroups of cameras.

8. The method for capturing motion of claim 7, further comprising orienting said first subgroups of cameras with respect to said second subgroups of cameras so that their respective fields of view substantially overlap.

9. The method for capturing motion of claim 7, wherein said first subgroups of cameras and said second subgroups of cameras each further comprise at least three cameras.

10. The method for capturing motion of claim 5, further comprising locating third subgroups of said plurality of facial motion cameras at respective corners of said motion capture volume and oriented generally toward a center of said motion capture volume.

11. The method for capturing motion of claim 5, further comprising disposing at least a first portion of said plurality of facial motion cameras at a first height above ground, and disposing at least a second portion of said plurality of facial motion cameras at a second height above ground greater than said first height.

12. The method for capturing motion of claim 11, further comprising orienting said first portion of said plurality of facial motion cameras slightly upward to not encompass within respective fields of view other ones of said first portion of said plurality of facial motion cameras disposed substantially across said motion capture volume.

13. The method for capturing motion of claim 5, further comprising fixing in place said plurality of facial motion cameras and said plurality of body motion cameras.

14. The method for capturing motion of claim 5, further comprising moving at least a portion of said plurality of facial motion cameras and at least a portion of said plurality of body motion cameras to track motion within said motion capture volume.

15. The method for capturing motion of claim 5, further comprising illuminating said motion capture volume with polarized light and providing at least a portion of said plurality of facial motion cameras and at least a portion of said plurality of body motion cameras with polarization filters to block said polarized light from other ones of said cameras disposed substantially across said motion capture volume.

16. The method for capturing motion of claim 5, further comprising recording audio from within said motion capture volume.