System and method for capturing facial and body motion

• US 20040179008A1
• Filed: 05/01/2003
• Published: 09/16/2004
• Est. Priority Date: 03/13/2003
• Status: Active Grant

• Current Assignee: Sony Corporation (Sony Group Corp.), Sony Pictures Entertainment Incorporated (Sony Group Corp.)
• Original Assignee: Sony Corporation (Sony Group Corp.), Sony Pictures Entertainment Incorporated (Sony Group Corp.)
• Inventors: Gordon, Demian, Chen, Jerome, Hastings, Albert Robert, Echegaray, Jody
• Granted Patent: US 7,218,320 B2
• Time in Patent Office: Days
• US Class Current: 345/419
• CPC Class Codes: G06F 3/011 Arrangements for interaction with the human body, e.g. for user immersion in virtual reality blind teaching G09B21/00
• CPC Class Codes: G06T 2207/10016 Video; Image sequence
• CPC Class Codes: G06T 2207/30201 Face
• CPC Class Codes: G06T 2207/30204 Marker
• CPC Class Codes: G06T 7/246 using feature-based methods, e.g. the tracking of corners or segments
• CPC Class Codes: G06V 10/10 Image acquisition document image scanning and transmission H04N1/00; control of digital cameras H04N23/60

Abstract

A system and method for capturing motion comprises a motion capture volume adapted to contain at least one actor having body markers defining plural body points and facial markers defining plural facial points. A plurality of body motion cameras and a plurality of facial motion cameras are arranged around a periphery of the motion capture volume. The facial motion cameras each have a respective field of view narrower than a corresponding field of view of the body motion cameras. The facial motion cameras are arranged such that all laterally exposed surfaces of the actor while in motion within the motion capture volume are within the field of view of at least one of the plurality of facial motion cameras at substantially all times. A motion capture processor is coupled to the plurality of facial motion cameras and the plurality of body motion cameras to produce a digital model reflecting combined body and facial motion of the actor. At least one microphone may be oriented to pick up audio from the motion capture volume.

Claims

1. A system for capturing motion, comprising:

a motion capture volume adapted to contain at least one actor having body markers defining plural body points and facial markers defining plural facial points;

a plurality of facial motion cameras arranged around said periphery of said motion capture volume, said facial motion cameras being arranged such that all laterally exposed surfaces of said at least one actor while in motion within said motion capture volume are within a field of view of at least one of said plurality of facial motion cameras at substantially all times; and

a motion capture processor coupled to said plurality of facial motion cameras to produce a digital representation reflecting facial motion of said at least one actor.

2. The system for capturing motion of claim 1, further comprising a plurality of body motion cameras arranged around a periphery of said motion capture volume, wherein said facial motion cameras each have a respective field of view narrower than a corresponding field of view of said body motion cameras, and wherein said motion capture processor is further coupled to said plurality of body motion cameras and said digital representation further reflects body motion of said at least one actor.

3. The system for capturing motion of claim 1, wherein said motion capture volume further comprises a rectangular area subdivided into a plurality of quadrants, said plurality of quadrants each further comprising plural edges coincident with said periphery of said motion capture volume, said plurality of facial motion cameras including first subgroups of cameras oriented toward respective ones of said edges.

4. The system for capturing motion of claim 3, wherein said plurality of facial motion cameras include second subgroups of cameras oriented toward respective ones of said edges and spatially separated from said first subgroups of cameras.

5. The system for capturing motion of claim 4, wherein said first subgroups of cameras are oriented with respect to said second subgroups of cameras so that their respective fields of view substantially overlap.

6. The system for capturing motion of claim 3, wherein said first subgroups of cameras and said second subgroups of cameras each further comprise at least three cameras.

7. The system for capturing motion of claim 3, wherein said plurality of facial motion cameras further comprise third subgroups of cameras disposed respectively at corners of said motion capture volume and oriented generally toward a center of said motion capture volume.

8. The system for capturing motion of claim 1, wherein at least a first portion of said plurality of facial motion cameras are disposed at a first height above ground, and at least a second portion of said plurality of facial motion cameras are disposed at a second height above ground greater than said first height.

9. The system for capturing motion of claim 8, wherein said first portion of said plurality of facial motion cameras are oriented slightly upward to not encompass within respective fields of view other ones of said first portion of said plurality of facial motion cameras disposed substantially across said motion capture volume.

10. The system for capturing motion of claim 2, wherein said plurality of facial motion cameras and said plurality of body motion cameras are fixed.

11. The system for capturing motion of claim 2, wherein a portion of said plurality of facial motion cameras and a portion of said plurality of body motion cameras are movable to track motion within said motion capture volume.

12. The system for capturing motion of claim 1, wherein said motion capture volume further comprises an area of approximately one hundred square feet.

13. The system for capturing motion of claim 2, wherein said plurality of facial motion cameras and said plurality of body motion cameras each further comprise polarized light sources oriented to illuminate said motion capture volume and polarized filter to block polarized light from other cameras disposed across said motion capture volume.

14. The system for capturing motion of claim 1, further comprising at least one microphone oriented to pick up audio from said motion capture volume.

15. A system for capturing combined facial and body motion, comprising:

a motion capture volume adapted to contain at least one actor having body markers defining plural body points and facial markers defining plural facial points;

a plurality of body motion cameras arranged around a periphery of said motion capture volume;

a plurality of facial motion cameras arranged around said periphery of said motion capture volume, said facial motion cameras each having a respective field of view narrower than a corresponding field of view of said body motion cameras;

said facial motion cameras being arranged such that all laterally exposed surfaces of said at least one actor while in motion within said motion capture volume are within said field of view of at least one of said plurality of facial motion cameras at substantially all times; and

a motion capture processor coupled to said plurality of body motion cameras and said plurality of facial motion cameras to produce a digital representation reflecting combined facial and body motion of said at least one actor.

16. The system for capturing motion of claim 15, wherein said plurality of facial motion cameras and said plurality of body motion cameras are fixed.

17. The system for capturing motion of claim 15, wherein a portion of said plurality of facial motion cameras and a portion of said plurality of body motion cameras are movable to track motion within said motion capture volume.

18. The system for capturing motion of claim 15, wherein said plurality of facial motion cameras and said plurality of body motion cameras each further comprise polarized light sources oriented to illuminate said motion capture volume and polarized filter to block polarized light from other cameras disposed across said motion capture volume.

19. A method for capturing motion, comprising:

defining a motion capture volume adapted to contain at least one actor having body markers defining plural body points and facial markers defining plural facial points;

arranging a plurality of body motion cameras around a periphery of said motion capture volume;

arranging a plurality of facial motion cameras around said periphery of said motion capture volume, said facial motion cameras each having a respective field of view narrower than a corresponding field of view of said body motion cameras, said facial motion cameras being arranged such that all laterally exposed surfaces of said at least one actor while in motion within said motion capture volume are within said field of view of at least one of said plurality of facial motion cameras at substantially all times; and

processing data from said plurality of facial motion cameras and said plurality of body motion cameras to produce a digital representation reflecting combined body and facial motion of said at least one actor.

20. The method for capturing motion of claim 19, further comprising subdividing said motion capture volume into a plurality of quadrants, said plurality of quadrants each further comprising plural edges coincident with said periphery of said motion capture volume, and orienting first subgroups of said plurality of facial motion cameras toward respective ones of said edges.

21. The method for capturing motion of claim 20, further comprising orienting second subgroups of said plurality of facial motion cameras toward respective ones of said edges and spatially separated from said first subgroups of cameras.

22. The method for capturing motion of claim 21, further comprising orienting said first subgroups of cameras with respect to said second subgroups of cameras so that their respective fields of view substantially overlap.

23. The method for capturing motion of claim 21, wherein said first subgroups of cameras and said second subgroups of cameras each further comprise at least three cameras.

24. The method for capturing motion of claim 19, further comprising locating third subgroups of said plurality of facial motion cameras at respective corners of said motion capture volume and oriented generally toward a center of said motion capture volume.

25. The method for capturing motion of claim 19, further comprising disposing at least a first portion of said plurality of facial motion cameras at a first height above ground, and disposing at least a second portion of said plurality of facial motion cameras at a second height above ground greater than said first height.

26. The method for capturing motion of claim 25, further comprising orienting said first portion of said plurality of facial motion cameras slightly upward to not encompass within respective fields of view other ones of said first portion of said plurality of facial motion cameras disposed substantially across said motion capture volume.

27. The method for capturing motion of claim 19, further comprising fixing in place said plurality of facial motion cameras and said plurality of body motion cameras.

28. The method for capturing motion of claim 19, further comprising moving at least a portion of said plurality of facial motion cameras and at least a portion of said plurality of body motion cameras to track motion within said motion capture volume.

29. The method for capturing motion of claim 19, further comprising illuminating said motion capture volume with polarized light and providing at least a portion of said plurality of facial motion cameras and at least a portion of said plurality of body motion cameras with polarization filters to block said polarized light from other ones of said cameras disposed substantially across said motion capture volume.

30. The method for capturing motion of claim 19, further comprising recording audio from within said motion capture volume.