System and method for performing motion capture and image reconstruction with transparent makeup

• US 20100231692A1
• Filed: 06/05/2009
• Published: 09/16/2010
• Est. Priority Date: 07/31/2006
• Status: Abandoned Application

• Current Assignee: Rearden Mova, LLC
• Original Assignee: OnLive Incorporated
• Inventors: LaSalle, Greg, Fontaine, Robin, Perlman, Stephen G.
• Application Number:
  US12/455,771
• Publication Number:
  US 20100231692A1
• Time in Patent Office: Days
• US Class Current: 348/48
• CPC Class Codes: G06F 3/005 Input arrangements through a video camera
• CPC Class Codes: G06T 7/20 Analysis of motion motion estimation for coding, decoding, compressing or decompressing digital video signals H04N19/43, H04N19/51
• CPC Class Codes: G06T 7/596 from three or more stereo images
• CPC Class Codes: H04N 5/2228 Video assist systems used in motion picture production, e.g. video cameras connected to viewfinders of motion picture cameras or related video signal processing

Abstract

A system and method are described for performing motion capture on a subject using transparent makeup, paint, dye or ink that is visible to certain cameras, but invisible to other cameras. For example, a system according to one embodiment of the invention comprises the application of makeup, paint, dye or ink on a subject in a random pattern that contains a phosphor that is transparent in the visible light spectrum, but is emissive in a non-visible spectrum such as the infrared (IR) or ultraviolet (UV) spectrum; using visible light such as ambient light or daylight to illuminate the subject; using a first plurality of cameras sensitive in the visible light spectrum to capture the normal coloration of the subject; and using a second plurality of cameras sensitive in a non-visible spectrum to capture the random pattern.

Claims

1. A method for performing motion capture, comprising:

applying a random pattern of makeup, paint, dye or ink which is transparent in visible light to specified regions of an object or a performer'"'"'s face, body and/or clothing;

applying electromagnetic radiation (EMR) to the makeup, paint, dye or ink thereby causing the makeup paint, dye or ink to emit EMR within a spectrum capable of being captured by a first plurality of cameras; and

capturing sequences of images of the random pattern with the first plurality of cameras as the object or performer moves and/or changes facial expressions during a motion capture session.

2. The method as in claim 1 further comprising:

correlating the random pattern across two or more images captured from two or more different cameras to create a 3-dimensional surface of the specified regions of the object or performer'"'"'s face, body, and/or clothing; and

generating motion data representing the movement of the 3-dimensional surface across the sequence of images using the tracked movement of the random pattern.

3. The method as in claim 1 wherein the makeup, paint, dye or ink comprises transparent ultraviolet (UV) makeup, paint, dye or ink and the spectrum capable of being captured by a first plurality of cameras is the ultraviolet spectrum.

4. The method as in claim 1 wherein applying EMR further comprises strobing an EMR source on and off, the EMR source charging the makeup, paint, dye or ink when on;

and

wherein capturing sequences of images further comprises strobing the shutters of the first plurality of cameras synchronously with the strobing of the EMR source to capture the sequences of images of the random pattern (“

glow frames”

) as the performer moves or changes facial expressions during a performance, wherein the shutters of the first plurality of cameras are open when the light source is on.

5. The method as in claim 4 further comprising:

strobing the shutters of a second plurality of cameras synchronously with the strobing of the light source to capture images of the performer in visible light (“

lit frames”

).

6. The method as in claim 5 wherein the EMR source is off when the shutters of the second plurality of cameras are open.

7. The method as in claim 5 further comprising strobing a visible light source on and off synchronously with the strobing of the shutters of the second plurality of cameras, wherein the shutters of the second plurality of cameras are open when the visible light source is on.

8. The method as in claim 7 wherein the first plurality of cameras are grayscale cameras and the second plurality of cameras are color cameras.

9. The method as in claim 5 wherein the first plurality of cameras are grayscale cameras and the second plurality of cameras are color cameras.

10. A method comprising:

applying a random pattern of makeup, paint, dye or ink which is transparent in visible light to specified regions of an object or a performer'"'"'s face, body and/or clothing;

strobing an electromagnetic radiation (EMR) source on and off, the EMR source causing the makeup, paint, dye or ink to emit EMR within a spectrum capable of being captured by a first plurality of cameras; and

strobing the shutters of the first plurality of cameras synchronously with the strobing of the light source to capture sequences of images of the random pattern (“

glow frames”

) as the performer moves or changes facial expressions during a performance, wherein the shutters of the first plurality of cameras are open when the light source is on and the shutters are closed when the light source is off.

11. The method as in claim 10 wherein the makeup is a phosphor makeup.

12. The method as in claim 10 further comprising:

tracking the motion of the random pattern over time; and

generating motion data representing the movement of the performer'"'"'s face and/or body using the tracked movement of the random pattern.

13. The method as in claim 10 further comprising:

strobing the shutters of a second plurality of cameras synchronously with the strobing of the EMR source to capture images of the performer in visible light (“

lit frames”

), wherein the shutters of the second plurality of cameras are open when the EMR source is off.

14. The method as in claim 13 wherein the first plurality of cameras are grayscale cameras and the second plurality of cameras are color cameras.

15. The method as in claim 13 further comprising strobing a visible light source on and off synchronously with the strobing of the shutters of the second plurality of cameras, wherein the shutters of the second plurality of cameras are open when the visible light source is on.

16. The method as in claim 15 wherein the camera shutters, EMF source and light source are controlled by synchronization signals from a computer system.

17. The method as in claim 13 further comprising:

separating the lit frames from the glow frames to generate two separate sets of image data.

18. The method as in claim 13 wherein the second plurality of cameras have a sensitivity which is different from the sensitivity of the first plurality of cameras.