Apparatus and method for performing motion capture using shutter synchronization

• US 7,605,861 B2
• Filed: 03/10/2005
• Issued: 10/20/2009
• Est. Priority Date: 03/10/2005
• Status: Active Grant

• Current Assignee: Rearden Mova, LLC
• Original Assignee: OnLive Incorporated
• Inventors: LaSalle, Greg, Van der Laan, Roger, Cotter, Timothy S., Speck, John, Periman, Stephen G., Pearce, Kenneth A.
• Primary Examiner(s): Ometz; David L
• Assistant Examiner(s): Bemben; Richard M
• Application Number:
  US11/077,628
• Publication Number:
  US 20060203096A1
• Time in Patent Office: 1,685 Days
• Field of Search: 348169-172, 348/218.1, 348/208.14, 348/162, 348/207.99, 348/239, 348370-371
• US Class Current: 348/371
• CPC Class Codes: G06V 40/16 Human faces, e.g. facial parts, sketches or expressions
• CPC Class Codes: G06V 40/176 Dynamic expression
• CPC Class Codes: H04N 23/74 by influencing the scene brightness using illuminating means
• CPC Class Codes: H04N 23/75 by influencing optical camera components

Abstract

A method is described comprising: applying phosphorescent paint to specified regions of a performer'"'"'s face and/or body; strobing a light source on and off, the light source charging the phosphorescent paint when on; and strobing the shutters of a first plurality of cameras synchronously with the strobing of the light source to capture images of the phosphorescent paint, wherein the shutters are open when the light source is off and the shutters are closed when the light source is open.

Claims

1. A method comprising:

applying phosphorescent paint to regions of a performer'"'"'s face and/or body;

strobing a light source on and off, the light source charging the phosphorescent paint when on; and

strobing the shutters of a first plurality of cameras synchronously with the strobing of the light source to capture sequences of images of the phosphorescent paint (“

glow frames”

) as the performer moves or changes facial expressions during a performance, wherein the shutters are open when the light source is off and the shutters are closed when the light source is on.

2. The method as in claim 1 further comprising:

tracking the motion of the phosphorescent paint over time; and

generating motion data representing the movement of the performer'"'"'s face and/or body using the tracked movement of the phosphorescent paint.

3. The method as in claim 1 wherein the phosphorescent paint is applied as a series of curves on the performer'"'"'s face.

4. The method as in claim 1 wherein the phosphorescent paint is applied as a series of markers at specified areas of the performer'"'"'s body.

5. The method as in claim 1 further comprising:

strobing the shutters of a second plurality of cameras synchronously with the strobing of the light source to capture images of the performer (“

lit frames”

), wherein the shutters of the second plurality of cameras are open when the light source is on and the shutters of the second plurality of cameras are closed when the light source is off.

6. The method as in claim 5 wherein the first plurality of cameras are grayscale cameras and the second plurality of cameras are color cameras.

7. The method as in claim 5 further comprising:

separating the lit frames from the glow frames to generate two separate sets of image data.

8. The method as in claim 5 wherein cameras capturing the lit frames have a sensitivity which is different from cameras capturing the glow frames.

9. The method as in claim 5 further comprising:

opening the shutters for a first period of time when the light source is on; and

opening the shutters for a second period of time when the light source is off;

wherein the first and second periods of time are unequal.

10. The method as in claim 5 wherein color cameras are used to capture the lit frames and grayscale cameras are used to capture the glow frames.

11. The method as in claim 10 wherein the grayscale cameras have a relatively higher sensitivity than the color cameras.

12. The method as in claim 10 wherein two different synchronization signals are used to control the shutters of the color and grayscale cameras.

13. The method as in claim 12 wherein the different synchronization signals are 180 degrees out of phase.

14. The method as in claim 1 wherein the light source comprises a light emitting diode (LED) array.

15. The method as in claim 1 wherein strobing the shutters comprises opening the shutters for a first period of time and closing the shutters for a second period of time, the second period of time being of a different duration than the first period of time.

16. The method as in claim 15 wherein the first period of time is longer than the second period of time.

17. The method as in claim 1 wherein the camera shutters are controlled by synchronization signals from a computer system.

18. The method as in claim 1 wherein strobing the shutters further comprises:

opening the shutters for a period of time when the light source is on to capture images of the performer'"'"'s face and/or body.

19. The method as in claim 18 wherein after being opened to capture a lit frame, the shutters are closed and then opened again when the light source is off to capture the next glow frame, and then closed and then opened again when the light source is on to capture the next lit frame.

20. The method as in claim 18 wherein strobing the shutters comprises opening the shutters for a first period of time and closing the shutters for a second period of time wherein the first period of time is not equal to the second period of time.

21. The method as in claim 20 further comprising:

opening the shutters for a relatively shorter period of time when the light source is on; and

opening the shutters for a relatively longer period of time when the light source is off.

22. The method as in claim 18 further comprising:

separating the lit frames from the glow frames to generate two separate sets of image data.

23. The method as in claim 18 further comprising:

alternating sensitivity, of the cameras between capturing the lit frames and the glow frames.

24. A system comprising:

a synchronization signal generator to generate one or more synchronization signals;

a light source configured to strobe on and off responsive to a first one of the one or more synchronization signals, the light source charging phosphorescent paint applied to regions of a performer'"'"'s face and/or body for a motion capture session; and

a plurality of cameras having shutters strobed synchronously with the strobing of the light source to capture sequences of images of the phosphorescent paint (“

glow frames”

) as the performer moves or changes facial expressions during a performance, wherein the shutters are open when the light source is off and the shutters are closed when the light source is on.

25. The system as in claim 24 further comprising:

an image processing device generating motion data representing the movement of the performer'"'"'s face and/or body using the tracked movement of the phosphorescent paint.

26. The system as in claim 24 wherein the phosphorescent paint is applied as a series of curves on the performer'"'"'s face.

27. The system as in claim 24 wherein the phosphorescent paint is applied as a series of markers at specified areas of the performer'"'"'s body.

28. The system as in claim 24 further comprising:

a second plurality of cameras having shutters strobed synchronously with the strobing of the light source to capture images of the performer (“

lit frames”

), wherein the shutters of the second plurality of cameras are open when the light source is on and the shutters of the second plurality of cameras are closed when the light source is off.

29. The system as in claim 28 further comprising an image processing device separating the lit frames from the glow frames to generate two separate sets of image data.

30. The system as in claim 28 wherein cameras capturing the lit frames have a sensitivity which is different from cameras capturing the glow frames.

31. The system as in claim 28 wherein at least some of the plurality of cameras are controlled to open the shutters for a relatively shorter period of time when the light source is on;

and open the shutters for a relatively longer period of time when the light source is off.

32. The system as in claim 28 wherein color cameras are used to capture the lit frames and grayscale cameras are used to capture the glow frames.

33. The system as in claim 28 wherein the first plurality of cameras are grayscale cameras and the second plurality of cameras are color cameras.

34. The system as in claim 33 wherein the grayscale cameras have a relatively higher sensitivity than the color cameras.

35. The system as in claim 33 wherein two different synchronization signals are used to control the shutters of the color and grayscale cameras.

36. The system as in claim 35 wherein the different synchronization signals are 180 degrees out of phase.

37. The system as in claim 24 wherein the light source comprises a light emitting diode (LED) array comprising at least one LED.

38. The system as in claim 24 wherein strobing the shutters comprises opening the shutters for a first period of time and closing the shutters for a second period of time, the second period of time being of a different duration than the first period of time.

39. The system as in claim 38 wherein the first period of time is longer than the second period of time.

40. The system as in claim 24 wherein the camera shutters are controlled by synchronization signals from a computer system.

41. The system as in claim 24 wherein strobing the shutters further comprises:

opening the shutters for a period of time when the light source is on to capture images of the performer'"'"'s face and/or body.

42. The system as in claim 41 wherein after being opened to capture a lit frame, the shutters are closed and then opened again when the light source is off to capture the next glow frame, and then closed and then opened again when the light source is on to capture the next lit frame.

43. The system as in claim 42 wherein the image processing device separates the lit frames from the glow frames to generate two separate sets of image data.

44. The system as in claim 41 wherein strobing the shutters comprises opening the shutters for a first period of time and closing the shutters for a second period of time, wherein the first period of time is not equal to the second period of time.

45. The system as in claim 44 wherein the shutters are opened for a relatively shorter period of time when the light source is on;

and

wherein the shutters are opened for a relatively longer period of time when the light source is off.

46. The system as in claim 41 wherein sensitivity of the cameras is alternated between capturing the lit frames and the glow frames.

47. A method comprising:

applying phosphorescent paint to regions of a face and/or body or a performer;

strobing a light source on and off, the light source charging the phosphorescent paint when on; and

strobing the shutters of a plurality of cameras synchronously with the strobing of the light source to capture a sequence of images of the phosphorescent paint (“

glow frames) and images of the object in motion (“

lit frames”

), wherein the shutters are closed and then opened when the light source is off to capture the glow frames and then closed and then opened when the light source is on to capture the lit frames.

48. The method as in claim 47 further comprising:

tracking the motion of the phosphorescent paint over time; and

generating motion data representing the movement of the object using the tracked movement of the phosphorescent paint.

49. The method as in claim 47 wherein strobing the shutters comprises opening the shutters for a first period of time and closing the shutters for a second period of time wherein the first period of time is not equal to the second period of time.

50. The method as in claim 49 further comprising:

opening the shutters for a relatively shorter period of time when the light source is on; and

opening the shutters for a relatively longer period of time when the light source is off.

51. The method as in claim 47 further comprising:

separating the lit frames from the glow frames to generate two separate sets of image data.

52. The method as in claim 47 further comprising:

alternating sensitivity of the cameras between capturing the lit frames and the glow frames.