Interactive systems and methods for video compositing

• US 8,824,861 B2
• Filed: 06/30/2009
• Issued: 09/02/2014
• Est. Priority Date: 07/01/2008
• Status: Active Grant

• Current Assignee: NOO INC.
• Original Assignee: YOOSTAR ENTERTAINMENT GROUP, INC
• Inventors: Gentile, Anthony, Gentile, John, Wilker, Scott, Waldrep, Mark, Veenstra, Matthew
• Primary Examiner(s): CHIO, TAT CHI
• Application Number:
  US12/495,517
• Publication Number:
  US 20100027961A1
• Time in Patent Office: 1,890 Days
• Field of Search: 386/210, 386/248, 386/287, 386/239, 386/240, 386/278, 386/280, 386/282, 386/285, 386/286, 348/592
• US Class Current: 386/278
• CPC Class Codes: G06F 3/0484 for the control of specific functions or operations, e.g. selecting or manipulating an object, an image or a displayed text element, setting a parameter value or selecting a range
• CPC Class Codes: G06T 5/50 using two or more images, e.g. averaging or subtraction
• CPC Class Codes: G11B 27/034 on discs G11B27/036, G11B27/038 take precedence
• CPC Class Codes: G11B 27/322 used signal is digitally coded
• CPC Class Codes: G11B 27/34 Indicating arrangements indicating means incorporated in magazine or cassette G11B23/046 and G11B23/0875; indicating measured values in general G01D
• CPC Class Codes: H04N 5/262 Studio circuits, e.g. for mixing, switching-over, change of character of image, other special effects ; Cameras specially adapted for the electronic generation of special effects
• CPC Class Codes: H04N 5/272 Means for inserting a foreground image in a background image, i.e. inlay, outlay
• CPC Class Codes: H04N 5/91 Television signal processing therefor

Abstract

Certain interactive role performance systems and methods are disclosed that allow users to select a role to play in a prerecorded video, such as a movie scene, and to replace an original actor of that role with the user'"'"'s performance. In certain examples, the user selects a prerecorded scene from which a character has been removed and records his or her performance. The interactive role performance system can insert the user'"'"'s performance in place of the original character, thereby creating the appearance that the user is interacting, such as in real-time, with the other characters in the movie scene. Moreover, systems and methods can further utilize background subtraction techniques to isolate the user image. In further examples, the recorded user performance is stored separately from the prerecorded video.

Claims

1. A method for combining video signals in an interactive video system, the method comprising:

receiving a first video signal associated with pre-washed video content, the pre-washed video content having been created from prerecorded video content having multiple frames by digitally removing an original character therefrom in selected ones of the multiple frames;

receiving metadata associated with the first video signal, the metadata being indicative of the selected frames;

receiving a second video signal in real-time comprising second video content having a background image and a video image of a participant playing a role of the original character in front of the background image;

filtering the second video signal in real-time to remove the background image to create a modified second video signal with the video image of the participant; and

overlaying in real-time the first video signal and, as determined by the metadata, the modified second video signal corresponding to the selected frames to create a combined video signal, whereby the video image of the participant in the selected frames appears in place of the original character of the prerecorded video content.

2. The method of claim 1, additionally comprising displaying the combined video signal in real time with respect to said receiving the second video signal.

3. The method of claim 1, wherein said filtering comprises:

receiving a third signal consisting essentially of the background image;

comparing the second video signal with the third signal to identify matching portions of the second video signal and the third signal; and

modifying the second video signal by designating the matching portions within the second video signal to be transparent.

4. The method of claim 1, wherein the prerecorded video content comprises a portion of at least one of a movie, a television show, and a commercial.

5. The method of claim 1, wherein the prerecorded video content comprises a still image.

6. The method of claim 1, wherein said receiving the first video signal comprises receiving the first video signal via a network.

7. The method of claim 1, additionally comprising displaying a script to the participant during said receiving the second video signal, wherein the script corresponds to lines of the original character.

8. The method of claim 1, additionally comprising storing the modified second video signal separate from the first video signal for later overlaying and displaying.

9. The method of claim 8, wherein said storing comprises uploading the modified second video signal onto a network.

10. The method of claim 1, wherein in the pre-washed video content, a second original character has been digitally removed from the prerecorded video content in a second selected ones of the multiple frames, additionally comprising:

receiving second metadata indicative of the second selected frames;

receiving a third video signal comprising a second video image of a second participant; and

overlaying the first video signal and, as determined by the second metadata, the third video signal corresponding to the second selected frames, whereby the second video image of the second participant in the second selected frames appears in place of the second original character of the prerecorded video content.

11. The method of claim 1, wherein said overlaying further comprises placing one or more elements of the prerecorded video content delineated by a matte layer in a foreground of the combined video content to overlay at least a portion of the video image of the participant.

12. The method of claim 1, wherein in the selected frames of the pre-washed video content, the original character has been digitally replaced with the background imagery surrounding the original character in the prerecorded video content.

13. The method of claim 1,

wherein the participant steps away from a video camera to enable the video camera to record a reference background image, and the participant then steps back in for generating the second video signal.

14. The method of claim 13,

wherein, prior to the participant stepping away from the video camera, the video camera records a reference image of the participant in front of the background image.

15. The method of claim 13,

wherein the filtering step includes setting each pixel of the background image as transparent or opaque, based on a color channel difference between the second video signal and the reference background image for the pixel.

16. The method of claim 1,

wherein the filtering step includes setting each pixel of the background image as transparent or opaque, based on a color channel difference between the second video signal and a reference background image for the pixel.

17. The method of claim 16, wherein the participant steps away from a video camera to permit the video camera to record the reference background image.

18. A system for interactive video entertainment, the system comprising:

a first database comprising, pre-washed video content having been created from prerecorded video content having multiple frames by digitally removing an original character therefrom in selected ones of the multiple frames, and metadata associated with the pre-washed video content, the metadata identifying the selected frames;

a video input configured to receive a real-time video signal comprising a video image of a participant and a background image; and

a processing module configured to execute on a computing device, the processing module configured to, filter in real-time the real-time video signal to remove the background image to create a modified real-time video signal, and overlay in real-time, based at least on the metadata, a first video signal created from the pre-washed video content and the modified real-time video signal corresponding to the selected frames to create combined video signal, whereby the video image of the participant in the selected frames appears in place of the original character of the prerecorded video content.

19. The system of claim 18, further comprising a video recorder configured to generate the real-time video signal and to transmit the real-time video signal to the video input.

20. The system of claim 19, wherein the video recorder comprises a webcam.

21. The system of claim 18, wherein the processing module is further configured to cause a display of the combined video signal.

22. The system of claim 18, wherein the processing module is further configured to cause a display of a character outline at a location where the original character was digitally removed from the prerecorded video content.

23. The system of claim 18, further comprising a second database configured to store second video content based only on the modified real-time video signal.

24. The system of claim 18, wherein the processing module comprises a graphics processing unit (GPU).

25. The system of claim 18, wherein in the selected frames of the pre-washed video content, the original character has been digitally replaced with the background imagery surrounding the original character in the prerecorded video content.

26. A system for combining video signals, the system comprising:

a first receiver for receiving a first video signal associated with pre-washed video content, the pre-washed video content having been created from prerecorded video content having multiple frames by digitally removing an original character therefrom in selected ones of the multiple frames, and for receiving metadata associated with the first video signal, the metadata being indicative of the selected frames;

a second receiver for receiving a second video signal in real-time comprising a video image of a participant and a background image;

a filter for filtering the second video signal in real-time to remove the background image and to create a modified second video signal; and

a video compositor for overlaying in real-time, based at least on said metadata, the first video signal and the modified second video signal corresponding to the selected frames to create a combined video signal, whereby the video image of the participant in the selected frames appears in place of the original character of the prerecorded video content.

27. The system of claim 26, further comprising a display for displaying in real time the combined video signal.

28. The system of claim 26, wherein in the selected frames of the pre-washed video content, the original character has been digitally replaced with the background imagery surrounding the original character in the prerecorded video content.

29. The system of claim 26,

wherein the participant steps away from a video camera to permit the video camera to record a reference background image, and then the participant steps back in for generating the second video signal.

30. The system of claim 29, wherein, prior to the participant stepping away from the video camera, the video camera records a reference image of the participant in front of the background image.

31. The system of claim 29,

wherein the filtering includes setting each pixel of the background image as transparent or opaque, based on a color channel difference between the second video signal and the reference background image for the pixel.

32. The system of claim 26,

wherein the filtering includes setting each pixel of the background image as transparent or opaque, based on a color channel difference between the second video signal and a reference background image for the pixel.

33. The system of claim 32, wherein the participant steps away from a video camera to permit the video camera to record the reference background image.

34. The system of claim 26, wherein said overlaying further comprises placing one or more elements of the prerecorded video content delineated by a matte layer in a foreground of the combined video content to overlay at least a portion of the video image of the participant.