Method and system for immersing face images into a video sequence

• US 7,826,644 B2
• Filed: 04/12/2010
• Issued: 11/02/2010
• Est. Priority Date: 12/31/2002
• Status: Expired due to Fees

• Current Assignee: F Poszat Hu LLC (Intellectual Ventures LLC)
• Original Assignee: Vmine Image Tech Co Ltd LLC (Intellectual Ventures LLC)
• Inventors: Jung, Namsoon, Sharma, Rajeev
• Primary Examiner(s): Seth; Manav
• Application Number:
  US12/758,172
• Publication Number:
  US 20100195913A1
• Time in Patent Office: 204 Days
• Field of Search: 382/118, 382/190, 382/276, 382282-287, 358/537, 715704-707, 725 32- 35
• US Class Current: 382/118
• CPC Class Codes: G06V 20/46 Extracting features or characteristics from the video content, e.g. video fingerprints, representative shots or key frames
• CPC Class Codes: G06V 40/16 Human faces, e.g. facial parts, sketches or expressions
• CPC Class Codes: G11B 27/036 Insert-editing
• CPC Class Codes: G11B 27/28 by using information signals recorded by the same method as the main recording G11B27/22 takes precedence

Abstract

The present invention is a system and method for immersing facial images of people captured automatically from an image or a sequence of images into a live video playback sequence. This method allows viewers to perceive a participation in the viewed “movie” segment. A format is defined for storing the video such that this live playback of the video sequence is possible. A plurality of Computer Vision algorithms in the invention processes a plurality of input image sequences from the means for capturing images, which is pointed at the users in the vicinity of the system and performs the head detection and tracking. The interaction in the invention can be performed either in real-time or off-line depending on the embodiment of the invention in an uncontrolled background.

Claims

1. An apparatus, comprising:

a processor;

memory; and

software, stored in the memory that, in response to execution by the processor, causes the apparatus to perform operations comprising;

receiving a plurality of input images,

extracting one or more facial images from the plurality of input images,

generating demographic classifications using the one or more facial images,

replacing at least one replaceable actor image with at least one of the one or more facial images, based on the demographic classifications, and

creating a movie using the replaced at least one replaceable actor image.

2. The apparatus of claim 1, wherein the memory is further configured to store a first database;

wherein the operations further comprise storing the at least one replaceable actor image in the first database; and

wherein replacing the at least one replaceable actor image with at least one of the one or more facial images comprises matching the at least one replaceable actor image retrieved from the first database with at least one of the one or more facial images.

3. The apparatus of claim 1, wherein the memory is further configured to store a second database, and

wherein the operations further comprise storing the one or more facial images in the second database.

4. The apparatus of claim 3, wherein storing the one or more facial images in the second database comprises:

receiving an indication of acceptance of at least one facial image of the one or more facial images and

storing the accepted at least one facial image in the second database.

5. The method of claim 4, wherein replacing the at least one replaceable actor image with at least one of the one or more facial images comprises replacing the at least one replaceable actor image with the accepted at least one facial image.

6. The apparatus of claim 3, wherein receiving a plurality of input images comprises retrieving the one or more facial images from the second database.

7. The apparatus of claim 1, further comprising changing content in the movie based on a controllable graphical object.

8. The apparatus of claim 1, wherein extracting the one or more facial images comprises extracting the one or more facial images in real-time.

9. The apparatus of claim 1, wherein the movie is configured to be stored in a participative movie format comprising a participative movie format property;

wherein the operations further comprise receiving a selection to customize the participative movie format property; and

wherein creating a movie comprises creating the movie based on the customized participative movie format property.

10. The apparatus of claim 1, wherein the apparatus further comprises a displaying device, and wherein the operations further comprise playing the movie on the displaying device.

11. A non-transitory computer-readable storage medium, configured to store software that, in response to execution by a processor, causes the processor to perform operations comprising:

receiving a plurality of input images,

extracting one or more facial images from the plurality of input images,

generating demographic classifications using the one or more facial images,

replacing at least one replaceable actor image with at least one of the one or more facial images, based on the demographic classifications, and

creating a movie using the replaced at least one replaceable actor image.

12. The non-transitory computer-readable storage medium of claim 11, wherein the operations further comprise storing the at least one replaceable actor image in a first database;

and

wherein replacing the at least one replaceable actor image with at least one of the one or more facial images comprises matching the at least one replaceable actor image retrieved from the first database with at least one of the one or more facial images.

13. The non-transitory computer-readable storage medium of claim 11, wherein the operations further comprise storing the one or more facial images in a second database.

14. The non-transitory computer-readable storage medium of claim 13, wherein storing the one or more facial images in the second database comprises:

receiving an indication of acceptance of at least one facial image of the one or more facial images; and

storing the accepted at least one facial image in the second database.

15. The non-transitory computer-readable storage medium of claim 14, wherein replacing the at least one replaceable actor image with at least one of the one or more facial images comprises replacing the at least one replaceable actor image with the accepted at least one facial image.

16. The non-transitory computer-readable storage medium of claim 13, wherein receiving a plurality of input images comprises retrieving the one or more facial images from the second database.

17. The non-transitory computer-readable storage medium of claim 11, further comprising changing content in the movie based on a controllable graphical object.

18. The non-transitory computer-readable storage medium of claim 11, wherein extracting the one or more facial images comprises extracting the one or more facial images in real-time.

19. The non-transitory computer-readable storage medium of claim 11, wherein the movie is configured to be stored in a participative movie format comprising a participative movie format property;

wherein the operations further comprise receiving a selection to customize the participative movie format property; and

wherein creating a movie comprises creating the movie based on the customized participative movie format property.

20. The non-transitory computer-readable storage medium of claim 11, wherein the operations further comprise playing the movie.