Method and system for immersing face images into a video sequence

• US 7,734,070 B1
• Filed: 12/16/2003
• Issued: 06/08/2010
• Est. Priority Date: 12/31/2002
• Status: Active Grant

• Current Assignee: F Poszat Hu LLC (Intellectual Ventures LLC)
• Original Assignee: Vmine Image Tech Co Ltd LLC (Intellectual Ventures LLC)
• Inventors: Jung, Namsoon, Sharma, Rajeev
• Primary Examiner(s): Seth; Manav
• Application Number:
  US10/738,453
• Time in Patent Office: 2,366 Days
• Field of Search: 382/118, 382/190, 382/276, 382282-287, 358/537, 715704-707, 725 32- 35
• US Class Current: 382/118
• CPC Class Codes: G06V 20/46 Extracting features or characteristics from the video content, e.g. video fingerprints, representative shots or key frames
• CPC Class Codes: G06V 40/16 Human faces, e.g. facial parts, sketches or expressions
• CPC Class Codes: G11B 27/036 Insert-editing
• CPC Class Codes: G11B 27/28 by using information signals recorded by the same method as the main recording G11B27/22 takes precedence

Abstract

The present invention is a system and method for immersing facial images of people captured automatically from an image or a sequence of images into a live video playback sequence. This method allows viewers to perceive a participation in the viewed “movie” segment. A format is defined for storing the video such that this live playback of the video sequence is possible. A plurality of Computer Vision algorithms in the invention processes a plurality of input image sequences from the means for capturing images, which is pointed at the users in the vicinity of the system and performs the head detection and tracking. The interaction in the invention can be performed either in real-time or off-line depending on the embodiment of the invention in an uncontrolled background.

Claims

1. A method, comprising:

receiving a plurality of input images;

extracting one or more facial images from the plurality of input images using a computer system;

generating, at the computer system, demographic classifications using the plurality of input images;

replacing at least one replaceable actor image in a video sequence with at least one of the plurality of input images based on the demographic classifications; and

creating a movie based on the replaced at least one replaceable actor image.

2. The method according to claim 1, further comprising receiving the one or more facial images automatically from one or more images.

3. The method according to claim 1, wherein receiving the plurality of input images comprises receiving at least one image from a scanned photograph.

4. The method according to claim 1, further comprising storing the at least one replaceable actor image using a participative movie format.

5. The method according to claim 1, wherein generating the demographic classifications using the plurality of input images comprises generating demographic classifications using an extracted facial image of the one or more extracted facial images.

6. The method according to claim 5, further comprising matching the one or more facial images with the at least one replaceable actor image, according to the demographic classifications.

7. The method according to claim 1, further comprising playing the movie in real-time.

8. The method according to claim 1, further comprising changing content in the movie based on a controllable graphical object.

9. The method according to claim 8, further comprising synchronizing dynamic content adjustment to give seamless flow of playing the movie.

10. The method according to claim 1, further comprising:

storing the one or more facial images in a database; and

retrieving a facial image of the stored one or more facial images from the database for use as an input image.

11. The method according to claim 1, further comprising retrieving the at least one of the replaceable actor image from a database.

12. The method according to claim 1, further comprising printing one or more screen shots of the created movie.

13. The method according to claim 1, further comprising:

showing visual information and instruction about creating the movie.

14. The method according to claim 1, wherein extracting the one or more facial images comprises extracting the one or more facial images in real-time.

15. The method according to claim 1, wherein extracting the one or more facial images comprises extracting the one or more facial images against an uncontrolled background.

16. An apparatus, comprising:

means for capturing a plurality of input images;

means for extracting one or more facial images from the plurality of input images;

means for generating demographic classifications using the plurality of input images;

means for replacing at least one replaceable actor image in a video sequence with at least one of the plurality of input images based on the demographic classifications; and

means for creating a movie based on the replaced at least one replaceable actor image.

17. The apparatus according to claim 16, further comprising means for receiving the one or more facial images automatically from one or more images.

18. The apparatus according to claim 16, further comprising means for receiving at least one image of the plurality of input images from a scanned photograph.

19. The apparatus according to claim 16, further comprising means for storing the at least one replaceable actor image using a participative movie format.

20. The apparatus according to claim 19, wherein the means for creating the movie comprises means for creating the movie using the participative movie format and

wherein the apparatus further comprises means for playing the movie created using the participative movie format.

21. The apparatus according to claim 16, wherein the means for generating the demographic classifications using the plurality of input images comprises means for generating demographic classifications using an extracted facial image of the one or more extracted facial images.

22. The apparatus according to claim 16, further comprising means for matching the one or more facial images with the at least one replaceable actor image, according to the demographic classifications.

23. The apparatus according to claim 16, further comprising means for playing the movie in real-time.

24. The apparatus according to claim 16, further comprising means for changing content in the movie based on a controllable graphical object.

25. The apparatus according to claim 24, further comprising means for enabling dynamic content adjustment to be synchronized to give seamless flow of playing the movie.

26. The apparatus according to claim 16, further comprising:

means for storing the one or more facial images in a database; and

means for retrieving a facial image of the stored one or more facial images form the database for use as an input image.

27. The apparatus according to claim 16, further comprising means for retrieving the at least one replaceable actor image from a database.

28. The apparatus according to claim 16, further comprising means for printing one or more screen shots of the created movie.

29. The apparatus according to claim 16, further comprising means for showing visual information and instruction.

30. The apparatus according to claim 16, wherein the means for extracting the one or more facial images comprises means for extracting the one or more facial images in real-time.

31. The apparatus according to claim 16, wherein the means for extracting the one or more facial images comprises means for extracting the one or more facial images against an uncontrolled background.