Video object clipping method and apparatus

• US 20030184562A1
• Filed: 03/20/2003
• Published: 10/02/2003
• Est. Priority Date: 03/29/2002
• Status: Active Grant

• Current Assignee: Kabushiki Kaisha Toshiba (Toshiba Corporation)
• Original Assignee: Kabushiki Kaisha Toshiba (Toshiba Corporation)
• Inventors: Ida, Takashi, Matsumoto, Nobuyuki, Takeshima, Hidenori, Hori, Osamu
• Granted Patent: US 6,999,103 B2
• Time in Patent Office: Days
• US Class Current: 345/620
• CPC Class Codes: H04N 5/272 Means for inserting a foreground image in a background image, i.e. inlay, outlay
• CPC Class Codes: H04N 5/275 Generation of keying signals

Abstract

A video object clipping method includes storing, in a storage unit, original images each including a video object to be clipped and reference alpha images representing objects prepared, determining a criteria original image and a criteria reference alpha image from the original images and the reference alpha images, determining a deformation parameter by deforming the criteria reference alpha image to correspond to the criteria original image, and deforming remaining ones of the reference alpha images according to the determined deformation parameter to generate output alpha images corresponding to the original images.

Claims

1. A video object clipping method comprising:

storing, in a storage unit, a plurality of original images each including a video object to be clipped and a plurality of reference alpha images representing a plurality of objects prepared;

determining a criteria original image and a criteria reference alpha image from the original images and the reference alpha images;

determining a deformation parameter by deforming the criteria reference alpha image to correspond to the criteria original image; and

deforming remaining ones of the reference alpha images according to the determined deformation parameter to generate output alpha images corresponding to the original images.

2. The method according to claim 1, wherein the plurality of original images comprise a string of sequential images including the same video object, and the plurality of reference alpha images comprise a string of reference alpha images representing the same object, and wherein

determining the deformation parameter includes determining the deformation parameter based on matching of a criteria sequential image corresponding to the criteria original image with the criteria reference alpha image.

3. The method according to claim 2, which includes generating a string of the reference alpha images at a frame rate in a time axis, determining a set of the criteria sequential image and the criteria reference alpha image, performing a phase matching by equalizing a frame rate between the string of the sequential images and the string of the reference alphas images, the frame rate being equalized by downsampling the string of the reference alpha images using the determined criteria alpha image, and generating the output alpha images using the string of reference alpha images having the same frame rate as that of the string of the sequential images.

4. The method according to claim 2, wherein the string of the reference alpha images has a higher frame rate than the string of the sequential images, and which includes generating a plurality of temporary reference alpha image strings with a plurality of phases by downsampling the string of the reference alpha images using a plurality of temporary criteria alpha images, generating a plurality of temporary output alpha image strings based on the temporary reference alpha image strings, and selecting one of the temporary output alpha images as a final output alpha image string.

5. The method according to claim 2, which includes attaching a first marking to each of the reference alpha images, attaching a second marking to the video object of the criteria sequential image, and performing a phase matching by finding the criteria reference alpha image and the criteria sequential image which are nearest in time to each other referring to the first marking and the second marking.

6. The method according to claim 2, which includes storing a background image of the criteria sequential image in the storage unit, detecting a position of the object by obtaining a difference between the background image and the criteria sequential image, and performing a phase matching by finding one of the criteria reference alpha images which is nearest in time to the criteria sequential image based on a detection result.

7. The method according to claim 2, which includes generating a first differential image between the criteria sequential image and a past image of the sequential images, generating a second differential image between the criteria sequential image and a future image of the sequential images, detecting a position of the object based on a logical multiplication image obtained by subjecting pixel values of the first differential image and the second differential image to a logical multiplication, and performing a phase matching by finding one of the reference alpha images which is nearest in time to the criteria sequential image based on a detection result.

8. The method according to claim 1, wherein the deformation parameter is a parameter representing at least one of a parallel movement of the video object, a scaling of the video object and a rotation of the video object.

9. The method according to claim 1, wherein determining the criteria original image and the criteria reference alpha image includes determining one of the plurality of reference alpha images as the criteria reference alpha image by displaying selectively the plurality of reference alpha images.

10. The method according to claim 1, wherein determining the criteria original image and the criteria reference alpha image includes determining one of the plurality of original images as the criteria original image by displaying selectively the plurality of original images.

11. The method according to claim 1, which includes preparing the reference alpha images with respect to the plurality of objects individually, and summing the objects to generate a composite image.

12. The method according to claim 8, which includes generating a string of composite output alpha images by summing the objects of the output alpha images.

13. The method according to claim 1, which includes correcting a position of contour of each of the objects based on the output alpha images.

14. A video object clipping apparatus comprising:

a storage unit configured to store a plurality of original images each including a video object to be clipped and a plurality of reference alpha images representing a plurality of objects prepared;

a critical image determination unit configured to determine a criteria original image and a criteria reference alpha image from the original images and the reference alpha images;

a deformation parameter determination unit configured to determine a deformation parameter by deforming the criteria reference alpha image to correspond to the criteria original image; and

a deform unit configured to deform remaining ones of the reference alpha images according to the determined deformation parameter to generate output alpha images corresponding to the original images.

15. The apparatus according to claim 14, wherein the plurality of original images comprise a string of sequential images including the same video object, the plurality of reference alpha images comprise a string of reference alpha images representing the same object, the deformation parameter determination unit comprises a determination unit configured to determine the deformation parameter based on matching of a criteria sequential image corresponding to the criteria original image and the criteria reference alpha image.

16. The apparatus according to claim 15, which includes means for generating a string of the reference alpha images at a frame rate in a time axis, means for determining a set of the criteria sequential image and the criteria reference alpha image, means for performing a phase matching by equalizing a frame rate between the string of the sequential images and the string of the reference alphas images, the frame rate being equalized by downsampling the string of the reference alpha images using the determined criteria alpha image, and means for generating the output alpha images using the string of the reference alpha images having the same frame rate as that of the string of the sequential images.

17. The apparatus according to claim 15, wherein the string of the reference alpha images has a higher frame rate than the string of the sequential images, and which includes means for generating a plurality of temporary reference alpha image strings with a plurality of phases by downsampling the string of the reference alpha images using a plurality of temporary criteria alpha images as a criteria, means for generating a plurality of temporary output alpha image strings based on the string of the temporary reference alpha images with plural phases, and means for selecting one of the temporary output alpha images as a final output alpha image string.

18. The apparatus according to claim 15, which includes means for preparing the reference alpha images with respect to the plurality of objects individually, and summing the objects to generate a composite image.

19. The apparatus according to claim 20, which includes means for generating a string of composite output alpha images by summing the objects of the output alpha images.

20. The apparatus according to claim 15, which includes means for attaching a first marking to each of the reference alpha images, means for attaching a second marking to the object of the criteria sequential image, and means for performing a phase matching by finding the criteria reference alpha image and the criteria sequential image which are nearest in time to each other referring to the first marking and the second marking.

21. The apparatus according to claim 15, wherein the storage unit stores a background image of the criteria sequential image, which includes means for detecting a position of the object by obtaining a difference between the background image and the criteria sequential image, and means for performing a phase matching by finding one of the criteria reference alpha images which is nearest in time to the criteria sequential image based on a detection result.

22. The apparatus according to claim 15, which includes means for generating a first differential image between a criteria sequential image of sequential images and a past image of the sequential images, means for generating a second differential image between the criteria sequential image and a future image of the sequential images, means for detecting a position of the object based on a logical multiplication image obtained by subjecting pixel values of the first differential image and the second differential image to a logical multiplication, and means for performing a phase matching by finding one of the reference alpha images which is nearest in time to the criteria sequential image based on a detection result.

23. The apparatus according to claim 14, wherein the criteria image determination unit comprises a determination unit configured to determine one of the plurality of reference alpha images as the criteria reference alpha image by displaying selectively the plurality of reference alpha images.

24. The apparatus according to claim 14, wherein the criteria determination unit comprises a determination unit configured to determine one of the plurality of original images as the criteria original image by displaying selectively the plurality of original images.

25. The apparatus according to claim 14, which includes means for correcting a position of contour of each of the objects based on the output alpha images.