Automatic surveillance video matting using a shape prior

• US 9,305,357 B2
• Filed: 11/07/2011
• Issued: 04/05/2016
• Est. Priority Date: 11/07/2011
• Status: Active Grant

• Current Assignee: General Electric Company
• Original Assignee: General Electric Company
• Inventors: Yu, Ting, Tu, Peter Henry, Liu, Xiaoming, Lim, Ser Nam
• Primary Examiner(s): Diep, Nhon
• Application Number:
  US13/290,928
• Publication Number:
  US 20130114942A1
• Time in Patent Office: 1,611 Days
• Field of Search: None
• US Class Current: 1/1
• CPC Class Codes: G06F 18/214 Generating training patterns; Bootstrap methods, e.g. bagging or boosting
• CPC Class Codes: G06T 11/60 Editing figures and text; Combining figures or text
• CPC Class Codes: G06T 2207/20081 Training; Learning
• CPC Class Codes: G06T 2207/20221 Image fusion; Image merging
• CPC Class Codes: G06T 2207/30196 Human being; Person
• CPC Class Codes: G06T 2207/30204 Marker
• CPC Class Codes: G06T 7/11 Region-based segmentation
• CPC Class Codes: G06T 7/194 involving foreground-background segmentation
• CPC Class Codes: G06T 7/33 using feature-based methods
• CPC Class Codes: G06V 10/7557 based on appearance, e.g. active appearance models [AAM]
• CPC Class Codes: H04N 5/272 Means for inserting a foreground image in a background image, i.e. inlay, outlay

Abstract

A novel technique for performing video matting, which is built upon a proposed image matting algorithm that is fully automatic is disclosed. The disclosed methods utilize a PCA-based shape model as a prior for guiding the matting process, so that manual interactions required by most existing image matting methods are unnecessary. By applying the image matting algorithm to these foreground windows, on a per frame basis, a fully automated video matting process is attainable. The process of aligning the shape model with the object is simultaneously optimized based on a quadratic cost function.

Claims

1. A method comprising:

computing an initial alpha matte; and

automatically updating the initial alpha matte, wherein automatically updating comprises;

incorporating one or more shape models into the alpha matte;

incorporating one or more alignment parameters into the alpha matte; and

optimizing the alpha matte; and

wherein incorporating the one or more alignment parameters into the alpha matte comprises;

calculating an updated alpha matte;

calculating a shape basis coefficient; and

calculating a transformation parameter.

2. The method of claim 1, wherein computing the initial alpha matte comprises calculating a Laplacian matting.

3. The method of claim 1, wherein incorporating the one or more shape models comprises selecting one or more prior shapes from a database.

4. The method of claim 1, wherein incorporating the one or more shape models comprises incorporating one or more principal component analysis (PCA) based shape models.

5. The method of claim 1, wherein each of the calculating steps occurs substantially simultaneously.

6. The method of claim 1, wherein optimizing the alpha matte comprises iteratively repeating the step of incorporating the one or more alignment parameters.

7. The method of claim 6, wherein iteratively repeating comprises iteratively repeating less than 20 times.

8. The method of claim 1, wherein calculating the initial alpha matte comprises extracting a foreground image from a background of a still image.

9. The method of claim 1, wherein calculating the initial alpha matte comprises extracting a foreground image from a background of a video frame.

10. The method of claim 1, wherein calculating the initial alpha matte comprises calculating a plurality of alpha mattes by extracting a foreground image from a background for each frame of a video comprising a plurality of frames.

11. The method of claim 1, wherein automatically updating the initial alpha matte does not include marking a foreground region of the alpha matte or a background region of the alpha matte.

12. A method for automatic video matting comprising:

obtaining a plurality of video images;

utilizing a respective one of a plurality of shape priors to estimate an alpha matte for each of the plurality of video images;

aligning each of the video images to estimate the alpha matte for each of the plurality of video images;

optimizing the alpha matte for each of the plurality of video images, wherein optimizing comprises utilizing a quadratic cost function, and simultaneously utilizing the respective one of the plurality of shape priors and aligning each of the video images.

13. The method of claim 12, wherein each of the plurality of video images is obtained from a person detector.

14. The method of claim 12, wherein optimizing comprises simultaneously calculating the alpha matte, shape basis coefficients and transformation parameters.

15. The method of claim 12, comprising iteratively repeating each of the utilizing and aligning steps.

16. A system comprising:

a database having a plurality of shape models stored therein; and

a processor configured to receive a plurality of images and to align a respective one of the plurality of shape models with each of the plurality of images to produce an alpha matte corresponding to each of the plurality of images,

wherein the processor is configured to calculate a quadratic cost function by simultaneously calculating the alpha matte, shape basis coefficients and transformation parameters.

17. The system of claim 16, comprising a detector configured to detect the plurality of images and transmit the plurality of images to the processor.

18. The system of claim 17, wherein the detector is a person detector and the plurality of images comprise video images.

19. The system of claim 16, wherein the processor is configured to align each of the plurality of images with a respective one of the plurality of shape models through a spatial transformation.