Robust and efficient foreground analysis for real-time video surveillance

• US 20090067716A1
• Filed: 01/20/2005
• Published: 03/12/2009
• Est. Priority Date: 01/20/2005
• Status: Active Grant

• Current Assignee: International Business Machines Corporation
• Original Assignee: International Business Machines Corporation
• Inventors: Hampapur, Arun, Lu, Zuoxuan, Brown, Lisa Marie, Tian, Ying-Li, Senior, Andrew William
• Granted Patent: US 7,620,266 B2
• Time in Patent Office: Days
• US Class Current: 382/173
• CPC Class Codes: G06V 10/28 Quantising the image, e.g. histogram thresholding for discrimination between background and foreground patterns
• CPC Class Codes: G06V 20/52 Surveillance or monitoring of activities, e.g. for recognising suspicious objects recognising microscopic objects G06V20/69
• CPC Class Codes: G06V 40/18 Eye characteristics, e.g. of the iris

Abstract

Systems and methods for foreground analysis in real-time video include background subtraction and foreground detection, shadow removal, quick lighting change adaptation, static foreground region detection, foreground fragment reduction, and frame level change detection. Processes include background image extraction and foreground detection, integrating texture information of the background image and a current frame to remove false positive foreground areas resulting from lighting changes, integrating pixel intensity information by determining a cross-correlation of intensities between a current frame and the background image for each pixel in a foreground mask to remove image shadows. Static foreground region detection and fragment reduction are also included.

Claims

1. A method for foreground analysis in real-time video, comprising the steps of:

determining mixtures of Gaussians for different aspects of an image;

detecting a static foreground region by using the mixtures of Gaussians to identify whether a pixel belongs to the static region or a background region; and

processing the image in accordance with two or more of;

detecting a static region to determine objects in a scene using the mixtures of Gaussians of the background region;

integrating texture information of the background image to a foreground mask to remove false positive foreground areas resulting from lighting changes; and

integrating pixel intensity information by determining a cross-correlation of intensities between a current frame and the background image for each pixel in a foreground mask to remove image shadows.

3. The method as recited in claim 1, wherein the step of integrating texture information of the background image to a foreground mask to remove false positive foreground areas resulting from lighting changes includes defining a texture similarity measure for a pixel between a current frame and die background image based on a pixel intensity gradient.

4. The method as recited in claim 3, wherein the texture similarity measure provides a criterion for removing the false positive foreground areas to perform the step of subtracting the background image.

6. The method as recited in claim 1, wherein the Gaussian mixtures include at least one of:

Gaussian mixtures for the background image, Gaussian mixtures for quick lighting changes, and Gaussian mixtures for repetitive variations in the image to determine if a foreground pixel belongs to a static region.

7. The method as recited in claim 1, further comprising the step of reducing foreground fragments by pushing back a static region representing the foreground fragment back to the background image at a point when the static region is largest.

8. The method as recited in claim 7, wherein die pushing back the static region includes resetting a pixel weight to a maximum value by exchanging mean and variance values between Gaussian distributions.

9. The method as recited in claim 1, further comprising the step of resetting the background image to a current frame during at least one of camera movement, camera blind and quick lighting changes.

10. A computer-readable medium storing a computer program to perform method steps for foreground analysis in real-time video as recited in claim 1.

2. (canceled)

5. (canceled)

11. A method for foreground analysis in real-time video, comprising the steps of:

determining mixtures of Gaussians for different aspects of an image;

detecting a static region by using the mixtures of Gaussians to identify whether a pixel belongs to the static region or a background region; and

reducing foreground fragments by pushing back the entire static region back to the background region at a point when the static region is largest.

12. The method as recited in claim 11, further comprising the step of integrating pixel intensity information by determining a cross-correlation of intensities between a current frame and the background image for each pixel in a foreground mask to remove image shadows.

13. The method as recited in claim 12, wherein the step of integrating pixel intensity information by determining a cross-correlation of intensities between a current frame and the background image for each pixel in a foreground mask to remove image shadows includes calculating a normalized cross-correlation (NCC) in a neighborhood of a given pixel and comparing the NCC value to a threshold to determine in the pixel is affected by shadow.

14. The method as recited in claim 11, further comprising the step of integrating texture information of the background image to a foreground mask to remove false positive foreground areas resulting from lighting changes.

15. The method as recited in claim 11, wherein the Gaussian mixtures include at least one of:

Gaussian mixtures for the background image, Gaussian mixtures for quick lighting changes, and Gaussian mixtures for repetitive variations in the image to determine if a foreground pixel belongs to a static region.

16. The method as recited in claim 11, wherein the pushing back the static region includes resetting a pixel weight to a maximum value by exchanging mean and variance values between Gaussian distributions.

17. The method as recited in claim 11, further comprising the step of resetting the background model to a current frame during at least one of camera movement, camera blind and quick lighting changes.

18. A computer-readable medium storing a computer program to perform method steps for foreground analysis in real-time video as recited in claim 11.

19. A system for foreground analysis in real-time video, comprising.

a background subtraction module which determines a mixture of Gaussians for a background region of an image and subtracts out portions of the image in accordance with an integration module;

and

the integration module detects at least static regions by using the mixtures of Gaussians to identify whether a pixel belongs to the static region or the background region and reduces foreground fragments of the static regions by pushing back each entire static region to the background region at a point when the static region is largest.

20. The system as recited in claim 19, wherein the integration module detects false positive foreground areas and background subtraction module removes false positive foreground areas by comparing texture information of the background image to a foreground mask to remove the false positive foreground areas resulting from quick lighting changes.

21. The system as recited in claim 19, wherein the integration module detects shadows and the background subtraction module removes shadows by integrating pixel intensity information by determining a cross-correlation of intensities between a current frame and the background image for each pixel in a foreground mask to remove image shadows.

22. The system as recited in claim 20, wherein the subtraction module computes pixel probabilities and determines, based on criteria, which densities represent the background image.

23. The system as recited in claim 20, wherein the integration module computes a normalized cross-correlation (NCC) in a neighborhood of a given pixel and compares the NCC value to a threshold to determine in the pixel is affected by shadow.

24. The system as recited in claim 20, wherein the integration module integrates texture information by computing a texture similarity measure for a pixel between a current frame and the background image based on a pixel intensity gradient.

25. The system as recited in claim 20, further comprising a camera for collecting video data.

26. The system as recited in claim 20, wherein the system detects static objects, camera movement, quick lighting changes and shadows.