Adaptive illuminance filter in a video analysis system

• US 9,111,353 B2
• Filed: 06/28/2013
• Issued: 08/18/2015
• Est. Priority Date: 06/29/2012
• Status: Active Grant

• Current Assignee: Intellective Ai, Inc.
• Original Assignee: Behavioral Recognition Systems Incorporated
• Inventors: Seow, Ming-Jung, Yang, Tao, Cobb, Wesley Kenneth
• Primary Examiner(s): CHAWAN, SHEELA C
• Application Number:
  US13/930,276
• Publication Number:
  US 20140003720A1
• Time in Patent Office: 781 Days
• Field of Search: 382/173, 382/100, 382/103, 382/106, 382/107, 382/155, 382/162, 382/168, 382/181, 382/193, 382/199, 382/209, 382/218, 382/219, 382/224, 382/232, 382/254, 382/274, 382/276, 382/282, 382286-294, 382/305, 382/312, 382/190, 707/791, 340/573.1, 340/948, 348/586, 348/155, 348/152, 348/153, 348/169, 348/587, 348/221.1, 348/208.4, 348/E5.058
• US Class Current: 1/1
• CPC Class Codes: G06T 2207/10016 Video; Image sequence
• CPC Class Codes: G06T 2207/10024 Color image
• CPC Class Codes: G06T 2207/20081 Training; Learning
• CPC Class Codes: G06T 2207/20084 Artificial neural networks [ANN]
• CPC Class Codes: G06T 2207/30232 Surveillance
• CPC Class Codes: G06T 7/254 involving subtraction of images
• CPC Class Codes: G06V 20/52 Surveillance or monitoring of activities, e.g. for recognising suspicious objects recognising microscopic objects G06V20/69

Abstract

Techniques are disclosed for removing false-positive foreground pixels resulting from environmental illumination effects. The techniques include receiving a foreground image and a background model, and determining an approximated reflectance component of the foreground image based on the foreground image itself and a background model image which is used as a proxy for an illuminance component of the foreground image. Pixels of the foreground image having approximated reflectance values less than a threshold value may be classified as false-positive foreground pixels and removed from the foreground image. Further, the threshold value used may be adjusted based on various factors to account for, e.g., different illumination conditions indoors and outdoors.

Claims

1. A computer-implemented method for removing false-positive foreground pixels from a video frame, comprising:

extracting, via one or more processors, a foreground image from the video frame using a background model image;

determining, based on at least the foreground image and portions of the background model image used as a proxy for an illuminance component of the foreground image, an approximated reflectance component of the foreground image, wherein the portions of the background image are pixels that correspond to respective pixels in the foreground image and wherein the approximated reflectance component provides approximated reflectance values for the respective pixels in the foreground image; and

removing from the foreground image pixels having approximated reflectance values less than a first threshold value.

2. The method of claim 1, wherein the approximated reflectance component is determined using ln(r(x,y))=ln(f(x,y))−

ln(b(x,y)), where r(x,y) is the approximated reflectance component, f(x,y) is the foreground image, and b(x,y) includes pixels of the background model image corresponding to those of the foreground image.

3. The method of claim 1, wherein the first threshold value used is a predefined value for an indoor environment or an outdoor environment, the predefined value being greater for the indoor environment than for the outdoor environment.

4. The method of claim 1, wherein the first threshold value is determined based on a noise level of the video frame.

5. The method of claim 4, wherein the first threshold value is determined based on a comparison of counts of static foreground pixels for the video frame and for a preceding video frame and a comparison of counts of dynamic foreground pixels for the video frame and for the preceding video frame,

wherein the static foreground pixels are foreground pixels whose values have not changed, or have changed less than a second threshold value, from a previous frame to the video frame, and

wherein the dynamic foreground pixels are foreground pixels whose values have changed, or have changed more than the second threshold value, from the previous frame to the video frame.

6. The method of claim 4, further comprising, upon determining that the noise level is below a third threshold value, halting the removal of the foreground image pixels.

7. The method of claim 4, wherein the noise level is used to determine one of disabled, outdoor enhance, outdoor aggressive, indoor regular, indoor aggressive, and indoor extreme settings for the first threshold value, and wherein the settings are associated with threshold values ordered as:

outdoor enhance threshold<

outdoor aggressive threshold<

indoor regular threshold<

indoor aggressive threshold<

indoor extreme threshold.

8. The method of claim 1, wherein the background model image is generated using per-pixel adaptive resonance theory (ART) networks, each ART network modeling a respective pixel.

9. The method of claim 1, further comprising, converting the foreground image and the background model image to grayscale before determining the approximated reflectance component.

10. A non-transitory computer-readable storage medium storing instructions, which when executed by a computer system, perform operations for analyzing a scene depicted in an input stream of video frames captured by a video camera, the operations comprising:

extracting a foreground image from the video frame using a background model image;

determining, based on at least the foreground image and the portions of the background model image used as a proxy for an illuminance component of the foreground image, an approximated reflectance component of the foreground image, wherein the portions of the background image are pixels that correspond to respective pixels in the foreground image and wherein the approximated reflectance component provides approximated reflectance values for the respective pixels in the foreground image; and

removing from the foreground image pixels having approximated reflectance values less than a first threshold value.

11. The computer-readable storage medium of claim 10, wherein the approximated reflectance component is determined using ln(r(x,y))=ln(f(x,y))−

ln(b(x,y)), where r(x,y) is the approximated reflectance component, f(x,y) is the foreground image, and b(x,y) includes pixels of the background model image corresponding to those of the foreground image.

12. The computer-readable storage medium of claim 10, wherein the first threshold value used is a predefined value for an indoor environment or an outdoor environment, the predefined value being greater for the indoor environment than for the outdoor environment.

13. The computer-readable storage medium of claim 10, wherein the first threshold value is determined based on a noise level of the video frame.

14. The computer-readable storage medium of claim 13, wherein the first threshold value is determined based on a comparison of counts of static foreground pixels for the video frame and for a preceding video frame and a comparison of counts of dynamic foreground pixels for the video frame and for the preceding video frame,

wherein the static foreground pixels are foreground pixels whose values have not changed, or have changed less than a second threshold value, from a previous frame to the video frame, and

wherein the dynamic foreground pixels are foreground pixels whose values have changed, or have changed more than the second threshold value, from the previous frame to the video frame.

15. The computer-readable storage medium of claim 13, the operations further comprising, upon determining that the noise level is below a third threshold value, halting the removal of the foreground image pixels.

16. The computer-readable storage medium of claim 13, wherein the noise level is used to determine one of disabled, outdoor enhance, outdoor aggressive, indoor regular, indoor aggressive, and indoor extreme settings for the first threshold value, and wherein the settings are associated with threshold values ordered as:

outdoor enhance threshold<

outdoor aggressive threshold<

indoor regular threshold<

indoor aggressive threshold<

indoor extreme threshold.

17. The computer-readable storage medium of claim 10, wherein the background model image is generated using per-pixel adaptive resonance theory (ART) networks, each ART network modeling a respective pixel.

18. The computer-readable storage medium of claim 10, the operations further comprising, converting the foreground image and the background model image to grayscale before determining the approximated reflectance component.

19. A system, comprising:

a processor; and

a memory, wherein the memory includes an application program configured to perform operations for removing false-positive foreground pixels from a video frame, the operations comprising;

extracting a foreground image from the video frame using a background model image,

determining, based on at least the foreground image and portions of the background model image used as a proxy for an illuminance component of the foreground image, an approximated reflectance component of the foreground image, wherein the portions of the background image are pixels that correspond to respective pixels in the foreground image and wherein the approximated reflectance component provides approximated reflectance values for the respective pixels in the foreground image, and

removing from the foreground image pixels having approximated reflectance values less than a first threshold value.

20. The system of claim 19, wherein the approximated reflectance component is determined using ln(r(x,y))=ln(f(x,y))−

ln(b(x,y)), where r(x,y) is the approximated reflectance component, f(x,y) is the foreground image, and b(x,y) includes pixels of the background model image corresponding to those of the foreground image.