Automatic gain control filter in a video analysis system

• US 9,317,908 B2
• Filed: 06/28/2013
• Issued: 04/19/2016
• Est. Priority Date: 06/29/2012
• Status: Active Grant

• Current Assignee: Intellective Ai, Inc.
• Original Assignee: Behavioral Recognition Systems Incorporated
• Inventors: Yang, Tao, Cobb, Wesley Kenneth, Seow, Ming-Jung
• Primary Examiner(s): YEH, EUENG NAN
• Application Number:
  US13/930,222
• Publication Number:
  US 20140003713A1
• Time in Patent Office: 1,026 Days
• Field of Search: None
• US Class Current: 1/1
• CPC Class Codes: G06F 18/23 Clustering techniques
• CPC Class Codes: G06F 18/24 Classification techniques
• CPC Class Codes: G06T 2207/10016 Video; Image sequence
• CPC Class Codes: G06T 2207/10024 Color image
• CPC Class Codes: G06T 2207/20081 Training; Learning
• CPC Class Codes: G06T 2207/20208 High dynamic range [HDR] image processing
• CPC Class Codes: G06T 5/92 based on global image properties
• CPC Class Codes: G06T 5/94 based on local image properties, e.g. for local contrast enhancement
• CPC Class Codes: G06T 7/10 Segmentation; Edge detection motion-based segmentation G06T7/215
• CPC Class Codes: G06T 7/11 Region-based segmentation
• CPC Class Codes: G06T 7/194 involving foreground-background segmentation
• CPC Class Codes: G06T 7/251 involving models

Abstract

Techniques are disclosed for analyzing a scene depicted in an input stream of video frames captured by a video camera. Bounding boxes are determined for a set foreground patches identified in a video frame. For each bounding box, the techniques include determining textures for first areas, each including a foreground pixel and surrounding pixels, and determining textures for second areas including pixels of the background model image corresponding to the pixels of the foreground areas. Further, for each foreground pixel in the bounding box area, a correlation score is determined based on the texture of the corresponding first area and second area. Pixels whose correlation scores exceed a threshold are removed from the foreground patch. The size of the bounding box may also be reduced to fit the modified foreground patch.

Claims

1. A computer-implemented method for filtering out false-positive foreground pixels, the method comprising:

extracting foreground patches from a video frame using a background model image, the foreground patches each including respective foreground pixels;

determining a bounding box for each foreground patch;

for each bounding box;

(1) for each foreground pixel in the foreground patch in the bounding box;

(A) determining a texture of a first area including the foreground pixel and pixels surrounding the foreground pixel and a texture of a second area including pixels of the background model image corresponding to the first area pixels, wherein the textures of the first area and the second area each represent a local variability of intensity values of respective pixels, and

(B) determining a correlation score based on the texture of the first area and the texture of the second area, and

(2) for foreground pixels having correlation scores which exceed a threshold value, removing the foreground pixels from the foreground patch in the bounding box, wherein the threshold value is proportional to a size of the bounding box; and

for each bounding box having foreground pixels removed, reducing a size of the bounding box based on the removal of the foreground pixels.

2. The method of claim 1, wherein determining texture of the first area includes determining gradient values for the foreground pixel and the pixels surrounding the foreground pixel, and wherein determining texture of the second area includes determining gradient values for the pixels of the background model image corresponding to the first area pixels.

3. The method of claim 2, wherein the gradient values are determined using an edge detection technique which is resilient to noise and invariant to illumination and/or color/hue changes.

4. The method of claim 2, wherein the gradient values are determined by applying one of a Sobel Operator and a Haar wavelet transform.

5. The method of claim 1, wherein the number of surrounding the foreground pixel used in the first area is proportional to the size of the bounding box.

6. The method of claim 1, further comprising, prior to determining the textures, converting the foreground and background pixels to grayscale.

7. A non-transitory computer-readable storage medium storing instructions, which when executed by a computer system, perform operations for filtering out false-positive foreground pixels, the operations comprising:

extracting foreground patches from a video frame using a background model image, the foreground patches each including respective foreground pixels;

determining a bounding box for each foreground patch;

for each bounding box;

(1) for each foreground pixel in the foreground patch in the bounding box;

(A) determining a texture of a first area including the foreground pixel and pixels surrounding the foreground pixel and a texture of a second area including pixels of the background model image corresponding to the first area pixels, wherein the textures of the first area and the second area each represent a local variability of intensity values of respective pixels, and

(B) determining a correlation score based on the texture of the first area and the texture of the second area, and

(2) for foreground pixels having correlation scores which exceed a threshold value, removing the foreground pixels from the foreground patch in the bounding box, wherein the threshold value is proportional to a size of the bounding box; and

for each bounding box having foreground pixels removed, reducing a size of the bounding box based on the removal of the foreground pixels.

8. The computer-readable storage medium of claim 7, wherein determining texture of the first area includes determining gradient values for the foreground pixel and the pixels surrounding the foreground pixel, and wherein determining texture of the second area includes determining gradient values for the pixels of the background model image corresponding to the first area pixels.

9. The computer-readable storage medium of claim 8, wherein the gradient values are determined using an edge detection technique which is resilient to noise and invariant to illumination and/or color/hue changes.

10. The computer-readable storage medium of claim 8, wherein the gradient values are determined by applying one of a Sobel Operator and a Haar wavelet transform.

11. The computer-readable storage medium of claim 7, wherein the number of surrounding the foreground pixel used in the first area is proportional to the size of the bounding box.

12. The computer-readable storage medium of claim 7, the operations further comprising, prior to determining the textures, converting the foreground and background pixels to grayscale.

13. A system, comprising:

a processor; and

a memory, wherein the memory includes an application program configured to perform operations for filtering out false-positive foreground pixels, the operations comprising;

extracting foreground patches from a video frame using a background model image, the foreground patches each including respective foreground pixels,

determining a bounding box for each foreground patch,

for each bounding box;

(1) for each foreground pixel in the foreground patch in the bounding box;

(A) determining a texture of a first area including the foreground pixel and pixels surrounding the foreground pixel and a texture of a second area including pixels of the background model image corresponding to the first area pixels, wherein the textures of the first area and the second area each represent a local variability of intensity values of respective pixels, and

(B) determining a correlation score based on the texture of the first area and the texture of the second area, and

(2) for foreground pixels having correlation scores which exceed a threshold value, removing the foreground pixels from the foreground patch in the bounding box, wherein the threshold value is proportional to a size of the bounding box, and

for each bounding box having foreground pixels removed, reducing a size of the bounding box based on the removal of the foreground pixels.

14. The system of claim 13,

wherein the number of surrounding the foreground pixel used in the first area is proportional to the size of the bounding box.