ADAPTIVE UPDATE OF BACKGROUND PIXEL THRESHOLDS USING SUDDEN ILLUMINATION CHANGE DETECTION

• US 20100208986A1
• Filed: 02/18/2009
• Published: 08/19/2010
• Est. Priority Date: 02/18/2009
• Status: Active Grant

• Current Assignee: Motorola Solutions, Inc., Pepperl + Fuchs GmbH
• Original Assignee: Behavioral Recognition Systems Incorporated
• Inventors: COBB, WESLEY KENNETH, Yang, Tao, Saitwal, Kishor Adinath, Blythe, Bobby Ernest
• Granted Patent: US 8,285,046 B2
• Time in Patent Office: Days
• US Class Current: 382/165
• CPC Class Codes: G06V 20/52 Surveillance or monitoring of activities, e.g. for recognising suspicious objects recognising microscopic objects G06V20/69

Abstract

Techniques are disclosed for a computer vision engine to update both a background model and thresholds used to classify pixels as depicting scene foreground or background in response to detecting that a sudden illumination changes has occurred in a sequence of video frames. The threshold values may be used to specify how much pixel a given pixel may differ from corresponding values in the background model before being classified as depicting foreground. When a sudden illumination change is detected, the values for pixels affected by sudden illumination change may be used to update the value in the background image to reflect the value for that pixel following the sudden illumination change as well as update the threshold for classifying that pixel as depicting foreground/background in subsequent frames of video.

Claims

1. A computer-implemented method for analyzing a sequence of video frames depicting a scene captured by a video camera, the method comprising:

classifying each of a plurality of pixels in a first frame of the sequence of video frames as depicting one of scene background and scene foreground, wherein each pixel is classified based on a comparison of (i) a relative deviation determined using a pixel value from the first frame and a corresponding pixel value in a background image and (ii) a classification threshold assigned to each pixel in the background image; and

upon determining that a group of pixels in the first frame were classified as depicting scene foreground as a result of a sudden illumination change (SIC);

for each pixel in the group of pixels;

re-classifying the pixel as depicting scene background, and

decreasing the classification threshold assigned to the pixel to a value below the relative deviation determined for the pixel; and

for one or more successive frames of video following the first frame;

updating the pixel values in the background image corresponding to the group of pixels, based on pixel values obtained from video frames following the first frame, and

increasing the classification threshold assigned to pixels in the background image for pixels corresponding to the group of pixels.

2. The computer-implemented method of claim 1, wherein the pixel values in the background image represent a pixel value expected to be present in a frame of video captured by the video camera when the pixel depicts scene background.

3. The computer-implemented method of claim 1, further comprising:

prior to classifying each of the plurality of pixels in the first frame of the sequence of video frames as depicting one of scene background and scene foreground, assigning an initial threshold to the pixel values in the background image based on a camera noise model and an initial radiance of a respective pixel in the background image.

4. The computer-implemented method of claim 1, wherein classifying each of the plurality of pixels in the first frame of the sequence of video frames as depicting one of scene background and scene foreground, comprises:

determining the relative deviation as ρ

_R, ρ

_G, and ρ

_B between a first pixel in the first frame and the corresponding pixel in the background image according to;

5. The computer-implemented method of claim 4, wherein the overall relative difference (Δ

) for the first pixel is calculated from the ρ

_R, ρ

_G, and ρ

_B values according to;

6. The computer-implemented method of claim 1, further comprising:

classifying one or more of the pixels in the group as being an SIC candidate pixel by evaluating a spatial neighborhood of each pixel in the group, wherein the spatial neighborhood comprises a 3×

3 grid of pixels with the pixel being evaluated at the center of the 3×

3 grid.

7. The computer-implemented method of claim 6, wherein determining that the group of pixels in the first frame were classified as depicting scene foreground as the result of the sudden illumination change (SIC) comprises, determining that a majority of pixels in the group are classified as SIC candidate pixels, based on the evaluations of the spatial neighborhoods of the pixels in the group.

8. The computer-implemented method of claim 1, wherein the pixel values in the background image corresponding to the group of pixels are updated according to the following:

bg_r=α

(bg_r)+(1−

α

)(raw_r) bg_g=α

(bg_g)+(1−

α

)(raw_g) bg_b=α

(bg_b)+(1−

α

)(raw_b)

wherein bg_r, bg_g, and bg_b represent color channel values for a respective pixel in the background image, raw_r, raw_g, and raw_b represent the R, G, and B color channels values for the corresponding color channel values in a current frame, and wherein a represents a value selected from a range of [0,1], inclusive.

9. A computer-readable storage medium containing a program which, when executed by a processor, performs an operation for analyzing a sequence of video frames depicting a scene captured by a video camera, the operation comprising:

classifying each of a plurality of pixels in a first frame of the sequence of video frames as depicting one of scene background and scene foreground, wherein each pixel is classified based on a comparison of (i) a relative deviation determined using a pixel value from the first frame and a corresponding pixel value in a background image and (ii) a classification threshold assigned to each pixel in the background image; and

upon determining that a group of pixels in the first frame were classified as depicting scene foreground as a result of a sudden illumination change (SIC);

for each pixel in the group of pixels;

re-classifying the pixel as depicting scene background, and

decreasing the classification threshold assigned to a value below the relative deviation determined for the pixel using the pixel value from the first frame and the corresponding pixel value from the background image; and

for one or more successive frames of video following the first frame;

updating the pixel values in the background image corresponding to the group of pixels, based on pixel values obtained from video frames following the first frame, and

increasing the classification threshold assigned to pixels in the background image for pixels corresponding to the group of pixels.

10. The computer-readable storage medium of claim 9, wherein the pixel values in the background image represent a pixel value expected to be present in a frame of video captured by the video camera when the pixel depicts scene background.

11. The computer-readable storage medium of claim 9, wherein the operation further comprises:

prior to classifying each of the plurality of pixels in the first frame of the sequence of video frames as depicting one of scene background and scene foreground, assigning an initial threshold to the pixel values in the background image based on a camera noise model and an initial radiance of a respective pixel in the background image.

12. The computer-readable storage medium of claim 9, wherein classifying each of the plurality of pixels in the first frame of the sequence of video frames as depicting one of scene background and scene foreground, comprises:

determining the relative deviation as ρ

_R, ρ

_G, and ρ

_B between a first pixel in the first frame and the corresponding pixel in the background image according to;

13. The computer-readable storage medium of claim 12, wherein the overall relative difference (Δ

) for the first pixel is calculated from the ρ

_R, ρ

_G, and ρ

_B values according to;

14. The computer-readable storage medium of claim 9, wherein the operation further comprises:

classifying one or more of the pixels in the group as being an SIC candidate pixel by evaluating a spatial neighborhood of each pixel in the group, wherein the spatial neighborhood comprises a 3×

3 grid of pixels with the pixel being evaluated at the center of the 3×

3 grid.

15. The computer-readable storage medium of claim 14, wherein determining that the group of pixels in the first frame were classified as depicting scene foreground as the result of the sudden illumination change (SIC) comprises, determining that a majority of pixels in the group are classified as SIC candidate pixels, based on the evaluations of the spatial neighborhoods of the pixels in the group.

16. The computer-readable storage medium of claim 9, wherein the pixel values in the background image corresponding to the group of pixels are updated according to the following:

bg_r=α

(bg_r)+(1−

α

)(raw_r) bg_g=α

(bg_g)+(1−

α

)(raw_g) bg_b=α

(bg_b)+(1−

α

)(raw_b)

wherein bg_r, bg_g, and bg_b represent color channel values for a respective pixel in the background image, raw_r, raw_g, and raw_b represent the R, G, and B color channels values for the corresponding color channel values in a current frame, and wherein a represents a value selected from a range of [0,1], inclusive.

17. A system, comprising:

a video input source configured to provide a sequence of video frames, each depicting a scene;

a processor; and

a memory containing a program, which, when executed on the processor is configured to perform an operation for analyzing the scene, as depicted by the sequence of video frames captured by the video input source, the operation comprising;

classifying each of a plurality of pixels in a first frame of the sequence of video frames as depicting one of scene background and scene foreground, wherein each pixel is classified based on a comparison of (i) a relative deviation determined using a pixel value from the first frame and a corresponding pixel value in a background image and (ii) a classification threshold assigned to each pixel in the background image,

upon determining that a group of pixels in the first frame were classified as depicting scene foreground as a result of a sudden illumination change (SIC);

for each pixel in the group of pixels;

re-classifying the pixel as depicting scene background; and

decreasing the classification threshold assigned to a value below the relative deviation determined for the pixel using the pixel value from the first frame and the corresponding pixel value from the background image, and

for one or more successive frames of video following the first frame;

updating the pixel values in the background image corresponding to the group of pixels, based on pixel values obtained from video frames following the first frame; and

increasing the classification threshold assigned to pixels in the background image for pixels corresponding to the group of pixels.

18. The system of claim 17, wherein the pixel values in the background image represent a pixel value expected to be present in a frame of video captured by the video camera when the pixel depicts scene background.

19. The system of claim 17, wherein the operation further comprises:

prior to classifying each of the plurality of pixels in the first frame of the sequence of video frames as depicting one of scene background and scene foreground, assigning an initial threshold to the pixel values in the background image based on a camera noise model and an initial radiance of a respective pixel in the background image.

20. The system of claim 17, wherein classifying each of the plurality of pixels in the first frame of the sequence of video frames as depicting one of scene background and scene foreground, comprises:

determining the relative deviation as ρ

_R, ρ

_G, and ρ

_B between a first pixel in the first frame and the corresponding pixel in the background image according to;

21. The system of claim 20, wherein the overall relative difference (Δ

) for the first pixel is calculated from the ρ

_R, ρ

_G, and ρ

_B values according to;

22. The system of claim 17, wherein the operation further comprises:

classifying one or more of the pixels in the group as being an SIC candidate pixel by evaluating a spatial neighborhood of each pixel in the group, wherein the spatial neighborhood comprises a 3×

3 grid of pixels with the pixel being evaluated at the center of the 3×

3 grid.

23. The system of claim 22, wherein determining that the group of pixels in the first frame were classified as depicting scene foreground as the result of the sudden illumination change (SIC) comprises, determining that a majority of pixels in the group are classified as SIC candidate pixels, based on the evaluations of the spatial neighborhoods of the pixels in the group.

24. The system of claim 17, wherein the pixel values in the background image corresponding to the group of pixels are updated according to the following:

bg_r=α

(bg_r)+(1−

α

)(raw_r) bg_g=α

(bg_g)+(1−

α

)(raw_g) b g_b=α

(bg_b)+(1−

α

)(raw_b)

wherein bg_r, bg_g, and bg_b represent color channel values for a respective pixel in the background image, raw_r, raw_g, and raw_b represent the R, G, and B color channels values for the corresponding color channel values in a current frame, and wherein a represents a value selected from a range of [0,1], inclusive.