BACKGROUND-FOREGROUND MODULE FOR VIDEO ANALYSIS SYSTEM

• US 20090087096A1
• Filed: 05/29/2008
• Published: 04/02/2009
• Est. Priority Date: 09/27/2007
• Status: Active Grant

• Current Assignee: Motorola Solutions, Inc.
• Original Assignee: Behavioral Recognition Systems Incorporated
• Inventors: Yang, Tao, Cobb, Wesley Kenneth, Saitwal, Kishor Adinath, Blythe, Bobby Ernest, Seow, Ming-Jung, Eaton, John Eric
• Granted Patent: US 8,094,943 B2
• Time in Patent Office: Days
• US Class Current: 382/190
• CPC Class Codes: G06V 10/28 Quantising the image, e.g. histogram thresholding for discrimination between background and foreground patterns

Abstract

Embodiments of the present invention provide a method and a module for identifying a background of a scene depicted in an acquired stream of video frames that may be used by a video-analysis system. For each pixel or block of pixels in an acquired video frame a comparison measure is determined. The comparison measure depends on difference of color values exhibited in the acquired video frame and in a background image respectively by the pixel or block of pixels and a corresponding pixel and block of pixels in the background image. To determine the comparison measure, the resulting difference is considered in relation to a range of possible color values. If the comparison measure is above a dynamically adjusted threshold, the pixel or the block of pixels is classified as a part of the background of the scene.

Claims

1. A method for analyzing a scene depicted in a sequence of video frames, the method comprising:

receiving a raw video frame of the sequence of video frames;

for each of a plurality of pixels of the raw video frame;

calculating a comparison measure between color-characteristics of a pixel in the raw video frame and color-characteristics of a corresponding pixel in a background image, wherein the comparison measure is defined in relation to a range of possible values of at least one color-characteristic of the pixel;

upon determining the comparison measure of the pixel is within a specified threshold, classifying the pixel of the raw frame as depicting a portion of a background of the scene; and

upon determining the comparison measure is outside the specified threshold, classifying the pixel of the raw frame as depicting a portion of a foreground of the scene.

2. The method of claim 1, wherein the specified threshold is selected based on a set of predefined brightness ranges for the at least one color-characteristic of the pixel, wherein the predefined brightness ranges include a range defining a dark pixel, a range defining a medium pixel, and a range defining a bright pixel.

3. The method of claim 1, wherein the calculating the comparison measure of the pixel in the raw video frame comprises:

for each of a plurality of color-characteristics of the pixel;

calculating a measure of closeness between a value of a color-characteristic of the pixel and a value of the color-characteristic of the corresponding pixel in the background image using a range of possible values of the color-characteristic.

4. The method of claim 3, wherein color-characteristics of the pixel and of the corresponding pixel in the background image are selected from a group consisting of a Red (R), Green (G), and Blue (B) color-channels.

5. The method of claim 3, wherein calculating the measure of closeness between color-characteristics of the pixel in the raw video frame and color-characteristics of the corresponding pixel in the background image, further comprises:

determining a first value based on a range of possible values of a first color-channel and a difference between values of the first color-channel of the pixel and of the corresponding pixel in the background image; and

determining a second value based on a range of possible values of a second color-channel and a difference between values of the second color-channel of the pixel and of the corresponding pixel in the background image, wherein the first color-channel and the second color-channel are different color-channels.

6. The method of claim 5, wherein the calculating a measure of closeness between color-characteristics of the pixel in the raw video frame and color-characteristics of the corresponding pixel in the background image, further comprises:

calculating an average of the first value and the second value.

7. The method of claim 5, wherein the calculating a measure of closeness between color-characteristics of the pixel in the raw video frame and color-characteristics of the corresponding pixel in the background image, further comprises:

determining a third value based on a range of possible values of a third color-channel and a difference between values of the third color-channel of the corresponding pixel in the background image, wherein the third color-channel is a different color-channel than the first color-channel and the second color-channel;

comparing the first value, the second value, and the third value to determine a nosiest color-channel of the pixel; and

calculating the comparison measure by calculating an average of two values selected from a group consisting of the first value, the second value, and the third value, wherein a value corresponding to the noisiest color-channel is disregarded, wherein the noisiest color-channel is a color-channel having a smallest value of the first value, second value, and the third value.

8. The method of claim 1, wherein the background image includes a short-term background image and a long-term background image.

9. The method of claim 1, further comprising, training the background image using a set of consecutive video frames of the sequence of video frames.

10. The method of claim 1, further comprising:

updating the background image using color-characteristic values of each pixel of the raw video frame classified as depicting a portion of the background of the scene.

11. The method of claim 1, further comprising:

for at least a first pixel classified as depicting a portion of a foreground of the scene, determining that the first pixel was classified as depicting a portion of the foreground of the scene within a specified marginal magnitude of the threshold;

calculating an average set of pixel color-characteristic values from a set of pixels neighboring the first pixel;

calculating a comparison measure for the average set of pixel color-characteristic values and color-characteristics of a corresponding pixel in a background image; and

upon determining the comparison measure for the average set of pixel color-characteristic values is within the specified threshold for classing a pixel as depicting scene background, re-classifying the first pixel of the raw frame as depicting a portion of a background of the scene.

12. A computer-readable storage medium containing a program which, when executed by a processor, performs an operation for analyzing a scene depicted in a sequence of video frames, the operation comprising:

receiving a raw video frame of the sequence of video frames; and

for each of a plurality of pixels of the raw video frame;

calculating a comparison measure between color-characteristics of a pixel in the raw video frame and color-characteristics of a corresponding pixel in a background image, wherein the comparison measure is defined in relation to a range of possible values of at least one color-characteristic of the pixel,

upon determining the comparison measure of the pixel is within a specified threshold, classifying the pixel of the raw frame as depicting a portion of a background of the scene, and

upon determining the comparison measure is outside the specified threshold, classifying the pixel of the raw frame as depicting a portion of a foreground of the scene.

13. The computer-readable storage medium of claim 12, wherein the specified threshold is selected based on a set of predefined brightness-ranges for the at least one color-characteristic of the pixel, wherein the predefined brightness ranges include a range defining a dark pixel, a range defining a medium pixel, and a range defining a bright pixel.

14. The computer-readable storage medium of claim 12, wherein the calculating the comparison measure of the pixel in the raw video frame comprises:

for each of a plurality of color-characteristics of the pixel;

calculating a measure of closeness between a value of a color-characteristic of the pixel and a value of the color-characteristic of the corresponding pixel in the background image using a range of possible values of the color-characteristic.

15. The computer-readable storage medium of claim 14, wherein calculating the measure of closeness between color-characteristics of the pixel in the raw video frame and color-characteristics of the corresponding pixel in the background image, further comprises:

determining a first value based on a range of possible values of a first color-channel and a difference between values of the first color-channel of the pixel and of the corresponding pixel in the background image; and

determining a second value based on a range of possible values of a second color-channel and a difference between values of the second color-channel of the pixel and of the corresponding pixel in the background image, wherein the first color-channel and the second color-channel are different color-channels.

16. The computer-readable storage medium of claim 15, wherein the calculating a measure of closeness between color-characteristics of the pixel in the raw video frame and color-characteristics of the corresponding pixel in the background image, further comprises:

calculating an average of the first value and the second value.

17. The computer-readable storage medium of claim 15, wherein the calculating a measure of closeness between color-characteristics of the pixel in the raw video frame and color-characteristics of the corresponding pixel in the background image, further comprises:

determining a third value based on a range of possible values of a third color-channel and a difference between values of the third color-channel of the corresponding pixel in the background image, wherein the third color-channel is a different color-channel than the first color-channel and the second color-channel;

comparing the first value, the second value, and the third value to determine a nosiest color-channel of the pixel; and

calculating the comparison measure by calculating an average of two values selected from a group consisting of the first value, the second value, and the third value, wherein a value corresponding to the noisiest color-channel is disregarded.

18. The computer-readable storage medium of claim 12, wherein the operation further comprises, training the background image using a set of consecutive video frames of the sequence of video frames.

19. The computer-readable storage medium of claim 12, wherein the operation further comprises updating the background image using color-characteristic values of each pixel of the raw video frame classified as depicting a portion of the background of the scene.

20. The computer-readable storage medium of claim 12, wherein the operation further comprises:

for at least a first pixel classified as depicting a portion of a foreground of the scene, determining that the first pixel was classified as depicting a portion of the foreground of the scene within a specified marginal magnitude of the threshold;

calculating an average set of pixel color-characteristic values from a set of pixels neighboring the first pixel;

calculating a comparison measure for the average set of pixel color-characteristic values and color-characteristics of a corresponding pixel in a background image; and

upon determining the comparison measure for the average set of pixel color-characteristic values is within the specified threshold for classing a pixel as depicting scene background, re-classifying the first pixel of the raw frame as depicting a portion of a background of the scene.

21. A system comprising,

a video input source configured to provide a sequence of video frames, each depicting a scene;

a processor; and

a memory containing a video analysis application, which when executed by the processor is configured to perform an operation for analyzing the scene depicted in the sequence of video frames, the operation comprising;

receiving a raw video frame from the video input source, and

for each of a plurality of pixels of the raw video frame;

calculating a comparison measure between color-characteristics of a pixel in the raw video frame and color-characteristics of a corresponding pixel in a background image, wherein the comparison measure is defined in relation to a range of possible values of at least one color-characteristic of the pixel,

upon determining the comparison measure of the pixel is within a specified threshold, classifying the pixel of the raw frame as depicting a portion of a background of the scene, and

upon determining the comparison measure is outside the specified threshold, classifying the pixel of the raw frame as depicting a portion of a foreground of the scene.

22. The system of claim 21, wherein the specified threshold is selected based on a set of predefined brightness ranges for the at least one color-characteristic of the pixel, wherein the predefined brightness ranges include a range defining a dark pixel, a range defining a medium pixel, and a range defining a bright pixel.

23. The system of claim 21, wherein the operation further comprises, training the background image using a set of consecutive video frames of the sequence of video frames.

24. The system of claim 21, wherein the operation further comprises:

updating the background image using color-characteristic values of each pixel of the raw video frame classified as depicting a portion of the background of the scene.

25. The system of claim 21, wherein the operation further comprises:

for at least a first pixel classified as depicting a portion of a foreground of the scene, determining that the first pixel was classified as depicting a portion of the foreground of the scene within a specified marginal magnitude of the threshold;

calculating an average set of pixel color-characteristic values from a set of pixels neighboring the first pixel;

calculating a comparison measure for the average set of pixel color-characteristic values and color-characteristics of a corresponding pixel in a background image; and

upon determining the comparison measure for the average set of pixel color-characteristic values is within the specified threshold for classing a pixel as depicting scene background, re-classifying the first pixel of the raw frame as depicting a portion of a background of the scene.