Incremental update for background model thresholds

• US 9,460,522 B2
• Filed: 10/29/2014
• Issued: 10/04/2016
• Est. Priority Date: 10/29/2014
• Status: Active Grant

• Current Assignee: Intellective Ai, Inc.
• Original Assignee: Behavioral Recognition Systems Incorporated
• Inventors: Saitwal, Kishor Adinath, Risinger, Lon, Cobb, Wesley Kenneth
• Primary Examiner(s): Liu, Li
• Application Number:
  US14/526,879
• Publication Number:
  US 20160125621A1
• Time in Patent Office: 706 Days
• Field of Search: None
• US Class Current: 1/1
• CPC Class Codes: G06T 2207/10016 Video; Image sequence
• CPC Class Codes: G06T 2207/10024 Color image
• CPC Class Codes: G06T 2207/20036 Morphological image processing
• CPC Class Codes: G06T 2207/20076 Probabilistic image processing
• CPC Class Codes: G06T 7/194 involving foreground-background segmentation
• CPC Class Codes: G06T 7/215 Motion-based segmentation
• CPC Class Codes: G06T 7/254 involving subtraction of images
• CPC Class Codes: G06T 7/90 Determination of colour characteristics
• CPC Class Codes: G06V 10/28 Quantising the image, e.g. histogram thresholding for discrimination between background and foreground patterns
• CPC Class Codes: G06V 10/30 Noise filtering
• CPC Class Codes: G06V 20/00 Scenes; Scene-specific elements control of digital cameras H04N23/60

Abstract

Techniques are disclosed for creating a background model of a scene using both a pixel based approach and a context based approach. The combined approach provides an effective technique for segmenting scene foreground from background in frames of a video stream. Further, this approach can scale to process large numbers of camera feeds simultaneously, e.g., using parallel processing architectures, while still generating an accurate background model. Further, using both a pixel based approach and context based approach ensures that the video analytics system can effectively and efficiently respond to changes in a scene, without overly increasing computational complexity. In addition, techniques are disclosed for updating the background model, from frame-to-frame, by absorbing foreground pixels into the background model via an absorption window, and dynamically updating background/foreground thresholds.

Claims

1. A computer-implemented method for updating a background model associated with a scene depicted in a sequence of video frames captured by a video camera, the method comprising:

receiving image data for a current video frame, wherein the image data classifies each pixel in the current video frame as depicting either foreground or background; and

for each pixel in the current video frame classified as depicting scene background, updating corresponding pixel data in the background model based on one or more color channel values of the pixel in the current video frame, wherein the pixel data includes (i) a distribution modeling each color channel value based on observed color channel values for the pixel in the sequence of video frames and (ii) a mean and a variance associated with the distribution for each color channel and wherein updating the corresponding pixel data in the background model based on the one or more color channel values of the pixel in the current video frame comprises, for each of the one or more color channels;

determining a difference between the color channel value in the current video frame and the mean for the color channel,

updating the mean for the color channel based on the difference and a user-specified weight, and

updating the variance for the color channel based on the difference and the weight.

2. The method of claim 1, wherein the distribution for each of the one or more color channels predicts an expected value that color channel when the pixel depicts an element of scene background in a video frame.

3. The method of claim 1, wherein each pixel in the current video frame is initially classified as depicting either foreground or background based on an evaluation comparing color channel values of each pixel with corresponding color channel values in the background model and a context based approach which selectively reclassifies one or more of the pixels initially classified as depicting foreground or background based on the classification of other pixels in the video as depicting either foreground or background.

4. The method of claim 1, further comprising, for each pixel in the current video frame classified as depicting scene foreground, updating corresponding pixel data in the background model based on one or more color channel values of the pixel in the current video frame and an absorption factor determined independently for each pixel in the current video frame that is classified as depicting scene foreground, wherein the absorption factor is based, in part, on a frequency at which the pixel is classified as depicting scene foreground over a background window.

5. A computer-readable storage medium containing a program, which when executed on a processor, performs an operation for updating a background model associated with a scene depicted in a sequence of video frames captured by a video camera, the operation comprising:

receiving image data for a current video frame, wherein the image data classifies each pixel in the current video frame as depicting either foreground or background; and

for each pixel in the current video frame classified as depicting scene background, updating corresponding pixel data in the background model based on one or more color channel values of the pixel in the current video frame, wherein the pixel data includes (i) a distribution modeling each color channel value based on observed color channel values for the pixel in the sequence of video frames and (ii) a mean and a variance associated with the distribution for each color channel and wherein updating the corresponding pixel data in the background model based on the one or more color channel values of the pixel in the current video frame comprises, for each of the one or more color channels;

determining a difference between the color channel value in the current video frame and the mean for the color channel,

updating the mean for the color channel based on the difference and a user-specified weight, and

updating the variance for the color channel based on the difference and the weight.

6. The computer-readable storage medium of claim 5, wherein the distribution for each of the one or more color channels predicts an expected value that color channel when the pixel depicts an element of scene background in a video frame.

7. The computer-readable storage medium of claim 5, wherein each pixel in the current video frame is initially classified as depicting either foreground or background based on an evaluation comparing color channel values of each pixel with corresponding color channel values in the background model and a context based approach which selectively reclassifies one or more of the pixels initially classified as depicting foreground or background based on the classification of other pixels in the video as depicting either foreground or background.

8. The computer-readable storage medium of claim 5, wherein the operation further comprises, for each pixel in the current video frame classified as depicting scene foreground, updating corresponding pixel data in the background model based on one or more color channel values of the pixel in the current video frame and an absorption factor determined independently for each pixel in the current video frame that is classified as depicting scene foreground, wherein the absorption factor is based, in part, on a frequency at which the pixel is classified as depicting scene foreground over a background window.

9. A system, comprising:

a video input source configured to provide a sequence of video frames, each depicting a scene;

a central processing unit (CPU);

a graphics processing unit (GPU); and

a memory containing a program, which, when executed on the CPU and GPU is configured to perform an operation for updating a background model associated with a scene depicted in a sequence of video frames captured by a video camera, the operation comprising;

receiving image data for a current video frame, wherein the image data classifies each pixel in the current video frame as depicting either foreground or background; and

for each pixel in the current video frame classified as depicting scene background, updating corresponding pixel data in the background model based on one or more color channel values of the pixel in the current video frame, wherein the pixel data includes a distribution modeling each color channel value based on observed color channel values for the pixel in the sequence of video frames and (ii) a mean and a variance associated with the distribution for each color channel and wherein updating the corresponding pixel data in the background model based on the one or more color channel values of the pixel in the current video frame comprises, for each of the one or more color channels;

determining a difference between the color channel value in the current video frame and the mean for the color channel;

updating the mean for the color channel based on the difference and a user-specified weight; and

updating the variance for the color channel based on the difference and the weight.

10. The system of claim 9, wherein the distribution for each of the one or more color channels predicts an expected value that color channel when the pixel depicts an element of scene background in a video frame.

11. The system of claim 9, wherein each pixel in the current video frame is initially classified as depicting either foreground or background based on an evaluation comparing color channel values of each pixel with corresponding color channel values in the background model and a context based approach which selectively reclassifies one or more of the pixels initially classified as depicting foreground or background based on the classification of other pixels in the video as depicting either foreground or background.

12. The system of claim 9, wherein the operation further comprises, for each pixel in the current video frame classified as depicting scene foreground, updating corresponding pixel data in the background model based on one or more color channel values of the pixel in the current video frame and an absorption factor determined independently for each pixel in the current video frame that is classified as depicting scene foreground, wherein the absorption factor is based, in part, on a frequency at which the pixel is classified as depicting scene foreground over a background window.