Stationary target detection by exploiting changes in background model

• US 8,401,229 B2
• Filed: 09/04/2008
• Issued: 03/19/2013
• Est. Priority Date: 09/04/2007
• Status: Active Grant

• Current Assignee: Motorola Solutions, Inc.
• Original Assignee: ObjectVideo Incorporated (Alarm.com Holdings, Inc.)
• Inventors: Hassan-Shafique, Khurram, Venetianer, Peter L., Yu, Li, Liu, Haiying
• Primary Examiner(s): Kassa, Yosef
• Application Number:
  US12/204,562
• Publication Number:
  US 20090060278A1
• Time in Patent Office: 1,657 Days
• Field of Search: 382/103, 382/209, 382/219, 382/278, 348/50, 348/575, 348/586
• US Class Current: 382/103
• CPC Class Codes: G06F 18/00 Pattern recognition
• CPC Class Codes: G06F 18/24 Classification techniques
• CPC Class Codes: G06V 20/52 Surveillance or monitoring of activities, e.g. for recognising suspicious objects recognising microscopic objects G06V20/69
• CPC Class Codes: G08B 13/19602 Image analysis to detect motion of the intruder, e.g. by frame subtraction

Abstract

A sequence of video frames of an area of interest is obtained. A first background model of the area of interest is constructed based on a first parameter. A second background model of the area of interest is constructed based on a second parameter, the second parameter being different from the first parameter. A difference between the first and second background models is determined. A stationary target is determined based on the determined difference. An alert concerning the stationary target is generated.

Claims

1. A computer-implemented method for image processing, comprising:

obtaining, in a computer, a sequence of video frames of an area of interest from a video camera;

constructing, in the computer, a first background model of the area of interest based on a first parameter;

constructing, in the computer, a second background model of the area of interest based on a second parameter, the second parameter being different from the first parameter;

determining, in the computer, a difference between the first and second background models;

determining, in the computer, one or more stationary targets based on the determined difference; and

generating, in the computer, one or more alerts concerning the one or more stationary targets.

2. The method as set forth in claim 1, wherein constructing the first background model of the area of interest based on the first parameter includes:

updating the first background model of the area of interest at a first learning rate; and

wherein

constructing the second background model of the area of interest based on the second parameter includes;

updating the second background model of the area of interest at a second learning rate, the second learning rate being slower than the first learning rate.

3. The method as set forth in claim 1, wherein constructing the first background model of the area of interest based on the first parameter includes:

updating the first background model of the area of interest in one or more frames; and

wherein

constructing the second background model of the area of interest based on the second parameter includes;

updating the second background model of the area of interest at a variable update rate based on the first background model.

4. The method as set forth in claim 3, further including:

(a) initializing the second background model by copying values of pixels of the first background model into values of corresponding pixels of the second background model;

(b) comparing values of a plurality of pixels of the first background model of an incoming video frame with values of corresponding pixels of the second background model of a previous video frame;

(c) based on comparing the values of the pixels, determining matching and non-matching pixels;

(d) copying the values of matching pixels of the first background model into the values of the corresponding matching pixels of the second background model;

(e) incrementing an age counter for at least one non-matching pixel of the second background model;

(f) comparing the age counter to a predetermined age counter threshold; and

(g) based on comparing the age counter, one of copying the value of the non-matching pixel of the first background model into the value of the corresponding pixel of the second background model or repeating (e)-(f).

5. The method as set forth in claim 1, wherein determining the stationary target includes:

generating one or more blobs based on the determined difference;

classifying the one or more generated blobs into one or more targets;

filtering the one or more blobs; and

identifying the stationary target.

6. The method as set forth in claim 5, wherein classifying the generated blobs into targets includes:

superimposing one or more generated blobs on the first background model;

computing first gradients of the first background model within the one or more superimposed blobs;

superimposing one or more generated blobs on the second background model;

computing second gradients of the second background model within the one or more superimposed blobs;

computing one or more correlations of the first and second gradients within the one or more superimposed blobs;

comparing the computed one or more correlations to a first threshold; and

based on comparing to the first threshold, discarding one or more blobs which correspond to a local change.

7. The method as set forth in claim 6, further including:

computing strengths of the first gradients at a boundary of one of the blobs superimposed on the first background model;

computing strengths of the second gradients at a boundary of one of the blobs superimposed on the second background model;

determining a ratio of the first gradient strengths to the second gradient strengths;

comparing the ratio to a second threshold; and

based on comparing to the second threshold, one of classifying the blob as a target insertion or a target removal.

8. The method as set forth in claim 7, further including:

initializing a list of targets in a first video frame, the list including a history of one or more associated targets;

comparing one or more generated blobs in incoming video frames to the targets on the list;

determining a matching or a non-matching blob;

for the matching blob, increasing a counter in the associated target history;

comparing the counter to a third threshold;

based on comparing to the third threshold, confirming the blob as the insertion target;

comparing the counter to a fourth threshold;

based on comparing to the fourth threshold, one of confirming the blob as the stationary target or discarding the blob as representing the local change.

9. A non-transitory computer-readable medium comprising software, which software, when executed by a computer system, causes the computer system to perform operations for detecting stationary targets in a video sequence, the computer-readable medium comprising:

instructions for receiving, in the computer system, a sequence of video frames of an area of interest from a video camera;

instructions for constructing, in the computer system, a first background model of the area of interest based on a first parameter;

instructions for constructing, in the computer system, a second background model of the area of interest based on a second parameter, the second parameter being different from the first parameter;

instructions for determining, in the computer system, a difference between the first and second background models;

instructions for determining, in the computer system, one or more stationary targets based on the determined difference; and

instructions for generating, in the computer system, one or more alerts concerning the one or more stationary targets.

10. The non-transitory medium as set forth in claim 9, further including:

instructions for updating the first background model of the area of interest at a first learning rate; and

instructions for updating the second background model of the area of interest at a second learning rate, the second learning rate being slower than the first learning rate.

11. The non-transitory medium as set forth in claim 9, further including:

instructions for updating the first background model of the area of interest in one or more frames; and

instructions for updating the second background model of the area of interest at a variable update rate.

12. The non-transitory medium as set forth in claim 11, further including:

(a) instructions for initializing the second background model by copying values of pixels of the first background model into values of corresponding pixels of the second background model;

(b) instructions for comparing values of a plurality of pixels of the first background model of an incoming video frame with values of corresponding pixels of the second background model of a previous video frame;

(c) instructions for determining matching and non-matching pixels based on the comparison;

(d) instructions for copying the values of matching pixels of the first background model into the values of the corresponding matching pixels of the second background model;

(e) instructions for incrementing an age counter for at least one non-matching pixel of the second background model;

(f) instructions for comparing the age counter to a predetermined age counter threshold; and

(g) instructions for one of copying the value of the non-matching pixel of the first background model into the value of the corresponding pixel of the second background model or repeating (e)-(f) based on the comparison of (f).

13. The non-transitory medium as set forth in claim 9, further including:

instructions for generating one or more blobs based on the determined difference;

instructions for classifying the one or more generated blobs into one or more targets;

instructions for filtering the one or more blobs; and

instructions for identifying the stationary target.

14. The non-transitory medium as set forth in claim 13, further including:

instructions for superimposing one or more generated blobs on the first background model;

instructions for computing first gradients of the first background model within the one or more superimposed blobs;

instructions for superimposing one or more generated blobs on the second background model;

instructions for computing second gradients of the second background model within the one or more superimposed blobs;

instructions for computing one or more correlations of the first and the second gradients within the one or more superimposed blobs;

instructions for comparing the computed one or more correlations to a first threshold; and

instructions for discarding one or more blobs which correspond to a local change based on comparing to the first threshold.

15. The non-transitory medium as set forth in claim 14, further including:

instructions for computing strengths of the first gradients at a boundary of one of the blobs superimposed on the first background model;

instructions for computing strengths of the second gradients at a boundary of one of the blobs superimposed on the second background model;

instructions for determining a ratio of the first gradient strengths to the second gradient strengths;

instructions for comparing the ratio to a second threshold; and

instructions for one of classifying the blob as a target insertion or a target removal based on comparing to the second threshold.

16. The non-transitory medium as set forth in claim 15, further including:

instructions for initializing a list of targets in a first video frame, the list including a history of one or more associated targets;

instructions for comparing one or more generated blobs in incoming video frames to the targets on the list;

instructions for determining a matching or a non-matching blob;

instructions for increasing a counter in the associated target history for the matching blob;

instructions for comparing the counter with a third threshold;

instructions for confirming the blob as the insertion target based on comparing to the third threshold;

instructions for comparing the counter with a fourth threshold;

instructions for one of confirming the blob as the stationary target or discarding the blob as the local change based on comparing to the fourth threshold.

17. A video processing system, comprising:

a background model engine to receive first, second, . . . , nth sequential video frames of an area of interest and construct first and second updatable background models, the background models being updated based on a corresponding first or second update parameters, which first and second parameters are pre-specified to differ from one another so that the constructed first and second background models are different from one another;

a change detecting engine to compare pairs of corresponding pixels in the first and second background models and determine a difference between the first and second background models;

a blob generating engine to generate one or more blobs based on the determined difference;

a blob classifying engine to determine one or more stationary targets in the area of interest based on the one or more generated blobs;

an alert generating engine to generate one or more alerts regarding the one or more stationary targets; and

one or more processors to execute the background model engine, the change detecting engine, the blob generating engine, the blob classifying engine, and the alert generating engine.

18. The system as set forth in claim 17, wherein the first parameter includes a first learning rate to update the first background model and the second parameter includes a second learning rate to update the second background model, the second learning rate being slower than the first learning rate.

19. The system as set forth in claim 17, wherein the first background model is updated in one or more frames and the second background model is updated based on the first background model with a variable update rate.

20. The system as set forth in claim 17, wherein the blob classifying engine includes:

a gradient analysis engine to determine one or more gradient correlations of gradients within one or more blobs superimposed on the corresponding first or second background model and a ratio of gradient strengths of the one or more blobs at a boundary of the corresponding first or second background model; and

a blob classifier to compare the one or more gradient correlations to a first threshold and, based on the comparison to the first threshold, classify the one or more blobs as representing a local change and to compare the ratio to a second threshold and classify the one or more blobs as one of representing a target removal or a target insertion based on the comparison to the second threshold.

21. The system as set forth in claim 17, further including:

a filter to filter one or more classified targets to verify one or more blobs representing the one or more stationary targets and filter out one or more blobs representing a local change.

22. An application-specific hardware to perform a method comprising:

receiving, in the application specific hardware, a sequence of video frames of an area of interest from a video camera;

constructing, in the application specific hardware, a first background model of the area of interest based on a first parameter;

constructing, in the application specific hardware, a second background model of the area of interest based on a second parameter, the second parameter being different from the first parameter;

determining, in the application specific hardware, a difference between the first and second background models;

determining, in the application specific hardware, one or more stationary targets based on the determined difference; and

generating, in the application specific hardware, one or more alerts concerning the one or more stationary targets.