View handling in video surveillance systems

• US 9,204,107 B2
• Filed: 03/15/2013
• Issued: 12/01/2015
• Est. Priority Date: 09/28/2004
• Status: Active Grant

• Current Assignee: Motorola Solutions, Inc.
• Original Assignee: Avigilon Fortress Corporation (Motorola Solutions, Inc.)
• Inventors: Yin, Weihong, Chosak, Andrew J., Clark, John I. W., Egnal, Geoffrey, Frazier, Matthew F., Haering, Niels, Lipton, Alan J., Madden, Donald G., Mannebach, Michael C., Myers, Gary W., Sfekas, James S., Venetianer, Pter L., Zhang, Zhong
• Primary Examiner(s): LI, TRACY Y
• Application Number:
  US13/838,665
• Publication Number:
  US 20130278764A1
• Time in Patent Office: 991 Days
• Field of Search: 348/143
• US Class Current: 1/1
• CPC Class Codes: G06T 7/11 Region-based segmentation
• CPC Class Codes: G06T 7/20 Analysis of motion motion estimation for coding, decoding, compressing or decompressing digital video signals H04N19/43, H04N19/51
• CPC Class Codes: G06T 7/254 involving subtraction of images
• CPC Class Codes: H04N 7/18 Closed-circuit television [CCTV] systems, i.e. systems in which the video signal is not broadcast
• CPC Class Codes: H04N 7/181 for receiving images from a plurality of remote sources
• CPC Class Codes: H04N 7/188 Capturing isolated or intermittent images triggered by the occurrence of a predetermined event, e.g. an object reaching a predetermined position signal generation from motion picture films H04N5/253

Abstract

A content analysis engine receives video input and performs analysis of the video input to produce one or more gross change primitives. A view engine coupled to the content analysis engine receives the one or more gross change primitives from the content analysis engine and provides view identification information. A rules engine coupled to the view engine receives the view identification information from the view engine and provides one or more rules based on the view identification information. An inference engine performs video analysis based on the one or more rules provided by the rules engine and the one or more gross change primitives.

Claims

1. An apparatus comprising:

a content analysis engine to receive video input and to perform analysis of the video input to produce one or more gross change primitives;

a view engine coupled to the content analysis engine to receive the one or more gross change primitives from the content analysis engine and to provide view identification information;

a rules engine coupled to the view engine to receive the view identification information from the view engine and to provide one or more rules based on the view identification information;

an inference engine to perform video analysis based on the one or more rules provided by the rules engine and the one or more gross change primitives;

memory to store the content analysis engine, the view engine, the rules engine, and the inference engine; and

at least one processor to implement the content analysis engine, the view engine, the rules engine, and the inference engine.

2. The apparatus of claim 1, wherein the content analysis engine comprises:

a gross change detector to analyze the video input and to determine if a gross change event has occurred.

3. The apparatus of claim 2, wherein the gross change detector performs:

determining one or more bad frames in the video input;

computing a number of consecutively determined bad frames; and

generating a gross change event if the number of consecutively determined bad frames is greater than a threshold.

4. The apparatus of claim 2, wherein the gross change detector performs:

determining one or more bad frames in the video input;

tracking consecutive bad frames using a histogram having classifications of bad frames;

generating a gross change event if a time duration of a number of consecutive bad frames in the histogram is greater than a threshold; and

detecting a classification of the gross change event using the histogram of consecutive bad frames.

5. The apparatus of claim 4, wherein the bad frames are tracked in the histogram based on classifications of the bad frames, wherein the classifications of the bad frames comprise at least one of:

a light-on bad frame, wherein the light-on bad frame indicates that a mean of the bad frame exceeds a mean of a reference frame by an amount;

a light-off bad frame, wherein the light-off bad frame indicates that the mean of the bad frame is less than the mean of the reference frame by the amount;

or

a camera-motion bad frame, wherein the camera-motion bad frame indicates that a similarity between the bad frame and the reference frame is lower than a threshold.

6. The apparatus of claim 2, wherein the gross change detector is implemented as a state machine using the at least one processor.

7. The apparatus of claim 6, wherein the state machine of the gross change detector includes at least one of the following events:

a good frame detected event;

a bad frame detected event;

a gross change motion event;

a gross change due to lighting event;

or a gross change due to a camera not moving event;

wherein at least one event of the state machine results in a state change and one or more actions of the state machine.

8. The apparatus of claim 2, wherein if a gross change event has occurred, the content analysis engine rebuilds a background model of a scene from the video input.

9. The apparatus of claim 2, wherein if a gross change event has occurred, the content analysis engine:

enters a warm-up state,

does not produce gross change primitives while in the warm-up state, and

after the warm-up state is finished, begins to perform analysis of the video input to produce one or more gross change primitives after the warm-up state is finished.

10. The apparatus of claim 1, wherein the content analysis engine comprises:

a gross change detector to operate on the video input and to determine whether the video input includes one or more bad frames.

11. The apparatus of claim 10, wherein to determine whether a frame of the video input is a bad frame, the gross change detector performs:

computing a total number of pixels in a foreground of a frame of the video input; and

determining the frame to be a bad frame if the total number of pixels in the foreground of the frame is greater than a threshold.

12. The apparatus of claim 10, wherein to determine whether a frame of the video input is a bad frame, the gross change detector performs:

applying an edge detection algorithm to the frame to obtain a frame edge image;

applying the edge detection algorithm to a reference image if there is no background model or to a mean of a background model to obtain a reference edge image;

generating a binary edge mask for the frame edge image;

generating a binary edge mask for the reference edge image;

collapsing the binary edge mask for the frame edge image into a row vector and a column vector;

collapsing the binary edge mask for the reference edge image into a row vector and a column vector;

computing a row correlation between the two row vectors;

computing a column correlation between the two column vectors; and

determining the frame to be a bad frame by comparing the row correlation and the column correlation to a threshold.

13. The apparatus of claim 10, wherein gross change detector classifies the one or more bad frames into classifications, wherein the classifications include at least one of:

a light-on bad frame, wherein the light-on bad frame indicates that a mean of the bad frame exceeds a mean of a reference frame by an amount;

a light-off bad frame, wherein the light-off bad frame indicates that the mean of the bad frame is less than the mean of the reference frame by the amount;

or

a camera-motion bad frame, wherein the camera-motion bad frame indicates that a similarity between the bad frame and the reference frame is lower than a threshold.

14. The apparatus of claim 1, further comprising:

a response engine coupled to the rules engine to receive a response description from the rules engine, the response engine to perform at least one of;

providing an alert, or

causing an action to occur.

15. A method of video processing using memory and at least one processor, comprising:

receiving video input;

performing analysis of the video input, using the at least one processor, to determine if a gross change event has occurred in the video input and to produce one or more gross change primitives if a gross change event is determined to have occurred in the video input;

generating view identification information using the at least one processor and the one or more gross change primitives;

selecting one or more rules using the at least one processor and the view identification information;

determining a response, using the at least one processor, based on the one or more selected rules and the one or more gross change primitives.

16. The method of claim 15, further comprising:

determining one or more bad frames in the video input;

computing a number of consecutively determined bad frames; and

generating a gross change event if the number of consecutively determined bad frames is greater than a threshold.

17. The method of claim 15, further comprising:

determining one or more bad frames in the video input;

tracking consecutive bad frames using a histogram having classifications of bad frames;

generating a gross change event if a time duration of a number of consecutive bad frames in the histogram is greater than a threshold; and

detecting a classification of the gross change event using the histogram of consecutive bad frames.

18. The method of claim 15, further comprising:

building a background model of a scene from the video input; and

if a gross change event is determined to have occurred in the video input, rebuilding the background model of the scene from the video input.

19. A non-transitory computer-readable medium containing instructions that, when executed by at least one processor, cause the at least one processor to perform at least the following:

receive video input;

perform analysis of the video input, using the at least one processor, to determine whether the video input includes one or more bad frames and to produce one or more gross change primitives using the one or more determined bad frames;

generate view identification information using the at least one processor and the one or more gross change primitives;

select one or more rules using the at least one processor and the view identification information;

determine a response, using the at least one processor, based on the one or more selected rules and the one or more gross change primitives.

20. The non-transitory computer-readable medium of claim 19, further comprising instructions that, when executed by the at least one processor, cause the at least one processor to further perform:

to determine whether a frame of the video input is a bad frame, compute a total number of pixels in a foreground of a frame of the video input; and

determine the frame to be a bad frame if the total number of pixels in the foreground of the frame is greater than a threshold.

21. The apparatus of claim 19, further comprising instructions that, when executed by the at least one processor, cause the at least one processor to further perform:

to determine whether a frame of the video input is a bad frame,

apply an edge detection algorithm to the frame to obtain a frame edge image;

apply the edge detection algorithm to a reference image if there is no background model or to a mean of a background model to obtain a reference edge image;

generate a binary edge mask for the frame edge image;

generate a binary edge mask for the reference edge image;

collapse the binary edge mask for the frame edge image into a row vector and a column vector;

collapse the binary edge mask for the reference edge image into a row vector and a column vector;

compute a row correlation between the two row vectors;

compute a column correlation between the two column vectors; and

determine the frame to be a bad frame by comparing the row correlation and the column correlation to a threshold.