Method and system to characterize video background changes as abandoned or removed objects

• US 10,235,599 B2
• Filed: 07/03/2018
• Issued: 03/19/2019
• Est. Priority Date: 03/04/2013
• Status: Active Grant

• Current Assignee: Tyco Fire & Security GmbH (Johnson Controls International plc)
• Original Assignee: Sensormatic Electronics LLC (Johnson Controls International plc)
• Inventors: Li, Changchun, Westmacott, Ian C., Ristivojevic, Mirko, Gruenke, Matthew A.
• Primary Examiner(s): Pham, Nam D
• Application Number:
  US16/026,783
• Publication Number:
  US 20180330189A1
• Time in Patent Office: 259 Days
• CPC Class Codes: G06F 18/22 Matching criteria, e.g. proximity measures
• CPC Class Codes: G06T 2207/10016 Video; Image sequence
• CPC Class Codes: G06T 2207/20036 Morphological image processing
• CPC Class Codes: G06T 2207/30232 Surveillance
• CPC Class Codes: G06T 5/00 Image enhancement or restoration
• CPC Class Codes: G06T 7/194 involving foreground-background segmentation
• CPC Class Codes: G06T 7/20 Analysis of motion motion estimation for coding, decoding, compressing or decompressing digital video signals H04N19/43, H04N19/51
• CPC Class Codes: G06T 7/215 Motion-based segmentation
• CPC Class Codes: G06T 7/254 involving subtraction of images
• CPC Class Codes: G06T 7/60 Analysis of geometric attributes
• CPC Class Codes: G06V 20/52 Surveillance or monitoring of activities, e.g. for recognising suspicious objects recognising microscopic objects G06V20/69
• CPC Class Codes: H04N 7/181 for receiving images from a plurality of remote sources

Abstract

A method and system for analyzing video data in a security system. An analysis compares a current frame to a background model. The analysis system compares the background model to the current frame to identify changed pixel patches. The analysis system uses morphological image processing to generate masks based on the changed pixel patches. Next, the analysis system applies the masks to the background model and the current frames to determine whether the changed pixel patches are characteristic of abandoned or removed objects within the video data.

Claims

1. A method for analyzing video data in a security system, the method comprising:

comparing a background model to video data to identify at least one patch of changed pixels between the background model and the video data;

generating masks based on the at least one patch of changed pixels that define contours around the at least one patch of changed pixels; and

applying the masks to the background model and the video data to determine whether the at least one patch of changed pixels is characteristic of an abandoned object or a removed object.

2. The method according to claim 1, further comprising computing a similarity score for the video data as a sum of the square of dot product between a gradient vector computed for a blurred mask and a gradient vector for the video data over border pixels of the patch of changed pixels.

3. The method according to claim 1, further comprising computing a similarity score for the background model as a sum of the square of dot product between a gradient vector computed for a blurred mask and a gradient vector for the background model over border pixels of the patch of changed pixels.

4. The method according to claim 1, further comprising associating metadata with the at least one frame and creating alerts based on the metadata.

5. The method according to claim 1, further comprising performing morphological image processing on the at least one patch of changed pixels.

6. The method according to claim 5, wherein the morphological image processing includes dilations.

7. The method according to claim 5, wherein the morphological image processing includes erosions.

8. The method according to claim 1, further comprising extracting edge pixels in the background model and the video data with an edge detection algorithm.

9. The method according to claim 8, further comprising calculating gradient magnitudes of the extracted edge pixels in the background model and the video data.

10. The method according to claim 9, further comprising summing gradient magnitudes of all edge pixels in the background model and the video data.

11. The method according to claim 1, further comprising blurring the video data and background model to reduce noise and variations.

12. The method according to claim 1, further comprising converting the mask to a high precision representation.

13. The method according to claim 1, further comprising continually updating the background model based determination of abandoned objects and/or removed objects.

14. A method for analyzing video data in a security system, the method comprising:

comparing a background model to the video data to identify at least one patch of changed pixels between the background model and the video data;

summing gradient magnitudes for edge pixels in the background model with respect to the at least one patch of changed pixels;

summing gradient magnitudes for edge pixels in the video data with respect to the at least one patch of changed pixels; and

characterizing objects in the video data as abandoned or removed based on a comparison of the sums of gradient magnitudes between the background model and the video data.

15. The method according to claim 14, further comprising continually updating the background model based determination of the abandoned objects and/or the removed objects.

16. The method according to claim 15, wherein the background model is temporarily stored in a processing pipeline.

17. The method according to claim 14, further comprising associating metadata with the video data and creating alerts based on the metadata.

18. The method according to claim 14, further comprising performing morphological image processing on the at least one patch of changed pixels.

19. The method according to claim 18, wherein the morphological image processing includes dilations.

20. A security system comprising:

video cameras to capture video data;

a video recorder to store the captured video data;

an analysis system that analyzes the video data of the security system; and

wherein the analysis system compares a background model to the video data to identify at least one patch of changed pixels between the background model and the video data, generates masks based on the at least one patch of changed pixels, which define contours around the at least one patch of changed pixels, applies the masks to the background model and the video data, and determines whether the at least one patch of changed pixels is characteristic of abandoned objects or removed objects.