Video surveillance system employing video primitives

• US 10,026,285 B2
• Filed: 02/16/2016
• Issued: 07/17/2018
• Est. Priority Date: 10/24/2000
• Status: Expired due to Fees

• Current Assignee: Avigilon Fortress Corporation
• Original Assignee: Avigilon Fortress Corporation
• Inventors: Venetianer, Peter L., Lipton, Alan J., Hu, Yongtong, Martone, Andrew J., Yin, Weihong, Yu, Li, Zhang, Zhong
• Primary Examiner(s): HALLENBECK-HUBER, JEREMIAH CHARLES
• Application Number:
  US15/044,902
• Publication Number:
  US 20160275766A1
• Time in Patent Office: 882 Days
• Field of Search: 348143, 348148-150, 348166, 348169, 348170, 37524002, 37524008
• CPC Class Codes: G06F 16/7837 using objects detected or recognised in the video content
• CPC Class Codes: G06F 16/7847 using low-level visual features of the video content
• CPC Class Codes: G06F 16/7854 using shape G06F16/7837 takes precedence
• CPC Class Codes: G06F 16/786 using motion, e.g. object motion or camera motion
• CPC Class Codes: G06V 20/52 Surveillance or monitoring of activities, e.g. for recognising suspicious objects recognising microscopic objects G06V20/69
• CPC Class Codes: G06V 20/53 Recognition of crowd images, e.g. recognition of crowd congestion
• CPC Class Codes: G08B 13/196 using television cameras
• CPC Class Codes: G08B 13/19604 involving reference image or background adaptation with time to compensate for changing conditions, e.g. reference image update on detection of light level change
• CPC Class Codes: G08B 13/19615 wherein said pattern is defined by the user
• CPC Class Codes: G08B 13/19697 Arrangements wherein non-video detectors generate an alarm themselves
• CPC Class Codes: G08B 21/0476 Cameras to detect unsafe condition, e.g. video cameras
• CPC Class Codes: G08B 29/188 Data fusion; cooperative systems, e.g. voting among different detectors
• CPC Class Codes: G08B 31/00 Predictive alarm systems characterised by extrapolation or other computation using updated historic data
• CPC Class Codes: H04N 7/18 Closed-circuit television [CCTV] systems, i.e. systems in which the video signal is not broadcast

Abstract

A video surveillance system is set up, calibrated, tasked, and operated. The system extracts video primitives and extracts event occurrences from the video primitives using event discriminators. The system can undertake a response, such as an alarm, based on extracted event occurrences.

Claims

1. A method of video surveillance, comprising:

receiving, by a computer system, a video comprising video images from a video sensor,

the computer system performing the steps of;

analyzing the video to detect stationary objects in the video using background subtraction of a background model;

analyzing the video to detect people in the video by using frame differencing to detect foreground objects;

upon detecting a first stationary object in the video, defining a zone as a portion of the video around the first stationary object, defining the zone being responsive to a location of the first stationary object, the zone being defined to be larger than an outer boundary of the first stationary object and smaller than a field of view of the video, a size of the zone allowing detection of multiple non-overlapping objects;

tracking a duration that the first stationary object remains stationary;

determining an occurrence of an external event;

issuing an alert in response to determining that the duration of time the first stationary object has remained stationary exceeds a threshold while no person of interest detected in the video has been inside the zone and the occurrence of the external event; and

determining a crowd density in the video,

wherein the size of the zone is determined dynamically and is responsive to the determined crowd density.

2. The method of claim 1, wherein the person of interest comprises any person detected in the video.

3. The method of claim 1, further comprising detecting a person leaving the first stationary object as the person of interest.

4. The method of claim 1, wherein the size of the zone is determined prior to the detecting of the first stationary object.

5. The method of claim 4, wherein the zone is one of a circle, ellipse or a rectangle.

6. The method of claim 4, wherein the size of the zone is selected responsive to a selection by a user.

7. The method of claim 1, wherein the person of interest comprises a person detected as leaving the first stationary object and detected as having been in the zone longer than a first period of time.

8. The method of claim 1, wherein the zone has a size defined in an image space that varies as a function of the location of the first stationary object.

9. The method of claim 1, wherein the zone is defined as a shape in the real-world of the video independent of detecting the first stationary object.

10. The method of claim 1, wherein the size of the zone is determined by performing an interpolation or an extrapolation of a first shape.

11. The method of claim 10, wherein the first shape is input by a user.

12. The method of claim 1, further comprising:

tracking a person responsible for leaving behind the first stationary object.

13. The method of claim 1, further comprising permitting a user to set, via a user interface, at least one parameter selected from the group consisting of:

a size of the zone, a shape of the zone, and the duration of time.

14. A method of video surveillance, comprising:

receiving, by a computer system, a video comprising video images from a video sensor,

the computer system performing the steps of;

analyzing the video to detect stationary objects in the video using background subtraction of a background model;

analyzing the video to detect people in the video by using frame differencing to detect foreground objects;

upon detecting a first stationary object in the video, defining a zone as a portion of the video around the first stationary object, defining the zone being responsive to a location of the first stationary object, the zone being defined to be larger than an outer boundary of the first stationary object and smaller than a field of view of the video, a size of the zone allowing detection of multiple non-overlapping objects;

tracking a duration that the first stationary object remains stationary;

determining an occurrence of an external event;

issuing an alert in response to the determining of the occurrence of the external event and determining that the duration of time the first stationary object has remained stationary exceeds a threshold while at least one of the following occurs;

no person of interest detected in the video has been detected as being in the zone, and

no person of interest detected in the video has been detected as being in the zone longer than a first period of time; and

determining a crowd density in the video,

wherein the size of the zone is determined dynamically and is responsive to the determined crowd density.

15. The method of claim 14, further comprising permitting a user to set, via a user interface, at least one parameter selected from the group consisting of:

a size of the zone, a shape of the zone, and the duration of time.

16. A method of video surveillance, comprising:

receiving, by a computer system, a video comprising video images from a video sensor,

the computer system performing the steps of;

analyzing the video to detect stationary objects in the video using background subtraction of a background model;

analyzing the video to detect people in the video by using frame differencing to detect foreground objects;

upon detecting a first stationary object in the video, defining a zone as a portion of the video around the first stationary object, defining the zone being responsive to a location of the first stationary object, the zone being defined to be larger than an outer boundary of the first stationary object and smaller than a field of view of the video, a size of the zone allowing detection of multiple non-overlapping objects;

tracking a duration that the first stationary object remains stationary;

determining an occurrence of an external event;

issuing an alert in response to determining that the duration of time the first stationary object has remained stationary exceeds a threshold while no person of interest detected in the video has been inside the zone and the occurrence of the external event; and

determining a security threat level,

wherein the size of the zone is responsive to the determined security threat level.

17. The method of claim 16, further comprising permitting a user to set, via a user interface, at least one parameter selected from the group consisting of:

a size of the zone, a shape of the zone, and the duration of time.

18. A method of video surveillance, comprising:

receiving, by a computer system, a video comprising video images from a video sensor,

the computer system performing the steps of;

analyzing the video to detect stationary objects in the video using background subtraction of a background model;

analyzing the video to detect people in the video by using frame differencing to detect foreground objects;

upon detecting a first stationary object in the video, defining a zone as a portion of the video around the first stationary object, defining the zone being responsive to a location of the first stationary object, the zone being defined to be larger than an outer boundary of the first stationary object and smaller than a field of view of the video, a size of the zone allowing detection of multiple non-overlapping objects;

tracking a duration that the first stationary object remains stationary;

determining an occurrence of an external event; and

issuing an alert in response to determining that the duration of time the first stationary object has remained stationary exceeds a threshold while no person of interest detected in the video has been inside the zone and the occurrence of the external event,

wherein the size of the zone is responsive to detection of other stationary objects prior to the detecting of the first stationary object.

19. The method of claim 18, further comprising:

analyzing the video to determine distances between other stationary objects and people putting down a corresponding one of the other stationary objects prior to the detecting of the first stationary object,

wherein the size of the zone is responsive to the analyzing of the video to determine distances between the other stationary objects and the people putting down the corresponding one of the other stationary objects.

20. The method of claim 18, further comprising:

analyzing the video to determine durations of time people put down other objects prior to the detecting of the first stationary object,

wherein the duration of time is determined in response to the analyzing of the video to determine the durations of time people put down the other objects.

21. The method of claim 18, further comprising permitting a user to set, via a user interface, at least one parameter selected from the group consisting of:

a size of the zone, a shape of the zone, and the duration of time.

22. A method of video surveillance, comprising:

receiving, by a computer system, a video comprising video images from a video sensor,

the computer system performing the steps of;

analyzing the video to detect stationary objects in the video using background subtraction of a background model;

analyzing the video to detect people in the video by using frame differencing to detect foreground objects;

upon detecting a first stationary object in the video, defining a zone as a portion of the video around the first stationary object, defining the zone being responsive to a location of the first stationary object, the zone being defined to be larger than an outer boundary of the first stationary object and smaller than a field of view of the video, a size of the zone allowing detection of multiple non-overlapping objects;

tracking a duration that the first stationary object remains stationary;

determining an occurrence of an external event;

issuing an alert in response to the determining of the occurrence of the external event and determining that the duration of time the first stationary object has remained stationary exceeds a threshold while, during a first period of time, a person of interest detected in the video has been detected as being in the zone only for part of the first period of time; and

determining a crowd density in the video,

wherein the size of the zone is determined dynamically and is responsive to the determined crowd density.

23. The method of claim 22, further comprising permitting a user to set, via a user interface, at least one parameter selected from the group consisting of:

a size of the zone, a shape of the zone, and the duration of time.