Scanning camera-based video surveillance system

• US 9,363,487 B2
• Filed: 09/08/2005
• Issued: 06/07/2016
• Est. Priority Date: 09/08/2005
• Status: Active Grant

• Current Assignee: Motorola Solutions, Inc.
• Original Assignee: Avigilon Fortress Corporation (Motorola Solutions, Inc.)
• Inventors: Chosak, Andrew J., Brewer, Paul C., Egnal, Geoffrey, Gupta, Himaanshu, Haering, Niels, Lipton, Alan J., Yu, Li
• Primary Examiner(s): Czekaj, Dave
• Assistant Examiner(s): Beck, Leron
• Application Number:
  US11/220,970
• Publication Number:
  US 20070052803A1
• Time in Patent Office: 3,925 Days
• Field of Search: 348143-169
• US Class Current: 1/1
• CPC Class Codes: G06V 20/52 Surveillance or monitoring of activities, e.g. for recognising suspicious objects recognising microscopic objects G06V20/69
• CPC Class Codes: G08B 13/19608 Tracking movement of a target, e.g. by detecting an object predefined as a target, using target direction and or velocity to predict its new position
• CPC Class Codes: G08B 13/19613 Recognition of a predetermined image pattern or behaviour pattern indicating theft or intrusion
• CPC Class Codes: G08B 13/1963 Arrangements allowing camera rotation to change view, e.g. pivoting camera, pan-tilt and zoom [PTZ]
• CPC Class Codes: G08B 13/19641 Multiple cameras having overlapping views on a single scene
• CPC Class Codes: G08B 13/19656 Network used to communicate with a camera, e.g. WAN, LAN, Internet
• CPC Class Codes: G08B 13/1968 Interfaces for setting up or customising the system
• CPC Class Codes: G08B 13/19682 Graphic User Interface [GUI] presenting system data to the user, e.g. information on a screen helping a user interacting with an alarm system
• CPC Class Codes: G08B 13/19689 Remote control of cameras, e.g. remote orientation or image zooming control for a PTZ camera
• CPC Class Codes: H04N 23/69 Control of means for changing angle of the field of view, e.g. optical zoom objectives or electronic zooming
• CPC Class Codes: H04N 7/181 for receiving images from a plurality of remote sources

Abstract

A video surveillance system may include at least one sensing unit capable of being operated in a scanning mode and a video processing unit coupled to the sensing unit, the video processing unit to receive and process image data from the sensing unit and to detect scene events and target activity.

Claims

1. A video surveillance system comprising:

at least one sensing unit operable in a scanning mode that obtains a first view of a scene, an acquisition mode that obtains a second view of the scene, and an interrogation mode; and

a video processing unit, coupled to the at least one sensing unit, that;

obtains video primitives from tracked and classified targets in the first view of the scene by processing image data received from the at least one sensing unit operating in the scanning mode;

detects a scene event or target activity in the first view of the scene by comparing the video primitives to at least one scene event rule or at least one target activity rule;

upon detection, depending on the scene event or target activity detected, automatically switches said at least one sensing unit from the scanning mode to the acquisition mode, wherein the second view of the scene obtained by the acquisition mode provides a closer view, at a variable level of zoom, of the detected scene event or target activity than said first view obtained by the scanning mode;

upon an occurrence of a return event of the acquisition mode, automatically switches the at least one sensing unit from the acquisition mode to the scanning mode;

upon an occurrence of an interrogation event of the acquisition mode, automatically switches said at least one sensing unit from the acquisition mode to the interrogation mode;

analyzes, using the interrogation mode, the detected scene event or target activity; and

upon an occurrence of a return event of the interrogation mode, automatically switches the at least one sensing unit from the interrogation mode.

2. The video surveillance system according to claim 1, wherein said video processing unit includes:

a vision module that receives video input from said sensing unit;

an inference module that receives output from said vision module; and

a response module that receives output from said inference module.

3. The video surveillance system according to claim 1, wherein:

the at least one sensing unit comprises a plurality of sensing units;

a first sensing unit of the plurality of sensing units is used in the scanning mode;

a second sensing unit of the plurality of sensing units tracks the scene event or the target activity during the acquisition mode and the interrogation mode; and

the first sensing unit is different than in the second sensing unit.

25. The video surveillance system according to claim 1, wherein the scene event and the target activity comprise at least one of crossing a virtual tripwire, appearing, disappearing, loitering, item inserting, or item removing.

26. The video surveillance system of claim 1, wherein the video processing unit simultaneously tracks a plurality of targets in the scene by iteratively:

acquiring each target of the plurality of targets in the acquisition mode, and

interrogating each target of the plurality of targets in the interrogation mode.

27. The video surveillance system of claim 26, wherein the video processing unit periodically switches the at least one sensing unit back and forth between the acquisition mode and the interrogation mode for each of the plurality of targets.

28. The video surveillance system of claim 26, wherein the video processing unit periodically revisits each target of the plurality of targets after the acquiring and the interrogating of each target of the plurality of targets.

29. The video surveillance system of claim 1, wherein:

an improved view includes one of the following;

an angle of view of the detected scene event or target activity,

an image resolution of the detected scene event or target activity, and

a specific part of the detected scene event or target activity;

the return event of the acquisition mode comprises passage of a specified amount of time during which the improved view has not been achieved by the at least one sensing unit; and

the interrogation event of the acquisition mode comprises achieving the improved view by the at least one sensing unit.

30. The video surveillance system of claim 29, wherein the return event of the interrogation mode comprises passage of a certain amount of time during which the at least one sensing unit maintains the improved view by panning, tilting, and/or zooming.

4. A method, to be performed by video processing equipment, of processing video obtained by a sensing unit, comprising:

scanning, by the sensing unit in a scanning mode, a scene for a target of interest;

obtaining, by processing image data recorded by the sensing unit in the scanning mode, video primitives from tracked and classified targets in the scene; and

detecting a scene event or target activity in a first view of the scene by comparing the video primitives to at least one scene event rule or at least one target activity rule;

upon detecting the scene event or target activity, depending on the scene event or target activity detected in the image data, automatically switching the sensing unit to an acquisition mode that obtains a second view of the scene that provides a closer view, at a variable level of zoom, of the detected scene event or target activity than said first view;

upon an occurrence of a return event of the acquisition mode, automatically switching the sensing unit from the acquisition mode to the scanning mode;

upon an occurrence of an interrogation event of the acquisition mode, automatically switching the sensing unit from the acquisition mode to an interrogation mode that further analyzes the detected scene event or target activity; and

upon an occurrence of a return event of the interrogation mode, automatically switching the sensing unit from the interrogation mode.

5. The method according to claim 4, wherein said processing image data recorded by the sensing unit in a scanning mode comprises:

identifying the presence, if any, of a target in the first view; and

generating at least one primitive to describe an action performed by the target.

6. The method according to claim 5, wherein said identifying the presence of a target comprises:

estimating and compensating for motion of said sensing unit.

7. The method according to claim 5, wherein said identifying the presence of a target comprises:

segmenting a video image corresponding to said first view into foreground and background components;

blobizing the foreground components to create one or more blobs; and

identifying one or more possible targets among said one or more blobs.

8. The method according to claim 7, wherein said identifying one or more possible targets comprises:

tracking said one or more possible targets; and

classifying said one or more targets.

9. The method according to claim 5, wherein said processing image data recorded by the sensing unit in a scanning mode includes utilizing at least one scene model to perform said identifying or said generating.

10. The method according to claim 4, wherein in said acquisition mode, the sensing unit performs operations comprising:

estimating scene motion; and

compensating for motion of the sensing unit being operated in the acquisition mode.

11. The method according to claim 4, wherein in said acquisition mode, the sensing unit performs operations comprising:

segmenting a video image corresponding to said first view into foreground and background components;

performing template matching on at least one foreground component; and

generating a primitive based on said template matching.

12. The method according to claim 11, wherein said template matching utilizes an estimate of scene motion.

13. The method according to claim 11, wherein said template matching utilizes at least one detected image feature.

14. The method according to claim 11, wherein said template matching utilizes a target model.

15. The method according to claim 11, further comprising utilizing a scene model to perform at least one activity selected from the group consisting of:

said segmenting a video image; and

said performing template matching; and

said generating a primitive.

16. The method according to claim 15, further comprising:

detecting at least one image feature based on a scene model.

17. The method according to claim 16, wherein said detecting employs a target model.

18. The method according to claim 4, wherein in the interrogation mode, the sensing unit performs operations comprising:

segmenting a video image corresponding to said first view into foreground and background components;

identifying and tracking the target; and

generating a primitive based on said identifying and tracking.

19. The method according to claim 18, wherein identifying and tracking the target comprises:

performing blobization on said foreground components;

tracking said target based on said blobization; and

classifying said target.

20. The method according to claim 18, wherein identifying and tracking the target comprises:

performing template matching on at least one foreground component; and

wherein said generating a primitive is based on said template matching.

21. The method according to claim 18, wherein said interrogation mode comprises utilizing a scene model to perform at least one activity selected from the group consisting of:

said segmenting a video image;

said identifying and tracking the target; and

said generating a primitive.

24. The method according to claim 4, wherein:

the sensing unit comprises a plurality of sensing units;

a first sensing unit of the plurality of sensing units is used in the scanning mode;

a second sensing unit of the plurality of sensing units tracks the scene event or target activity during the acquisition mode and the interrogation mode; and

the first sensing unit is different than in the second scanning unit.

22. A non-transitory computer-readable storage medium containing instructions that, when executed by a processor, cause video processing equipment to perform operations comprising:

scanning, using a sensing unit in a scanning mode, a scene for a target of interest in a first view;

determining, using image data obtained from the scanning, video primitives from tracked and classified targets in the first view;

comparing the video primitives to at least one scene event rule or at least one target activity rule;

detecting, based on the comparing, a scene event or target activity in the first view;

upon the detecting, automatically switching the sensing unit from the scanning mode to an acquisition mode that obtains a second view of the detected scene event or target activity that is closer than the first view;

upon an occurrence of a return event of the acquisition mode, automatically switching the sensing unit from the acquisition mode to the scanning mode;

upon an occurrence of an interrogation event of the acquisition mode, automatically switching the sensing unit from the acquisition mode to an interrogation mode that further analyzes the detected scene event or target activity; and

upon an occurrence of a return event of the interrogation mode, automatically switching the sensing unit from the interrogation mode.

23. A video surveillance system comprising:

at least one sensing unit that selectively operates in a scanning mode, an acquisition mode, and an interrogation mode;

a computer system coupled to said at least one sensing unit to receive video output from said at least one sensing unit, said computer system including at least one processor, and a computer-readable medium coupled to said at least one processor that executes instructions stored on the computer-readable medium that control the computing system to perform operations comprising;

scanning, using the scanning mode, a scene by following a predefined scan path;

determining, using video recorded by the scanning, video primitives from a plurality of targets in the scene;

comparing the video primitives to at least one scene event rule or at least one target activity rule;

detecting, based on the comparing, a scene event or target activity involving a first target of the plurality of targets;

upon the detecting, automatically switching the at least one sensing unit from the sensing mode to the acquisition mode that obtains a zoomed view of the first target;

upon an occurrence of a return event of the acquisition mode, automatically switching the at least one sensing unit from the acquisition mode to the scanning mode;

upon an occurrence of an interrogation event of the acquisition mode, automatically switching the sensing unit from the acquisition mode to the interrogation mode that further analyzes the detected scene event or target activity; and

upon an occurrence of a return event of the interrogation mode, automatically switching the at least one sensing unit from the interrogation mode.