Intelligent event determination and notification in a surveillance system

• US 20050271250A1
• Filed: 03/15/2005
• Published: 12/08/2005
• Est. Priority Date: 03/16/2004
• Status: Active Grant

• Current Assignee: 3VR Security Incorporated (Identiv, Inc.)
• Original Assignee: 3VR Security Incorporated (Identiv, Inc.)
• Inventors: Russell, Stephen G., Vallone, Robert P., Freeman, J. Andrew, Kirkman, Thomas W., Marinelli, Marcus S.
• Granted Patent: US 7,847,820 B2
• Time in Patent Office: Days
• US Class Current: 382/103
• CPC Class Codes: G06V 20/40 in video content extracting overlay text G06V20/62; video retrieval G06F16/70; processing of video elementary streams in video servers H04N21/234; processing of video elementary streams in video clients H04N21/44
• CPC Class Codes: G06V 20/52 Surveillance or monitoring of activities, e.g. for recognising suspicious objects recognising microscopic objects G06V20/69
• CPC Class Codes: G06V 40/168 Feature extraction; Face representation
• CPC Class Codes: G06V 40/172 Classification, e.g. identification
• CPC Class Codes: H04N 21/44008 involving operations for analysing video streams, e.g. detecting features or characteristics in the video stream arrangements characterised by components specially adapted for monitoring, identification or recognition of video in broadcast systems H04H60/59
• CPC Class Codes: H04N 5/77 between a recording apparatus and a television camera
• CPC Class Codes: H04N 5/781 on disks or drums
• CPC Class Codes: H04N 7/181 for receiving images from a plurality of remote sources

Abstract

A method that determines whether a detected event is a significant event requiring action in a video surveillance system. Determined event attributes and object attributes are analyzed to determine if the potential event should trigger an action by the surveillance system. If the potential event should trigger an action, at least one action is triggered. Actions may include relevant event attribute and object attribute information. Potential event may include events detected by a sensor, or external events communicated by an external system to the surveillance system. Event attributes may include location and type of the potential event. Object attributes may include an identification of an object, and attributes associated with the identified object. If an object cannot be positively identified, object attributes may include potential object identifications or determined group associations of the object.

Claims

1. A method of determining an occurrence of a significant event requiring action in a surveillance system, the method comprising computer-implemented steps of:

detecting a potential event;

determining event attributes about the potential event;

determining object attributes about an object in a video surveillance image associated with the potential event;

analyzing the event attributes and object attributes to determine whether the potential event should trigger an action by the video surveillance system; and

if the potential event should trigger an action by the video surveillance system, then determining at least one action to be triggered by the video surveillance system.

2. The method of claim 1 wherein the step of detecting a potential event comprises:

receiving input from a sensor.

3. The method of claim 2 wherein sensor is a motion detection sensor.

4. The method of claim 1 wherein the step of detecting a potential event comprises:

detecting motion in a video feed from a camera in the surveillance system.

5. The method of claim 1 wherein the step of detecting a potential event comprises:

receiving a potential event indicator, from an external system, by the surveillance system.

6. The method of claim 1, wherein the step of detecting a potential event includes receiving information from a plurality of sources.

7. The method of claim 6, wherein the plurality of sources includes at least two sources that independently detect the potential event.

8. The method of claim 7, wherein the at least two sources include at least two surveillance cameras.

9. The method of claim 1, wherein the potential event is a motion detection event, and the event attributes include location and time of motion detection.

10. The method of claim 1, wherein the step of determining object attributes comprises:

identifying an object in the video surveillance image; and

determining attributes about the identified object.

11. The method of claim 10, wherein the step of identifying an object in the video surveillance image comprises:

comparing a set of identifying information extracted from the image to a plurality of stored sets of identifying information;

calculating a confidence score for each comparison;

for each stored set of identifying information whose comparison confidence score exceeds a predetermined minimum threshold, determining an object associated with the stored set of identifying information; and

identifying a best matching object for the object in the image by determining which object has a greatest number of comparison confidence scores that exceed the predetermined minimum threshold.

12. The method of claim 1, wherein the step of determining object attributes comprises:

identifying a list of known objects that may match the object in the video surveillance image; and

determining common attributes about objects in the list of known objects.

13. The method of claim 12, wherein the step of identifying a list of known objects that may match the object in the video surveillance image comprises:

comparing a set of identifying information extracted from the image to a plurality of stored sets of identifying information;

calculating a confidence score for each comparison;

for each stored set of identifying information whose comparison confidence score exceeds a predetermined minimum threshold, determining an object associated with the stored set of identifying information and including the determined associated object in the list of known objects.

14. The method of claim 1, wherein the step of determining object attributes comprises:

determining a group associated with the object in the video surveillance image; and

determining attributes about the group associated with the identified object.

15. The method of claim 1, wherein the step of determining object attributes comprises:

determining a group not associated with the object in the video surveillance image; and

determining attributes about the determined group; and

determining that the object in the video surveillance image does not have the determined attributes of the determined group.

16. The method of claim 1, wherein the event attributes include a time, and the object attributes include an authorization time range, wherein the step of analyzing comprises:

determining that the potential event should trigger an action by the video surveillance system if the potential event time is not with the object authorization time range.

17. The method of claim 1, wherein the step of determining at least one action comprises analyzing the event attributes and object attributes to determine:

at least one person to notify about the potential event, and

and a method of notification associated with the at least one person.

18. The method of claim 17, wherein the determined method of notification includes sending a multimedia e-mail that includes the surveillance video image associated with the potential event.

19. The method of claim 1, wherein at least one determined action includes at least one determined event attribute.

20. The method of claim 1, wherein at least one determined action includes at least one determined object attribute.

21. The method of claim 1, wherein the step of determining at least one action to be triggered by the video surveillance system further comprises determining at least two actions.

22. The method of claim 1, wherein the detected potential event is a first potential event, additionally comprising the steps of:

determining a second potential event;

determining second event attributes and second object attributes about the second potential event;

analyzing the second event attributes and second object attributes to determine whether the second potential event should trigger an action by the video surveillance system; and

if the second potential event should trigger an action by the video surveillance system, then determining a second action to be triggered by the video surveillance system, wherein the second action is the same as an action determined by the first potential event.

23. A computer-readable medium carrying one or more sequences of instructions which, when executed by one or more processors, causes the one or more processors to perform the method recited in claim 1.

24. A computer-readable medium carrying one or more sequences of instructions which, when executed by one or more processors, causes the one or more processors to perform the method recited in claim 2.

25. A computer-readable medium carrying one or more sequences of instructions which, when executed by one or more processors, causes the one or more processors to perform the method recited in claim 3.

26. A computer-readable medium carrying one or more sequences of instructions which, when executed by one or more processors, causes the one or more processors to perform the method recited in claim 4.

27. A computer-readable medium carrying one or more sequences of instructions which, when executed by one or more processors, causes the one or more processors to perform the method recited in claim 5.

28. A computer-readable medium carrying one or more sequences of instructions which, when executed by one or more processors, causes the one or more processors to perform the method recited in claim 6.

29. A computer-readable medium carrying one or more sequences of instructions which, when executed by one or more processors, causes the one or more processors to perform the method recited in claim 7.

30. A computer-readable medium carrying one or more sequences of instructions which, when executed by one or more processors, causes the one or more processors to perform the method recited in claim 8.

31. A computer-readable medium carrying one or more sequences of instructions which, when executed by one or more processors, causes the one or more processors to perform the method recited in claim 9.

32. A computer-readable medium carrying one or more sequences of instructions which, when executed by one or more processors, causes the one or more processors to perform the method recited in claim 10.

33. A computer-readable medium carrying one or more sequences of instructions which, when executed by one or more processors, causes the one or more processors to perform the method recited in claim 11.

34. A computer-readable medium carrying one or more sequences of instructions which, when executed by one or more processors, causes the one or more processors to perform the method recited in claim 12.

35. A computer-readable medium carrying one or more sequences of instructions which, when executed by one or more processors, causes the one or more processors to perform the method recited in claim 13.

36. A computer-readable medium carrying one or more sequences of instructions which, when executed by one or more processors, causes the one or more processors to perform the method recited in claim 14.

37. A computer-readable medium carrying one or more sequences of instructions which, when executed by one or more processors, causes the one or more processors to perform the method recited in claim 15.

38. A computer-readable medium carrying one or more sequences of instructions which, when executed by one or more processors, causes the one or more processors to perform the method recited in claim 16.

39. A computer-readable medium carrying one or more sequences of instructions which, when executed by one or more processors, causes the one or more processors to perform the method recited in claim 17.

40. A computer-readable medium carrying one or more sequences of instructions which, when executed by one or more processors, causes the one or more processors to perform the method recited in claim 18.

41. A computer-readable medium carrying one or more sequences of instructions which, when executed by one or more processors, causes the one or more processors to perform the method recited in claim 19.

42. A computer-readable medium carrying one or more sequences of instructions which, when executed by one or more processors, causes the one or more processors to perform the method recited in claim 20.

43. A computer-readable medium carrying one or more sequences of instructions which, when executed by one or more processors, causes the one or more processors to perform the method recited in claim 21.

44. A computer-readable medium carrying one or more sequences of instructions which, when executed by one or more processors, causes the one or more processors to perform the method recited in claim 22.

45. A surveillance system that determines an occurrence of a significant event requiring action, the system comprising:

means for detecting a potential event;

means for determining event attributes about the potential event;

means for determining object attributes about an object in a video surveillance image associated with the potential event;

means for analyzing the event attributes and object attributes to determine if the potential event should trigger an action; and

means for determining at least one action.

46. A surveillance system comprising:

at least one video camera;

a database storing object information and surveillance system management information;

a notification terminal;

means for detecting a potential event;

a recognition module for identifying an object in a video image;

an event determination component that determines whether a detected potential event should trigger an action by analyzing attributes of an object in a video image associated with the detected potential event, said object being identified by the recognition module, and event attributes of the potential event, wherein attributes about the object and the potential event are stored in the database; and

an action determination component that determines an alert to send to the notification terminal upon the event determination component determining that a potential event should trigger an action.