Correlation system for correlating sensory events and legacy system events

• US 10,862,744 B2
• Filed: 03/06/2020
• Issued: 12/08/2020
• Est. Priority Date: 10/04/2007
• Status: Active Grant

• Current Assignee: SecureNet Solutions Group, LLC
• Original Assignee: SecureNet Solutions Group, LLC
• Inventors: Donovan, John J, Hussain, Daniar
• Primary Examiner(s): La, Anh V
• Application Number:
  US16/810,838
• Publication Number:
  US 20200259699A1
• Time in Patent Office: 277 Days
• Field of Search: 340540, 340506, 34053925, 340521, 340937, 348143, 348159
• CPC Class Codes: G08B 13/19645 Multiple cameras, each having view on one of a plurality of scenes, e.g. multiple cameras for multi-room surveillance or for tracking an object by view hand-over
• CPC Class Codes: G08B 13/19656 Network used to communicate with a camera, e.g. WAN, LAN, Internet
• CPC Class Codes: G08B 13/19671 Addition of non-video data, i.e. metadata, to video stream
• CPC Class Codes: G08B 13/19693 using multiple video sources viewed on a single or compound screen
• CPC Class Codes: G08B 13/19697 Arrangements wherein non-video detectors generate an alarm themselves
• CPC Class Codes: H04L 41/0631 using root cause analysis; using analysis of correlation between notifications, alarms or events based on decision criteria, e.g. hierarchy, tree or time analysis
• CPC Class Codes: H04L 41/069 using logs of notifications; Post-processing of notifications
• CPC Class Codes: H04L 41/22 comprising specially adapted graphical user interfaces [GUI]
• CPC Class Codes: H04L 43/16 Threshold monitoring
• CPC Class Codes: H04L 65/65 Network streaming protocols, e.g. real-time transport protocol [RTP] or real-time control protocol [RTCP]
• CPC Class Codes: H04L 67/12 specially adapted for proprietary or special-purpose networking environments, e.g. medical networks, sensor networks, networks in vehicles or remote metering networks
• CPC Class Codes: H04N 5/77 between a recording apparatus and a television camera
• CPC Class Codes: H04N 7/181 for receiving images from a plurality of remote sources
• CPC Class Codes: H04W 4/70 Services for machine-to-machine communication [M2M] or machine type communication [MTC]

Abstract

Monitoring systems and methods for use in security, safety, and business process applications utilizing a correlation engine are disclosed. Sensory data from one or more sensors are captured and analyzed to detect one or more events in the sensory data. The events are correlated by a correlation engine, optionally by weighing the events based on attributes of the sensors that were used to detect the primitive events. The events are then monitored for an occurrence of one or more correlations of interest, or one or more critical events of interest. Finally, one or more actions are triggered based on a detection of one or more correlations of interest, one or more anomalous events, or one or more critical events of interest. Events may come from sensory devices, legacy systems, third-party systems, anonymous tips, and other data sources. The present invention may be used to increase business productivity by improving security, safety, and increasing profitability of business processes.

Claims

1. A monitoring system comprising a non-transitory, physical storage medium storing computer-readable program code, the program code executable by a hardware processor, the program code when executed by the hardware processor causes the hardware processor to implement:

a sensory event analytics module to receive sensory data about a physical environment from one or more sensors and to process the sensory data from the one or more sensors to detect one or more sensory events, wherein the one or more sensors comprises at least an Internet Protocol (IP) video camera, and wherein the one or more sensory events is selected from the group consisting of a person detected, a face detected, a vehicle detected, and a license plate detected;

a legacy event analytics module to receive legacy system data from one or more legacy systems and to process the legacy system data from the one or more legacy systems to detect one or more legacy events, wherein the one or more legacy systems is selected from the group consisting of an access control system, a personnel system, a license plate system, an inventory system, a law enforcement system, and a lighting system;

an event queue having access to an event database to store the sensory events and the legacy events for later retrieval as stored sensory events and stored legacy events;

a correlation module to calculate one or more historical correlations by automatically analyzing the stored sensory events and the stored legacy events across at least one of time and space, wherein the correlation module is adapted to monitor continuously and in real-time the sensory events and the legacy events to identify one or more critical events, and wherein the one or more critical events are based at least on the one or more historical correlations among the stored sensory events and the stored legacy events; and

an alerting module to send one or more alerts based on the one or more critical events.

2. The monitoring system of claim 1, further comprising program code to implement:

a network analytics module to receive Internet Protocol (IP) data of the one or more sensors, wherein the IP data comprises at least an IP address and a network status of at least one of the sensors, and wherein the network analytics module is adapted to process the IP data of the one or more sensors to detect one or more network events.

3. The monitoring system of claim 2, further comprising program code to implement:

a normalization module to normalize the sensory events, the legacy events, and the network events into a standardized data format to generate normalized sensory events, normalized legacy events, and normalized network events.

4. The monitoring system of claim 3, further comprising program code to implement:

an event queue to queue the normalized sensory events, the normalized legacy events, and the normalized network events for storage in the event database.

5. The monitoring system of claim 4, wherein the correlation module is adapted to evaluate one or more historical correlations by automatically analyzing the stored sensory events, the stored legacy events, and the stored network events, across at least one of time and space, for one or more historical correlations among the stored sensory events, the stored legacy events, and the stored network events.

6. The monitoring system of claim 5, wherein the correlation module is adapted to monitor continuously and in real-time the normalized sensory events, the normalized legacy events, and the normalized network events to identify the one or more critical events, and wherein the one or more critical events are based at least on the one or more historical correlations among the stored sensory events, the stored legacy events, and the stored network events.

7. The monitoring system of claim 1, wherein the one or more historical correlations are calculated by applying a weighting of relative importance of the stored sensory events based on a quality of the sensory data from the one or more sensors.

8. The monitoring system of claim 1, wherein the one or more historical correlations are calculated by applying a weighting of relative importance of the legacy system data based on a quality of data produced by the one or more legacy systems.

9. The monitoring system of claim 1, wherein the one or more critical events are determined from one or more safety procedures, and wherein the one or more alerts are sent when one or more of the safety procedures are violated.

10. The monitoring system of claim 1, wherein the one or more sensors are further selected from the group consisting of a temperature probe, a pressure probe, an altitude meter, a speedometer, a revolutions per minute (RPM) meter, a blood pressure meter, a heart rate meter, a chlorine meter, a radon meter, a dust particle meter, a pollution meter, a CO₂ meter, a bacteria meter, a water meter, an electrical meter, and combinations thereof.

11. The monitoring system of claim 1, wherein the one or more legacy systems are further selected from the group consisting of a personnel system, a police dispatch system, a currency system, and combinations thereof.

12. The monitoring system of claim 1, wherein the one or more legacy systems further comprise at least a personnel system, and wherein the storage medium further comprises program code, which when executed causes the hardware processor to execute steps comprising:

retrieving experience levels of personnel from the personnel system;

correlating the experience levels of personnel with the sensory events; and

generating one or more alerts based on the correlation between the experience levels of personnel and the sensory events.

13. The monitoring system of claim 1, wherein the storage medium further comprises program code, which when executed causes the hardware processor to execute steps comprising:

performing one or more actions based on a correlation level exceeding a predetermined threshold.

14. The monitoring system of claim 1, wherein the storage medium further comprises program code, which when executed causes the hardware processor to execute steps comprising:

generating one or more new rules based on events correlated and alerts generated.

15. The monitoring system of claim 1, wherein the storage medium further comprises program code, which when executed causes the hardware processor to execute steps comprising:

receiving tip data from one or more external sources;

generating tip events based on the tip data;

correlating one or more tip events with the sensory events; and

generating one or more alerts based on the correlation between the tip events and the sensory events.

16. The monitoring system of claim 1, wherein the sensory events are weighted based at least on one or more attribute data of the one or more sensors used to capture the sensory data.

17. The monitoring system of claim 16, wherein the attribute data comprises a quality of sensory data produced by the sensors.

18. The monitoring system of claim 16, wherein the attribute data comprises an age of the sensors used to capture the sensory data.

19. The monitoring system of claim 16, wherein the attribute data comprises a time since the sensors were last maintained.

20. The monitoring system of claim 16, wherein the attribute data comprises an integrity of the sensors used to capture the sensory data.

21. The monitoring system of claim 16, wherein the attribute data comprises a reliability of the sensors used to capture the sensory data.

22. The monitoring system of claim 16, wherein the attribute data comprises a reliability of power that is powering the sensors.

23. The monitoring system of claim 16, wherein the attribute data comprises a reliability of a transmission and a bandwidth of a communication link to the sensors.

24. The monitoring system of claim 1, wherein the storage medium further comprises program code, which when executed causes the hardware processor to implement steps comprising:

detecting at least one compound event, the at least one compound event comprising one or more sensory events and one or more legacy system events.

25. The monitoring system of claim 1, further comprising program code to implement:

a hierarchical storage manager having access to a hierarchy of two or more data storage devices, wherein the two or more data storage devices are adapted to store data from the one or more sensors.

26. The monitoring system of claim 25, wherein the hierarchical storage manager is adapted to manage storage and cascade of data through the hierarchy of two or more data storage devices based at least on the sensory events.

27. The monitoring system of claim 25, wherein the hierarchical storage manager is adapted to manage storage and cascade of data through the hierarchy of two or more data storage devices based at least on attribute data.

28. The monitoring system of claim 1, further comprising program code to implement:

an IP network to communicate between the sensory event analytics module, the legacy event analytics module, the event database, and the correlation module.

29. A monitoring method executable by a hardware processor, the method comprising:

receiving sensory data about a physical environment from one or more sensors;

processing the sensory data from the one or more sensors to detect one or more sensory events, wherein the one or more sensors comprises at least an Internet Protocol (IP) video camera, and wherein the one or more sensory events is selected from the group consisting of a person detected, a face detected, a vehicle detected, and a license plate detected;

receiving legacy system data from one or more legacy systems;

processing the legacy system data from the one or more legacy systems to detect one or more legacy events, wherein the one or more legacy systems is selected from the group consisting of an access control system, a personnel system, an inventory system, a law enforcement system, and a lighting system;

storing the sensory events and the legacy events for later retrieval as stored sensory events and stored legacy events;

calculating one or more historical correlations by automatically analyzing the stored sensory events and the stored legacy events across at least one of time and space;

monitoring continuously and in real-time the sensory events and the legacy events to identify one or more critical events, wherein the one or more critical events are based at least on the one or more historical correlations among the stored sensory events and the stored legacy events; and

sending one or more alerts based on the one or more critical events.