CORRELATION ENGINE FOR SECURITY, SAFETY, AND BUSINESS PRODUCTIVITY

• US 20150319352A1
• Filed: 04/02/2014
• Published: 11/05/2015
• Est. Priority Date: 10/04/2007
• Status: Active Grant

• Current Assignee: SecureNet Solutions Group, LLC
• Original Assignee: KD Secure LLC
• Inventors: Donovan, John J., Hussain, Daniar
• Granted Patent: US 9,344,616 B2
• Time in Patent Office: Days
• US Class Current: 1/1
• CPC Class Codes: G06F 3/0481 based on specific properties of the displayed interaction object or a metaphor-based environment, e.g. interaction with desktop elements like windows or icons, or assisted by a cursor's changing behaviour or appearance
• CPC Class Codes: G06F 3/0605 by facilitating the interaction with a user or administrator
• CPC Class Codes: G06F 3/0649 Lifecycle management
• CPC Class Codes: G06F 3/0685 Hybrid storage combining heterogeneous device types, e.g. hierarchical storage, hybrid arrays
• CPC Class Codes: G08B 13/19645 Multiple cameras, each having view on one of a plurality of scenes, e.g. multiple cameras for multi-room surveillance or for tracking an object by view hand-over
• CPC Class Codes: G08B 13/19656 Network used to communicate with a camera, e.g. WAN, LAN, Internet
• CPC Class Codes: G08B 13/19671 Addition of non-video data, i.e. metadata, to video stream
• CPC Class Codes: G08B 13/19693 using multiple video sources viewed on a single or compound screen
• CPC Class Codes: G08B 13/19697 Arrangements wherein non-video detectors generate an alarm themselves
• CPC Class Codes: H04L 41/0631 using root cause analysis; using analysis of correlation between notifications, alarms or events based on decision criteria, e.g. hierarchy, tree or time analysis
• CPC Class Codes: H04L 41/0654 using network fault recovery ring fault isolation or reconfiguration in loop networks without recovery actions by a network management system H04L12/437
• CPC Class Codes: H04L 67/12 specially adapted for proprietary or special-purpose networking environments, e.g. medical networks, sensor networks, networks in vehicles or remote metering networks
• CPC Class Codes: H04N 23/661 Transmitting camera control signals through networks, e.g. control via the Internet
• CPC Class Codes: H04N 23/80 Camera processing pipelines; Components thereof
• CPC Class Codes: H04N 23/90 Arrangement of cameras or camera modules, e.g. multiple cameras in TV studios or sports stadiums
• CPC Class Codes: H04N 7/181 for receiving images from a plurality of remote sources
• CPC Class Codes: H04N 7/183 for receiving images from a single remote source

Abstract

The present invention is a correlation engine for use in security, safety, and business monitoring applications. Sensory data from one or more sensors are captured and analyzed to detect one or more events in the sensory data. The events are correlated by the correlation engine by weighing the events based on attributes of the sensors that were used to detect the primitive events. The events are then monitored for an occurrence of one or more correlations of interest. Finally, one or more actions are triggered based on a detection of one or more anomalous events or events of interest. Events may come from sensory devices, legacy systems, third-party systems, anonymous tips, and other data sources. The present invention may be used to increase business productivity by improving security, safety, and increasing profitability of business processes.

Claims

1-20. -20. (canceled)

21. A computer-readable, non-transitory data storage medium containing computer-executable program code comprising:

a correlation engine to correlate one or more events received from one or more sensory devices, wherein the events are weighted based on attributes of the sensory devices.

22. The storage medium of claim 21, further comprising program code for:

one or more data analytics engines to process sensory data from one or more sensors to detect one or more primitive events in the sensory data.

23. The storage medium of claim 22, further comprising program code for:

a normalization engine to normalize the primitive events detected in the sensory data into a standardized format before correlating two or more normalized primitive events.

24. The storage medium of claim 21, further comprising program code for:

one or more data analytics engines to process information from one or more legacy systems external to the sensors to detect one or more additional primitive events, comprising primitive legacy events, from one or more legacy systems.

25. The storage medium of claim 21, further comprising program code for:

perform a historical correlation by automatically analyzing the primitive events across time for one or more historical correlations between primitive events that occurred historically.

26. The storage medium of claim 25, further comprising program code for:

a monitoring engine to monitor continuously and in real-time one or more sensory primitive events from the sensors for exceeding a threshold determined from the one or more historical correlations to determine one or more critical events.

27. The storage medium of claim 26, further comprising program code for:

an action engine to trigger one or more actions based on the detected critical event.

28. The storage medium of claim 27, wherein the actions comprising at least an alert to one or more recipients in a company'"'"'s hierarchy.

29. The storage medium of claim 27, further comprising program code for:

a learning engine for automatically learn which critical events to detect.

30. The storage medium of claim 21, further comprising program code for:

a privacy filter to filter out primitive events based on a set of privacy rules, wherein the set of privacy rules are defined by a system administrator and are designed to protect a privacy of individuals where said system is being used.

31. The storage medium of claim 21, further comprising program code for:

a business filter to filter out primitive events based on a set of business rules, wherein the set of business rules are defined by a system administrator and are designed to customize said system to business processes of an organization.

32. The storage medium of claim 21, further comprising program code for:

an event queue for storing events before correlation by the correlation engine.

33. The storage medium of claim 21, wherein the events are primitive events.

34. The storage medium of claim 21, wherein the events are compound events composed of two or more primitive events.

35. The storage medium of claim 21, further comprising program code for:

storing the sensory data and the events corresponding to the sensory data in a data storage device, wherein the events are stored with an index back to the sensory data corresponding to locations of occurrence of the events within the sensory data.

36. The storage medium of claim 21, wherein the attributes are updated based on one or more external events.

37. The storage medium of claim 21, further comprising program code to:

retrieve experience levels of personnel from a personnel system;

correlate the experience levels of personnel with the primitive sensory events; and

generate one or more alerts based on the correlation between the experience levels of personnel and the primitive sensory events.

38. The storage medium of claim 21, further comprising program code to:

retrieve financial system information from the financial system;

process the financial system information to generate one or more primitive financial events;

correlate the primitive financial events with the primitive sensory events; and

generate one or more alerts based on the correlation between the primitive financial events and the primitive sensory events.

39. The storage medium of claim 21, further comprising program code to:

retrieve security information from the security legacy system;

process the security information to generate one or more primitive security events;

correlate the primitive security events with the primitive sensory events; and

generate one or more alerts based on the correlation between the primitive security events and the primitive sensory events.

40. The storage medium of claim 21, further comprising program code to:

perform one or more actions based on a system-wide correlation level exceeding a predetermined threshold,

wherein the system-wide correlation level is determined based on the equation;

41. The storage medium of claim 21, further comprising program code to:

detect compound events composed of two or more primitive events;

correlate one or more compound events with one or more primitive events; and

generate one or more alerts based on the correlation between the compound events and the primitive events.

42. The storage medium of claim 21, further comprising program code to:

generate one or more alerts based on the correlation between events detected now and events that occurred historically.

43. The storage medium of claim 21, further comprising program code to:

correlate events across space for events occurring substantially simultaneously across multiple sensors located in different locations across space; and

generate one or more alerts based on the correlation between events occurring substantially simultaneously across multiple sensors located in different locations across space.

44. The storage medium of claim 21, further comprising program code to:

time correlate the primitive events across time;

space correlate the primitive events across space; and

evaluate one or more rules based on the correlation performed in the time correlating step and the space correlating step; and

generate one or more alerts based at least on the time correlation and the space correlation.

45. The storage medium of claim 21, further comprising program code to:

receive tip data from one or more external sources;

determine attribute data for the tip data, the attribute data representing a reliability of the source of the tip data;

generate tip events based on the tip data and the attribute data;

correlate one or more tip events with one or more events from the sensory data; and

generate one or more alerts based on the correlation between the tip events and the sensory data.

46. The storage medium of claim 21, further comprising program code to:

cascade the sensory data down a data storage hierarchy comprising two or more data storage devices based on an importance of the sensory data,

wherein the importance of the sensory data is based on an attribute selected from the group consisting of a resolution of the sensor used to capture the sensory data, an age of the sensor used to capture the sensory data, a time since last maintenance of the sensor used to capture the sensory data, a location of the sensor used to capture the sensory data, and a reliability of the sensor used to capture the sensory data.

47. The storage medium of claim 21, further comprising program code to:

monitor network status of the one or more sensors;

generate network events reflective of the network status of the one or more sensors;

correlate one or more network events with one or more primitive events from the sensory data; and

generate one or more alerts based on the correlation between the network events and the primitive sensory events.

48. The storage medium of claim 21, further comprising program code to:

capture the attribute data for at least one of the sensors, the attribute data comprising information about the sensors used to capture the sensory data;

correlate the primitive events utilizing weights corresponding to the sensor used to capture the sensory data that generated the primitive events; and

generate one or more alerts based on the correlation performed by weighing the sensory data by the attribute data corresponding to the sensor used to capture the sensory data.

49. The storage medium of claim 21, wherein the attributes comprise a quality of sensory data produced by the sensor.

50. The storage medium of claim 21, wherein the attributes comprise an age of the sensor used to capture the sensory data.

51. The storage medium of claim 21, wherein the attributes comprises a time since the sensor was last maintained.

52. The storage medium of claim 21, wherein the attributes comprise an integrity of the sensor used to capture the sensory data.

53. The storage medium of claim 21, wherein the attributes comprise a reliability of the sensor used to capture the sensory data.

54. The storage medium of claim 21, wherein the attributes comprise reliability of power to the sensor

55. The storage medium of claim 21, wherein the attributes comprise reliability of a transmission and a bandwidth of a communication link to the sensor.

56. The storage medium of claim 21, further comprising program code to:

determine the attributes based on past evidence of usefulness of sensory data from a given sensor.

57. The storage medium of claim 21, further comprising program code to:

determine the attributes based on susceptibility of the corresponding sensor to noise, interference, or overexposure.

58. The storage medium of claim 21, further comprising program code to:

determine the attribute data based on weather conditions around the corresponding sensor.

59. The storage medium of claim 21, further comprising program code to:

determine the attribute data based on a type of the corresponding sensor.

60. The storage medium of claim 27, wherein the one or more actions include an action to reboot a sensor upon detection of a failure of said sensor.

61. The storage medium of claim 21, further comprising:

a network management module for monitoring a network status of the sensors.

62. The storage medium of claim 21, wherein the sensors include at least a video camera.

63. The storage medium of claim 21, wherein the sensors include at least a video camera and a microphone.

64. The storage medium of claim 21, wherein the sensors are selected from the group consisting of

65. The storage medium of claim 21, further comprising program code to:

store events in an events database for later retrieval

66. The storage medium of claim 21, further comprising program code to:

retrieve configuration information from a configuration database.

67. The storage medium of claim 27, further comprising program code to:

store actions performed in an actions database.

68. The storage medium of claim 21, further comprising program code for:

a rules engine to evaluate one or more rules.

69. The storage medium of claim 21, further comprising program code for:

a map module for displaying sensors and corresponding sensor locations on a map.

70. The storage medium of claim 69, wherein the map is selected from the group consisting of a topological map, a physical map, a street map, a satellite map, a network map, and an interior building map.

71. The storage medium of claim 21, further comprising program code for:

processing information from one or more legacy systems to detect one or more legacy events; and

correlating the legacy events with the one or more events form the sensors.

72. The storage medium of claim 71, further comprising program code for:

performing one or more historical correlations by automatically analyzing the one or more events from the sensors and said legacy events across time or space for one or more historical correlations between the events from the sensors and said legacy events.

73. The storage medium of claim 72, further comprising program code for:

monitoring continuously and in real-time sensory data from the sensors for exceeding a threshold determined from the one or more historical correlations to determine one or more critical events.

74. The storage medium of claim 73, further comprising program code for:

initiating one or more actions based on said one or more critical events.

75. A computer-readable and non-transitory storage medium, comprising computer-executable program code, which when executed by a computer processor causes the computer processor to:

process sensory data from one or more sensors to detect one or more primitive events in the sensory data;

perform one or more correlations by automatically analyzing said primitive events across time or space for one or more correlations between said primitive events, wherein said primitive events are weighted based on attributes of the sensors used to capture the sensory data;

monitor continuously and in real-time said sensory data for exceeding a threshold determined from said one or more correlations to determine one or more critical events; and

initiate one or more actions based on said one or more critical events.

76. The storage medium of claim 75, comprising additional program code which when executed causes the processor to:

storing the sensory data and the primitive events corresponding to the sensory data in a data storage device, wherein the primitive events are stored with an index back to the sensory data corresponding to locations of occurrence of the primitive events within the sensory data.

80. The storage medium of claim 75, wherein the actions comprising at least an alert to one or more recipients in a company'"'"'s hierarchy.

81. The storage medium of claim 75, wherein the attribute data are updated based on one or more external events.

82. The storage medium of claim 75, further comprising program code, which when executed causes the processor to:

retrieve experience levels of personnel from a personnel system;

correlate the experience levels of personnel with the primitive sensory events; and

generate one or more alerts based on the correlation between the experience levels of personnel and the primitive sensory events.

83. The storage medium of claim 75, further comprising program code, which when executed causes the processor to:

retrieve financial system information from the financial system;

process the financial system information to generate one or more primitive financial events;

correlate the primitive financial events with the primitive sensory events; and

generate one or more alerts based on the correlation between the primitive financial events and the primitive sensory events.

84. The storage medium of claim 75, further comprising program code, which when executed causes the processor to:

retrieve security information from the security legacy system;

process the security information to generate one or more primitive security events;

correlate the primitive security events with the primitive sensory events; and

generate one or more alerts based on the correlation between the primitive security events and the primitive sensory events.

85. The storage medium of claim 75, further comprising program code, which when executed causes the processor to:

perform one or more actions based on a system-wide correlation level exceeding a predetermined threshold,

wherein the system-wide correlation level is determined based on the equation;

86. The storage medium of claim 75, further comprising program code, which when executed causes the processor to:

detect compound events composed of two or more primitive events;

correlate one or more compound events with one or more primitive events; and

generate one or more alerts based on the correlation between the compound events and the primitive events.

87. The storage medium of claim 75, further comprising program code, which when executed causes the processor to:

generate one or more alerts based on the correlation between events detected now and events that occurred historically.

88. The storage medium of claim 75, further comprising program code, which when executed causes the processor to:

correlate events across space for events occurring substantially simultaneously across multiple sensors located in different locations across space; and

generate one or more alerts based on the correlation between events occurring substantially simultaneously across multiple sensors located in different locations across space.

89. The storage medium of claim 75, further comprising program code, which when executed causes the processor to:

time correlate the primitive events across time;

space correlate the primitive events across space; and

evaluate one or more rules based on the correlation performed in the time correlating step and the space correlating step; and

generate one or more alerts based at least on the time correlation and the space correlation.

90. The storage medium of claim 75, further comprising program code, which when executed causes the processor to:

generate one or more new rules based on the primitive events correlated in the correlating step and the actions performed in the action step.

91. The storage medium of claim 75, further comprising program code, which when executed causes the processor to:

receive tip data from one or more external sources;

determine attribute data for the tip data, the attribute data representing a reliability of the source of the tip data;

generate tip events based on the tip data and the attribute data;

correlate one or more tip events with one or more events from the sensory data; and

generate one or more alerts based on the correlation between the tip events and the sensory data.

92. The storage medium of claim 75, further comprising program code, which when executed causes the processor to:

cascade the sensory data down a data storage hierarchy comprising two or more data storage devices based on an importance of the sensory data,

wherein the importance of the sensory data is based on an attribute selected from the group consisting of a resolution of the sensor used to capture the sensory data, an age of the sensor used to capture the sensory data, a time since last maintenance of the sensor used to capture the sensory data, a location of the sensor used to capture the sensory data, and a reliability of the sensor used to capture the sensory data.

93. The storage medium of claim 75, further comprising program code, which when executed causes the processor to:

monitor network status of the one or more sensors;

generate network events reflective of the network status of the one or more sensors;

correlate one or more network events with one or more primitive events from the sensory data; and

generate one or more alerts based on the correlation between the network events and the primitive sensory events.

94. The storage medium of claim 75, further comprising program code, which when executed causes the processor to:

capture the attribute data for at least one of the sensors, the attribute data comprising information about the sensors used to capture the sensory data;

correlate the primitive events utilizing weights corresponding to the sensor used to capture the sensory data that generated the primitive events; and

generate one or more alerts based on the correlation performed by weighing the sensory data by the attribute data corresponding to the sensor used to capture the sensory data.

95. The storage medium of claim 75, wherein the attribute data comprises a quality of sensory data produced by the sensor.

96. The storage medium of claim 75, wherein the attribute data comprises an age of the sensor used to capture the sensory data.

97. The storage medium of claim 75, wherein the attribute data comprises a time since the sensor was last maintained.

98. The storage medium of claim 75, wherein the attribute data comprises an integrity of the sensor used to capture the sensory data.

99. The storage medium of claim 75, wherein the attribute data comprises a reliability of the sensor used to capture the sensory data.

100. The storage medium of claim 75, wherein the attribute data comprises reliability of power to the sensor

101. The storage medium of claim 75, wherein the attribute data comprises reliability of a transmission and a bandwidth of a communication link to the sensor.

102. The storage medium of claim 75, further comprising program code, which when executed causes the processor to:

determine the attribute data based on past evidence of usefulness of sensory data from a given sensor.

103. The storage medium of claim 75, further comprising program code, which when executed causes the processor to:

determine the attribute data based on susceptibility of the sensor to noise, interference, or overexposure.

104. The storage medium of claim 75, further comprising program code, which when executed causes the processor to:

determine the attribute data based on weather conditions around the sensor.

105. The storage medium of claim 75, further comprising program code, which when executed causes the processor to:

determine the attribute data based on a type of the sensor.

106. The storage medium of claim 75, further comprising program code, which when executed causes the processor to:

normalize the primitive events detected in the sensory data into a standardized format before correlating two or more normalized primitive events.

107. The storage medium of claim 75, further comprising program code, which when executed causes the processor to:

filter out primitive events based on a set of privacy rules, wherein the set of privacy rules are defined by a system administrator and are designed to protect a privacy of individuals where said system is being used.

108. The storage medium of claim 75, further comprising program code, which when executed causes the processor to:

filter out primitive events based on a set of business rules, wherein the set of business rules are defined by a system administrator and are designed to customize said system to business processes of an organization.

109. The storage medium of claim 75, wherein the one or more actions include an action to reboot a sensor upon detection of a failure of said sensor.