SYSTEMS AND/OR METHODS FOR EVENT STREAM DEVIATION DETECTION

• US 20120158360A1
• Filed: 12/17/2010
• Published: 06/21/2012
• Est. Priority Date: 12/17/2010
• Status: Active Grant

• Current Assignee: Software AG
• Original Assignee: Software AG
• Inventors: CAMMERT, Michael, Heinz, Christoph, Riemenschneider, Tobias, Krmer, Jrgen
• Granted Patent: US 9,659,063 B2
• Time in Patent Office: Days
• US Class Current: 702/179
• CPC Class Codes: G06F 16/24568 Data stream processing; Continuous queries
• CPC Class Codes: G06F 16/2462 Approximate or statistical queries
• CPC Class Codes: G06F 16/2474 Sequence data queries, e.g. querying versioned data
• CPC Class Codes: G06Q 10/20 Administration of product repair or maintenance

Abstract

Certain example embodiments described herein relate to systems and/or methods for event stream deviation detection. More particularly, certain example embodiments described herein relate to maintaining short and long-term statistics of an incoming stream of event data. In certain example embodiments, a deviation is calculated based at least in part on the long-term and short-term statistics. The deviation may then be compared to a threshold value. In certain example embodiments, the estimations required for the statistics are done with Kernel Density Estimators (KDEs).

Claims

1. A deviation detection method for use with a processing system including at least one processor, the method comprising:

(a) receiving at least one stream of event data at the processing system, the event data including at least one attribute;

(b) updating, via the at least one processor, a long-term statistic corresponding to a first estimate of a probability density function (PDF) of at least one monitored attribute in the at least one stream of event data over a first time window;

(c) updating, via the at least one processor, a short-term statistic corresponding to a second estimate of the PDF of the at least one monitored attribute in the at least one stream of event data over a second time window, the second time window being of a shorter duration than the first time window;

(d) computing, via the at least one processor, first and second distances between an ideal density distribution and the long- and short-term statistics, respectively;

(e) computing, via the at least one processor, a current deviation based at least in part on the first and second distances;

(f) comparing the current deviation to a threshold value; and

(g) repeating (a)-(f) as further monitored events are provided by the at least one stream of event data.

2. The method of claim 1, wherein the ideal density distribution of the stream of event data is based on multiple ideal values.

3. The method of claim 1, wherein the short-term statistic is a short-term Kernel Density Estimator (KDE) and the long-term statistic is a long-term KDE.

4. The method of claim 1, wherein the second time window spans substantially the entirety of the at least one stream of event data.

5. The method of claim 1, wherein the threshold value is specified by a user.

6. The method of claim 1, wherein the threshold value is dynamically determined by the processing system.

7. The method of claim 1, wherein the calculating of the long-term statistic includes using Cluster Kernels.

8. The method of claim 1, further comprising displaying a result of the comparing of the current deviation to the threshold value on a display in communication with the processing system.

9. The method of claim 1, further comprising outputting a result of the comparing of the current deviation to the threshold value to a data store or external application.

10. The method of claim 1, further comprising issuing a notification when the current deviation exceeds the threshold value.

20. A non-transitory computer readable storage medium tangibly storing instructions that, when processed by at least one processor, executing a method according to claim 1.

11. A deviation detection method for use with a processing system including at least one processor, the method comprising:

receiving at least one stream of event data at the processing system;

maintaining, over a first time period, a short-term kernel density estimator (KDE) for at least one monitored event in the at least one stream of event data;

maintaining, over a second time period, a long-term KDE for the at least one monitored event in the at least one stream of event data;

calculating a deviation from at least one predefined probability density function (PDF) in dependence on the short- and long-term KDEs; and

comparing the deviation to a threshold to detect a deviation in the at least one stream of event data.

12. A deviation detection system for event processing, comprising:

an adapter configured to receive at least one stream of event data;

at least one processor configured to;

calculate a long-term statistic corresponding to a first estimate of a probability density function (PDF) of at least one monitored attribute in the at least one stream of event data over a first time window;

calculate a short-term statistic corresponding to a second estimate of the PDF of the at least one monitored attribute in the at least one stream of event data over a second time window, the second time window being of a shorter duration than the first time window;

compute first and second distances between an ideal density distribution and the long- and short-term statistics, respectively;

compute a current deviation based at least in part on the first and second distances;

compare the current deviation to a threshold value; and

repeat the calculating, computing, and comparing as further monitored events are received in the at least one stream of event data.

13. The system of claim 12, wherein the short-term statistic is a short-term Kernel Density Estimator (KDE) and the long-term statistic is a long-term KDE.

14. The system of claim 12, wherein the second time window spans substantially the entirety of the at least one stream of event data.

15. The system of claim 12, wherein the threshold value is specified by a user.

16. The system of claim 12, wherein the threshold value is dynamically determined by the processing system.

17. The system of claim 12, wherein the calculating of the long-term statistic includes using cCuster Kernels.

18. The system of claim 12, further comprising a user interface configured to display a result of the comparing of the current deviation to the threshold.

19. The system of claim 12, further comprising a data store configured to log information concerning detected deviations.