Systems and/or methods for interactive exploration of dependencies in streaming data

• US 9,792,259 B2
• Filed: 12/17/2015
• Issued: 10/17/2017
• Est. Priority Date: 12/17/2015
• Status: Active Grant

• Current Assignee: Software AG
• Original Assignee: Software AG
• Inventors: Heinz, Christoph, Krishnamurthy, Rajmohan
• Primary Examiner(s): Channavajjala, Srirama
• Assistant Examiner(s): Le, Jessica N
• Application Number:
  US14/973,031
• Publication Number:
  US 20170177546A1
• Time in Patent Office: 670 Days
• Field of Search: 707102, 707719
• CPC Class Codes: G06F 16/24568 Data stream processing; Continuous queries
• CPC Class Codes: G06F 17/18 for evaluating statistical data , e.g. average values, frequency distributions, probability functions, regression analysis forecasting specially adapted for a specific administrative, business or logistic context G06Q10/04
• CPC Class Codes: G06F 3/0482 Interaction with lists of selectable items, e.g. menus
• CPC Class Codes: G06F 3/04842 Selection of displayed objects or displayed text elements G06F3/0482 takes precedence
• CPC Class Codes: G06F 30/20 Design optimisation, verification or simulation optimisation, verification or simulation of circuit designs G06F30/30
• CPC Class Codes: G06N 20/00 Machine learning
• CPC Class Codes: G06T 11/206 Drawing of charts or graphs

Abstract

Certain example embodiments allow for the continuous discovery and analysis of dependencies in a multidimensional event stream in which various regression models are continuously modified in real time and regression models are switched, if necessary or desirable. The event streams in certain example embodiments are continuously arriving Big Data streams of heterogeneous types, and the discovery and analysis is done dynamically in real time.

Claims

1. An information processing system, comprising:

an interface configured to receive at least one selected stream of events from an event bus that receives a plurality of streams of events from a plurality of different devices, each event in each stream in the plurality of streams having at least one attribute;

a model store, backed by a non-transitory computer readable storage medium; and

processing resources including at least one processor and a memory coupled thereto, the processing resources being configured to at least;

receive first user input identifying the at least one selected stream;

receive second user input identifying at least two attributes of interest, the attributes of interest being selected from the attributes of the events in the at least one selected stream;

execute a continuous query on the at least one selected stream to attempt to, directly or indirectly, mathematically compute a statistical model including the at least two identified attributes of interest, the statistical model having a type;

store a representation of the mathematically computed statistical model to the model store;

generate for output to the user a user-interactive display including a representation of the mathematically computed statistical model, the user-interactive display being dynamically changeable based upon at least (a) an update to the mathematically computed statistical model that results from the execution of the continuous query, and (b) receipt of third user input that identifies a sub-stream of the at least one selected stream;

determine whether the mathematically computed statistical model sufficiently fits for the at least two identified attributes of interest;

in response to a determination that the mathematically computed statistical model does not fit for the at least two identified attributes of interest, change the type of the statistical model to a new type so that the new type of statistical model better fits for the at least two identified attributes of interest; and

receive user input indicative of a replay request and a past time period associated with the replay request.

2. The system of claim 1, wherein the statistical model at least initially is a linear regression model.

3. The system of claim 2, wherein the new type of statistical model is a non-linear regression model.

4. The system of claim 1, wherein the determination of whether the mathematically computed statistical model sufficiently fits for the at least two identified attributes of interest is based on a comparison of the correlation coefficient for the mathematically computed statistical model with a predefined threshold.

5. The system of claim 1, wherein the processing resources are further configured to at least store the representation of the mathematically computed statistical model to the model store at a predetermined interval and/or upon an occurrence of a predetermined event that has been automatically detected by the processing resources.

6. The system of claim 1, wherein the processing resources are further configured to at least:

retrieve a historical model from the model store in response to a user selection, the retrieved historical model having an associated original temporal resolution; and

apply the retrieved historical model with respect to the at least two identified attributes of interest at a temporal resolution that differs from the original temporal resolution associated with the retrieved historical model.

7. The system of claim 1, wherein the processing resources are further configured to at least:

retrieve a historical model from the model store in response to a user selection, the retrieved historical model having an associated original temporal resolution; and

apply the retrieved historical model against data currently streaming in from the at least one selected stream.

8. The system of claim 1, wherein the first user input identifies a plurality of selected streams.

9. The system of claim 8, wherein the second user input identifies at least first and second attributes of interest, the first attribute of interest being selected from the attributes of the events in one of the plurality of selected streams and the second attribute of interest being selected from the attributes of the events in another one of the plurality of selected streams.

10. The system of claim 1, wherein the user-interactive display further includes a dynamically updatable visual representation of a relationship between one or more of the at least two identified attributes of interest and at least one other, different attribute of the events in the at least one selected stream.

11. The system of claim 10, wherein the processing resources are further configured to at least:

automatically detect whether a highly correlated relationship exists between one or more of the at least two identified attributes of interest and each of the other, different attribute of the events in the at least one selected stream; and

cause the user-interactive display to highlight each automatically detected highly correlated relationship.

12. The system of claim 1, wherein the processing resources are further configured to at least:

automatically detect sub-streams of the at least one selected stream of potential interest; and

enable the user to view such automatically detected sub-streams.

13. The system of claim 1, wherein the processing resources are further configured to at least execute the continuous query, or a different continuous query, on a selected sub-stream to attempt to, directly or indirectly, mathematically compute another statistical model including the at least two identified attributes of interest within the selected sub-stream.

14. The system of claim 1, wherein the processing resources are further configured to at least:

retrieve one or more historical models from the model store in response to the receipt of the user input indicative of the replay request, the one or more retrieved historical models having been mathematically computed in a time period that accords with the past time period associated with the replay request; and

generate for output a replay visually demonstrating how the one or more retrieved historical models developed over the past time period associated with the replay request.

15. An non-transitory computer readable storage medium including instructions that, when executed by at least one hardware processor of a computer system, performs functionality comprising:

using processing resources including at least one processor and a memory,

receiving, over an electronic interface to an event bus that receives a plurality of streams of events from a plurality of different devices, at least one selected stream of events, each event in each stream in the plurality of streams having at least one attribute;

receiving first user input identifying the at least one selected stream;

receiving second user input identifying at least two attributes of interest, the attributes of interest being selected from the attributes of the events in the at least one selected stream;

executing a continuous query on the at least one selected stream to attempt to, directly or indirectly, mathematically compute a statistical model including the at least two identified attributes of interest, the statistical model having a type;

storing a representation of the mathematically computed statistical model to a model store that is backed by a non-transitory computer readable storage medium;

generating, for output to the user, a user-interactive display including a representation of the mathematically computed statistical model, the user-interactive display being dynamically changeable based upon at least (a) an update to the mathematically computed statistical model that results from the execution of the continuous query, and (b) receipt of third user input that identifies a sub-stream of the at least one selected stream;

determining whether the mathematically computed statistical model sufficiently fits for the at least two identified attributes of interest;

in response to a determination that the mathematically computed statistical model does not fit for the at least two identified attributes of interest, changing the type of the statistical model to a new type so that the new type of statistical model better fits for the at least two identified attributes of interest; and

receive user input indicative of a replay request and a past time period associated with the replay request.

16. The non-transitory computer readable storage medium of claim 15, wherein the determination of whether the mathematically computed statistical model sufficiently fits for the at least two identified attributes of interest is based on a comparison of the correlation coefficient for the mathematically computed statistical model with a predefined threshold.

17. The non-transitory computer readable storage medium of claim 15, wherein the representation of the mathematically computed statistical model is stored to the model store at a predetermined interval and/or upon an occurrence of a predetermined event that has been automatically detected by the processing resources.

18. The non-transitory computer readable storage medium of claim 15, wherein further instructions are executable to at least:

retrieve a historical model from the model store in response to a user selection, the retrieved historical model having an associated original temporal resolution; and

apply the retrieved historical model (a) with respect to the at least two identified attributes of interest at a temporal resolution that differs from the original temporal resolution associated with the retrieved historical model and/or (b) against data currently streaming in from the at least one selected stream.

19. The non-transitory computer readable storage medium of claim 15, wherein the second user input identifies at least first and second attributes of interest, the first attribute of interest being selected from the attributes of the events in one of a plurality of selected streams and the second attribute of interest being selected from the attributes of the events in another one of the plurality of selected streams.

20. The non-transitory computer readable storage medium of claim 15, wherein the user-interactive display further includes a dynamically updatable visual representation of a relationship between one or more of the at least two identified attributes of interest and at least one other, different attribute of the events in the at least one selected stream.

21. The non-transitory computer readable storage medium of claim 20, wherein further instructions are executable to at least:

automatically detect whether a highly correlated relationship exists between one or more of the at least two identified attributes of interest and each of the other, different attribute of the events in the at least one selected stream; and

cause the user-interactive display to highlight each automatically detected highly correlated relationship.

22. The non-transitory computer readable storage medium of claim 15, wherein further instructions are executable to at least:

automatically detect sub-streams of the at least one selected stream of potential interest; and

enable the user to view such automatically detected sub-streams.

23. The non-transitory computer readable storage medium of claim 15, wherein further instructions are executable to at least execute the continuous query, or a different continuous query, on a selected sub-stream to attempt to, directly or indirectly, mathematically compute another statistical model including the at least two identified attributes of interest within the selected sub-stream.

24. The non-transitory computer readable storage medium of claim 15, wherein further instructions are executable to at least:

retrieve one or more historical models from the model store in response to the receipt of the user input indicative of the replay request, the one or more retrieved historical models having been mathematically computed in a time period that accords with the past time period associated with the replay request; and

generate for output a replay visually demonstrating how the one or more retrieved historical models developed over the past time period associated with the replay request.

25. An information processing method, comprising:

using processing resources including at least one processor and a memory,

receiving, over an electronic interface to an event bus that receives a plurality of streams of events from a plurality of different devices, at least one selected stream of events, each event in each stream in the plurality of streams having at least one attribute;

receiving first user input identifying the at least one selected stream;

receiving second user input identifying at least two attributes of interest, the attributes of interest being selected from the attributes of the events in the at least one selected stream;

executing a continuous query on the at least one selected stream to attempt to, directly or indirectly, mathematically compute a statistical model for the at least two identified attributes of interest, the statistical model having a type;

storing a representation of the mathematically computed statistical model to a model store that is backed by a non-transitory computer readable storage medium;

generating, for output to the user, a user-interactive display including a representation of the mathematically computed statistical model, the user-interactive display being dynamically changeable based upon at least (a) an update to the mathematically computed statistical model that results from the execution of the continuous query, and (b) receipt of third user input that identifies a sub-stream of the at least one selected stream;

determining whether the mathematically computed statistical model sufficiently fits for the at least two identified attributes of interest;

in response to a determination that the mathematically computed statistical model does not fit for the at least two identified attributes of interest, automatically changing the type of the statistical model to a new type so that the new type of statistical model better fits for the at least two identified attributes of interest; and

receiving user input indicative of a replay request and a past time period associated with the replay request.

26. The method of claim 25, further comprising:

retrieving a historical model from the model store in response to a user selection, the retrieved historical model having an associated original temporal resolution; and

applying the retrieved historical model (a) with respect to the at least two identified attributes of interest at a temporal resolution that differs from the original temporal resolution associated with the retrieved historical model and/or (b) against data currently streaming in from the at least one selected stream.

27. The method of claim 25, wherein the second user input identifies at least first and second attributes of interest, the first attribute of interest being selected from the attributes of the events in one of a plurality of selected streams and the second attribute of interest being selected from the attributes of the events in another one of the plurality of selected streams.

28. The method of claim 25, wherein the user-interactive display further includes a dynamically updatable visual representation of a relationship between one or more of the at least two identified attributes of interest and at least one other, different attribute of the events in the at least one selected stream and/or across different streams.

29. The method of claim 25, further comprising:

automatically detecting sub-streams of the at least one selected stream of potential interest;

enabling the user to view such automatically detected sub-streams; and

executing the continuous query, or a different continuous query, on a selected sub-stream to attempt to, directly or indirectly, mathematically compute another statistical model including the at least two identified attributes of interest within the selected sub-stream.

30. The method of claim 25, further comprising:

retrieving one or more historical models from the model store in response to the receipt of the user input indicative of the replay request, the one or more retrieved historical models having been mathematically computed in a time period that accords with the past time period associated with the replay request; and

generating for output a replay visually demonstrating how the one or more retrieved historical models developed over the past time period associated with the replay request.