Lineage information for streaming event data and event lineage graph structures for visualization

• US 9,542,662 B2
• Filed: 12/30/2010
• Issued: 01/10/2017
• Est. Priority Date: 12/30/2010
• Status: Active Grant

• Current Assignee: SAP SE
• Original Assignee: SAP SE
• Inventors: Roesch, Philipp, Weng, Sebastian
• Primary Examiner(s): BOUTAH, ALINA A
• Application Number:
  US12/982,012
• Publication Number:
  US 20120173747A1
• Time in Patent Office: 2,203 Days
• Field of Search: 709224-225, 709/231
• US Class Current: 1/1
• CPC Class Codes: G06Q 10/063 Operations research, analysis or management

Abstract

Implementations of the present disclosure include methods for providing transparency in streaming event data. In some implementations, methods include receiving a plurality of events, each event comprising event data and being generated by an event source in response to a real-world activity, processing the plurality of events using one or more complex event processing (CEP) rules to generate a complex event, in response to generating the complex event, generating at least one lineage event that comprises lineage information, the lineage information comprising information corresponding to one or more source events, each of the one or more source events contributing to the complex event, and storing the lineage event and the one or more source events in an event archive provided as a computer-readable storage medium.

Claims

1. A computer-implemented method for providing transparency in streaming event data, the method comprising:

receiving, at one or more computing devices, a plurality of events, each event comprising event data and being generated by an event source in response to a real-world activity;

processing, using the one or more computing devices, the plurality of events using one or more complex event processing (CEP) rules to generate a complex event;

in response to generating the complex event, generating at least one lineage event that comprises lineage information, the lineage information comprising respective event information representative of each of one or more source events and respective rule information representative of each of the one or more CEP rules, each of the one or more source events contributing to the complex event, the event information enabling lineage tracking and retrieval of the one or more source events from an event archive, the rule information enabling retrieval of the one or more CEP rules from the event archive and defining a condition of applying an action in response to the complex event;

storing the lineage event and the one or more source events in the event archive, the event archive being provided as a non-transitory computer-readable storage medium;

receiving the lineage event, the complex event and each of the one or more source events at a graph builder, the graph builder provided as a program that is executed using one or more processors;

processing the lineage event, the complex event and each of the one or more source events using the graph builder to generate a set of graphs, the set comprising one or more graphs; and

generating a multi-dimensional visualization based on the set of graphs, the multi-dimensional visualization being navigable based on user input.

2. The method of claim 1, wherein the lineage information comprises one or more of an identification of the complex event, an identification of each of the one or more source events, an identification of a CEP rule of the one or more CEP rules, the CEP rule being a rule that resulted in generation of the complex event.

3. The method of claim 1, wherein the lineage information comprises an identification of a CEP engine that generated the complex event.

4. The method of claim 1, further comprising providing an event bus that is executed using one or more computing devices, the event bus receiving each of the plurality of events and routes the plurality of events based on one or more subscriptions.

5. The method of claim 4, wherein processing is performed using a CEP engine, the CEP engine subscribing to events received by the event bus.

6. The method of claim 5, wherein the CEP engine publishes the complex event to the event bus, the event bus providing the complex event to one or more event-based applications that are each executed using one or more computing devices.

7. The method of claim 1, wherein the event archive subscribes to lineage events and receives the lineage event in response to publication of the lineage event.

8. The method of claim 7, wherein the event archive subscribes to the one or more source events and receives each of the one or more source events in response to publication of the one or more source events.

9. The method of claim 1, further comprising:

receiving a query, the query identifying the complex event;

retrieving the lineage event from the event archive based on the query;

retrieving each of the one or more source events from the event archive based on the lineage information of the lineage event; and

transmitting the one or more source events for display to a user.

10. The method of claim 9, wherein the query is generated by an event-based application and the one or more source events are transmitted to the event-based application for display to the user.

11. The method of claim 1, wherein the graph is provided as an acyclic graph comprising a plurality of nodes and a plurality of edges between nodes, each node corresponding one of the complex event and one of the one or more source events.

12. The method of claim 1, further comprising generating a self-organizing map (SOM) based on the graph, the multi-dimensional visualization being generated based on the SOM.

13. The method of claim 12, wherein the one or more graphs are provided as a directed ordered acyclic graph that is provided as input to the SOM.

14. The method of claim 12, wherein the SOM comprises a node grid that is organized in hyperbolic space.

15. The method of claim 12, further comprising mapping the SOM to a multi-dimensional unit disk to provide a projection of the SOM in multi-dimensional space.

16. The method of claim 15, wherein the multi-dimensional unit disk is provided as a Poincaré

disk.

17. The method of claim 1, further comprising:

receiving the user input; and

animating, in response to the user input, the multi-dimensional visualization to move a first node from a projection center toward a periphery of the multi-dimensional visualization and move a second node toward the projection center.

18. A computer-readable storage medium coupled to one or more processors and having instructions stored thereon which, when executed by the one or more processors, cause the one or more processors to perform operations for providing transparency in streaming event data, the operations comprising:

receiving, at one or more computing devices, a plurality of events, each event comprising event data and being generated by an event source in response to a real-world activity;

processing, using the one or more computing devices, the plurality of events using one or more complex event processing (CEP) rules to generate a complex event;

in response to generating the complex event, generating at least one lineage event that comprises lineage information, the lineage information comprising respective event information representative of each of one or more source events and respective rule information representative of each of the one or more CEP rules, each of the one or more source events contributing to the complex event, the event information enabling lineage tracking and retrieval of the one or more source events from an event archive, the rule information enabling retrieval of the one or more CEP rules from the event archive and defining a condition of applying an action in response to the complex event;

storing the lineage event and the one or more source events in the event archive, the event archive being provided as a non-transitory computer-readable storage medium;

receiving the lineage event, the complex event and each of the one or more source events at a graph builder, the graph builder provided as a program that is executed using one or more processors;

processing the lineage event, the complex event and each of the one or more source events using the graph builder to generate a graph; and

generating a multi-dimensional visualization based on the graph, the multi-dimensional visualization being navigable based on user input.

19. A system, comprising:

a computing device; and

a computer-readable storage device coupled to the computing device and having instructions stored thereon which, when executed by the computing device, cause the computing device to perform operations for providing transparency in streaming event data, the operations comprising;

receiving a plurality of events, each event comprising event data and being generated by an event source in response to a real-world activity;

processing the plurality of events using one or more complex event processing (CEP) rules to generate a complex event;

in response to generating the complex event, generating at least one lineage event that comprises lineage information, the lineage information comprising respective event information representative of each of one or more source events and respective rule information representative of each of the one or more CEP rules, each of the one or more source events contributing to the complex event, the event information enabling retrieval of the one or more source events from an event archive, the rule information enabling lineage tracking and retrieval of the one or more source events from an event archive, the rule information enabling retrieval of the one or more CEP rules from the event archive and defining a condition of applying an action in response to the complex event;

storing the lineage event and the one or more source events in the event archive the event archive being provided as a non-transitory computer-readable storage medium;

receiving the lineage event, the complex event and each of the one or more source events at a graph builder, the graph builder provided as a program that is executed using one or more processors;

processing the lineage event, the complex event and each of the one or more source events using the graph builder to generate a graph; and

generating a multi-dimensional visualization based on the graph, the multi-dimensional visualization being navigable based on user input.