Efficient management of data quality for streaming event data

• US 8,655,825 B2
• Filed: 03/10/2011
• Issued: 02/18/2014
• Est. Priority Date: 03/10/2011
• Status: Active Grant

• Current Assignee: SAP SE
• Original Assignee: SAP AG (SAP SE)
• Inventors: Roesch, Philipp, Weng, Sebastian
• Primary Examiner(s): Chen, Alan
• Assistant Examiner(s): Bharadwaj, Kalpana
• Application Number:
  US13/044,681
• Publication Number:
  US 20120233107A1
• Time in Patent Office: 1,076 Days
• Field of Search: 706/47
• US Class Current: 706/47
• CPC Class Codes: G06N 5/02 Knowledge representation; Symbolic representation
• CPC Class Codes: G06N 5/025 Extracting rules from data

Abstract

Implementations of the present disclosure include methods for managing data quality for event data. In some implementations, methods include receiving, at one or more computing devices, a plurality of events, each event comprising event data and being generated by an event source in response to a real-world activity, processing, using the one or more computing devices, the plurality of events using one or more complex event processing (CEP) rules to generate a complex event, in response to generating the complex event, annotating the complex event with quality information corresponding to each of the plurality of events to provide a quality-enriched event, and publishing the quality-enriched event.

Claims

1. A computer-implemented method for managing data quality for event data, the method comprising:

receiving, at one or more computing devices, a plurality of events, each event comprising event data and being generated by an event source in response to a real-world activity;

processing, using the one or more computing devices, the plurality of events using one or more complex event processing (CEP) rules to generate a complex event, the CEP rules being used to recognize a pattern of events from the plurality of events, the complex event corresponding to the pattern of events;

in response to generating the complex event, annotating the complex event with quality information corresponding to each of the plurality of events to provide a quality-enriched event; and

publishing the quality-enriched event.

2. The method of claim 1, wherein the quality information is determined based on each of the plurality of events.

3. The method of claim 2, wherein the quality information comprises a number of the plurality of events that contribute to the complex event.

4. The method of claim 2, wherein the quality information comprises one or more timestamps, each of the one or more timestamps corresponding to an event of the plurality of events.

5. The method of claim 1, further comprising:

determining an event source identifier;

determining event source quality information based on the event source identifier; and

transmitting the event source quality information for use by one or more event-based applications.

6. The method of claim 5, further comprising receiving an event identifier, wherein the event source identifier is based on the event identifier.

7. The method of claim 5, wherein the event source quality information comprises static quality information.

8. The method of claim 7, wherein the event source quality information further comprises quasi-static quality information.

9. The method of claim 5, wherein the event source quality information comprises quasi-static quality information.

10. The method of claim 5, wherein determining event source quality information comprises:

indexing a database using the event source identifier; and

retrieving the event source quality information from the database, the database being provided as a computer-readable storage medium.

11. The method of claim 5, further comprising:

receiving the event source quality information; and

storing the event source quality information to a database, the database being provided as a computer-readable storage medium.

12. The method of claim 11, further comprising subscribing for published quality information, wherein receiving the event source quality information is in response to subscribing and publication of the event source quality information.

13. The method of claim 5, further comprising:

receiving event information, the event information corresponding to an event source associated with the event source identifier; and

updating the event source quality information based on the event information.

14. The method of claim 1, wherein publishing comprises transmitting the quality-enriched event for use by one or more event-based applications.

15. The method of claim 1, further comprising:

comparing a quality value of the quality information to a threshold quality value; and

discarding the quality information when the quality value is less than the threshold quality value.

16. A non-transitory computer-readable storage medium coupled to one or more processors and having instructions stored thereon which, when executed by the one or more processors, cause the one or more processors to perform operations for managing data quality for event data, the operations comprising:

receiving a plurality of events, each event comprising event data and being generated by an event source in response to a real-world activity;

processing the plurality of events using one or more complex event processing (CEP) rules to generate a complex event, the CEP rules being used to recognize a pattern of events from the plurality of events, the complex event corresponding to the pattern of events;

in response to generating the complex event, annotating the complex event with quality information corresponding to each of the plurality of events to provide a quality-enriched event; and

publishing the quality-enriched event.

17. A system, comprising:

a computing device; and

a computer-readable storage device coupled to the computing device and having instructions stored thereon which, when executed by the computing device, cause the computing device to perform operations for managing data quality for event data, the operations comprising;

receiving a plurality of events, each event comprising event data and being generated by an event source in response to a real-world activity;

processing the plurality of events using one or more complex event processing (CEP) rules to generate a complex event, the CEP rules being used to recognize a pattern of events from the plurality of events the complex event corresponding to the pattern of events;

in response to generating the complex event, annotating the complex event with quality information corresponding to each of the plurality of events to provide a quality-enriched event; and

publishing the quality-enriched event.