Real time event stream processor to ensure up-to-date and accurate result

• US 20060282695A1
• Filed: 06/09/2005
• Published: 12/14/2006
• Est. Priority Date: 06/09/2005
• Status: Active Grant

• Current Assignee: Microsoft Technology Licensing LLC (Microsoft Corporation)
• Original Assignee: Microsoft Corporation
• Inventors: Mital, Vijay, Chkodrov, Gueorgui B.
• Granted Patent: US 7,512,829 B2
• Time in Patent Office: Days
• US Class Current: 713/502
• CPC Class Codes: G06F 1/14 Time supervision arrangements, e.g. real time clock

Abstract

Identifying a transaction from a real time event stream having latency. A method of the invention receives events from the real time event stream where events define a plurality of transactions to be identified. Each of the transactions includes a first event and a second event; the first event has a first latency relative to the real time event stream and the second event has a second latency relative to the real time event stream. The first event is identified from the received real time event stream, and a time is record when the first event is received. The method determines a time period during which the second event occurs as a function of the first latency, the recorded time of the first event, and the second latency. The transaction having the first and second events is identified based on the determined time period of the second event.

Claims

1. A computerized method for identifying from a real time event stream a transaction including events having latency, said computerized method comprising:

receiving events from the real time event stream wherein the events which define a plurality of transactions to be identified, wherein each of the plurality of transactions includes a first event and a second event, said first event having a first latency relative to the real time event stream and said second event having a second latency relative to the real time event stream;

identifying the first event from the received real time event stream;

recording a time when the first event is received;

determining a time period during which the second event occurs as a function of the first latency, the recorded time of the first event, and the second latency; and

identifying the transaction having the first event and the second event based on the determined time period of the second event.

2. The computerized method of claim 1 further comprising converting a partially ordered event stream to a fully ordered event stream by buffering and sorting.

3. The computerized method of claim 1 further comprising increasing or decreasing the first latency or the second latency in response to an input from a user or wherein the first latency or the second latency is based on one or more of the following:

a pre-determined latency, a latency provided by the real time stream of events, a minimum latency as a function of the real time stream of events, or a latency specified by the user.

4. The computerized method of claim 1 wherein the second event is related to the first event according to the defined transaction.

5. The computerized method of claim 1 further comprising receiving a query from the user for retrieving received events relating to the identified transaction and continuously processing the query with respect to the real time event stream to provide a query result which is independent of latency.

6. The computerized method of claim 5 wherein the query includes query parameters based on the first event, the second event, and the transaction.

7. The computerized method of claim 1 wherein one or more computer-readable media have computer-executable instructions for performing the computerized method of claim 1.

8. A system for identifying from a real time stream of events a transaction including events having latency, said system comprising:

an interface for receiving a plurality of events from the real time stream of events which define a plurality of transactions to be identified, wherein each of the plurality of transaction includes a first event and a second event relating to the first event, said first event having a first latency relative to the real time stream of events and said second event having a second latency relative to the real time stream of events;

a processor associated with the interface for identifying the first event from the plurality of events received by the interface;

a memory for storing data relating to a time when the first event is received, said memory also storing data relating to a first latency and storing data relating to a second latency;

wherein said processor determines a time period during which the second event will occur in the stream of events is to be received by the interface as a function of the stored data relating to the first latency, the recorded time of the first event, and the second latency; and

wherein said processor identifies the defined transaction having the first event and the second event based on the determined time period of the second event.

9. The system of claim 8 wherein the processor converts a partially ordered event stream to a fully ordered event stream by buffering and sorting.

10. The system of claim 8 wherein the processor further increases or decreases the recorded time of the first event in response to the transmitted data from the interface or wherein the processor increases or decreases the first latency and the second latency in response to the transmitted data from the interface and wherein the memory stores data relating to the adjusted first latency and adjusted second latency.

11. The system of claim 9 wherein the data stored in the memory relating the first latency or the second latency is based on one or more of the following:

a pre-determined latency, a latency provided by the real time stream of events, a minimum latency as a function of the real time stream of events, or a latency specified by the user.

12. The system of claim 8 wherein the processor further retrieves the received relating to the identified transaction in response to a query from the user.

13. The system of claim 8 wherein the memory stores data relating to the first event and the second event of the identified transaction.

14. The system of claim 8 wherein the defined transaction includes a third event, and wherein the data relating to the first event and the second event is removed from the memory after the processor identifies the third event of the defined transaction.

15. One or more computer-readable media having one or more computer-executable components for identifying from a real time stream of events a transaction including events having latency, said computer-executable components comprising:

a processing agent component including a plurality of processing nodes for receiving an input real time stream of events relating to a transaction, said transaction including a first event and a second event, said first event having a first latency relative to the real time stream of events and said second event having a second latency relative to the real time stream of events;

an event stream processing coordinator associated with the processing agent component for identifying the first event from the received events from the input real time stream of events, said processing agent component records a watermark indicative of a time when the first event is received;

wherein said event stream processing coordinator determines a time period during which the second event will be found in the stream of events as a function of the first latency, the recorded watermark of the first event, and the second latency; and

wherein the event stream processing coordinator identifies the transaction having the first event and the second event based on the determined time period of the second event.

16. The computer-readable media of claim 15 wherein the event stream processing coordinator further modifies the recorded watermark of the first event in response to the transmitted data from the processing agent component.

17. The computer-readable media of claim 15 wherein the event stream processing coordinator converts a partially ordered event stream to a fully ordered event stream by buffering and sorting.

18. The computer-readable media of claim 15 wherein the event stream processing coordinator increases or decreases the first latency and the second latency in response to the transmitted data from the processing agent component or wherein the first latency or the second latency is based on one or more of the following:

a pre-determined latency, a latency provided by the real time stream of events, a minimum latency as a function of the real time stream of events, or a latency specified by the user.

19. The computer-readable media of claim 15 wherein the defined transaction includes a third event and wherein the processing agent component outputs the first event and the second event to an output event stream in response to identifying the third event by the event processing component.

20. The computer-readable media of claim 15 further comprising an interface component for receiving a query from a user for retrieving the identified transaction, said query includes one or more query parameters based on the first event, the second event, and the transaction.