Real-time event processing system with analysis engine using recovery information

• US 6,502,133 B1
• Filed: 03/25/1999
• Issued: 12/31/2002
• Est. Priority Date: 03/25/1999
• Status: Expired due to Term

• Current Assignee: Sound View Innovations, LLC (Sound View Innovation Holdings, LLC)
• Original Assignee: Lucent Technologies, Inc. (Nokia Corporation)
• Inventors: Baulier, Gerald D., Blott, Stephen M., Bohannon, Philip L., Branch, Benson L., Cliff, Jr., Thomas M.
• Primary Examiner(s): Harrell, Robert B.
• Application Number:
  US09/276,221
• Time in Patent Office: 1,377 Days
• Field of Search: 709/200, 709/223, 709/224, 714/100, 714/1, 714/2, 714/15, 714/16, 714/17, 714/18, 714/19, 714/20, 714/21, 714/37, 714/38, 714/48, 714/43
• US Class Current: 709/224
• CPC Class Codes: G06F 11/1471 involving logging of persistent data for recovery
• CPC Class Codes: G06F 11/1474 in transactions G06F16/20 takes precedence
• CPC Class Codes: G06F 11/3476 Data logging G06F11/14, G06F11/2205 take precedence
• CPC Class Codes: G06F 2201/86 Event-based monitoring
• CPC Class Codes: G06F 9/542 Event management; Broadcasting; Multicasting; Notifications
• CPC Class Codes: G06F 9/544 Buffers; Shared memory; Pipes
• CPC Class Codes: G06F 9/547 Remote procedure calls [RPC]; Web services

Abstract

A real-time event processing system (EPS) for processing a sequence of events generated by one or more applications. In an illustrative embodiment, the EPS includes a set of real-time analysis engines (RAEs) operating in parallel, e.g., a set of clusters each including one or more RAEs, and one or more mappers for mapping a given input event to a particular one of the clusters. A main-memory database system is coupled to the RAEs, and the RAEs process events associated with input streams from one or more data sources and deliver output streams to one or more data sinks. The data source and data sinks may be, e.g., network elements, clients, databases, etc. The events are processed in accordance with services implemented in the RAEs, and utilize data stored in a memory portion of the main-memory database system accessible to the RAEs. The data may include, e.g., a subscription table storing subscription information indicating the service or services that should be executed for a given event. The services are generated in a service authoring environment (SAE) in the EPS, using a declarative language. The SAE generates the services in the form of object code components, e.g., dynamically linked libraries, which may be dynamically linked into the RAEs without interrupting event processing. Recovery information regarding a recovery point for a given RAE or set of RAEs in the EPS may be stored in a memory portion of the main-memory database system, and utilized to implement a roll-back of the RAE to the recovery point.

Claims

1. An apparatus for processing events generated by at least one system application, the apparatus comprising:

a processor for executing code to implement at least a portion of at least one real-time analysis engine, wherein the real-time analysis engine processes the events, and wherein associated with the real-time analysis engine in a main-memory database system is recovery information regarding a recovery point for the real-time analysis engine.

2. The apparatus of claim 1 wherein serial tags are assigned to each of the events in accordance with an arrival order of the events.

3. The apparatus of claim 2 wherein the serial tags are assigned in a monotonically increasing sequence in accordance with the arrival order of the events.

4. The apparatus of claim 3 wherein the tags and the corresponding events are stored in an event log.

5. The apparatus of claim 2 wherein the recovery information includes the serial tag for each event for which processing in the real-time analysis engine has reached a commit operation.

6. The apparatus of claim 5 wherein the serial tags for each event for which processing in the real-time analysis engine has reached a commit operation are stored in a memory portion of the main-memory database system accessible to the real-time analysis engine.

7. The apparatus of claim 6 wherein after an event is processed and the results of the processing stored in an external database storage element, the serial tag of the event is stored in a serial tag log associated with that storage element.

8. The apparatus of claim 7 wherein the serial tags stored in the memory portion of the main-memory database system and the serial tags stored in the serial tag log are utilized to implement a roll-back of the real-time analysis engine to the recovery point.

9. The apparatus of claim 1 wherein the real-time analysis engine receives the events in the form of one or more input streams from at least one data source, and delivers information representative of processed events in one or more output streams to at least one data sink.

10. The apparatus of claim 9 wherein at least one of the data source and the data sink comprises at least one of a network element of the system, a client associated with the application, and a database element.

11. The apparatus of claim 1 further including a plurality of processors for executing code to implement the at least one real-time analysis engine.

12. The apparatus of claim 1 wherein the processor executes code to implement at least a portion of the main-memory database system.

13. A method of processing events generated by at least one system application, the method comprising the steps of:

processing the events in at least one real-time analysis engine; and

storing in a main-memory database system associated with the real-time analysis engine recovery information regarding a recovery point for the real-time analysis engine.

14. The method of claim 13 further including the step of assigning serial tags to each of the events in accordance with an arrival order of the events.

15. The method of claim 14 wherein the recovery information includes the serial tag for each event for which processing in the real-time analysis engine has reached a commit operation.

16. The method of claim 15 further including the step of storing the serial tags for each event for which processing in the real-time analysis engine has reached a commit operation in a memory portion of the main-memory database system accessible to the real-time analysis engine.

17. The method of claim 16 further including the step of, after an event is processed and the results of the processing stored in an external database storage element, storing the serial tag of the event in a serial tag log associated with that storage element.

18. The method of claim 17 further including the step of utilizing the serial tags stored in the memory portion of the main-memory database system and the serial tags stored in the serial tag log to implement a roll-back of the real-time analysis engine to a recovery point.

19. The method of claim 14 wherein the assigning step includes assigning the serial tags in a monotonically increasing sequence in accordance with the arrival order of the events.

20. The method of claim 14 further including storing the tags and the corresponding events in an event log.

21. An article of manufacture comprising a machine-readable medium for storing one or more programs for processing events generated by at least one system application, wherein the programs when executed by a processor implement the steps of:

(i) processing the events in at least one real-time analysis engine; and

(ii) storing in a main-memory database system associated with the real-time analysis engine recovery information regarding a recovery point for the real-time analysis engine.