Sequence event processing using append-only tables

• US 8,131,696 B2
• Filed: 12/13/2006
• Issued: 03/06/2012
• Est. Priority Date: 05/19/2006
• Status: Active Grant

• Current Assignee: Oracle International Corporation (Oracle Corporation)
• Original Assignee: Oracle International Corporation (Oracle Corporation)
• Inventors: Chandrasekaran, Sashikanth
• Primary Examiner(s): HARPER, ELIYAH STONE
• Application Number:
  US11/638,737
• Publication Number:
  US 20070271280A1
• Time in Patent Office: 1,910 Days
• Field of Search: 707/104.1, 707/203, 707/100, 707/10, 707/705, 707/758, 707/769
• US Class Current: 707/705
• CPC Class Codes: G06F 11/28 by checking the correct order of processing G06F11/08 - G06F11/26 take precedence; monitoring patterns of pulse trains H03K5/19
• CPC Class Codes: G06F 9/542 Event management; Broadcasting; Multicasting; Notifications

Abstract

Techniques for processing events are provided. In one embodiment, a plurality of records is received. The plurality of records is generated by one or more computer systems and represents a plurality of events that have occurred in these computer systems. The plurality of events is processed, where processing the plurality of events comprises storing the plurality of records in an append-only sequence. The append-only sequence is a storage representation of the plurality of events that allows only appending new records that represent new events but does not allow modifying and deleting existing records that represent already existing events. One or more expressions are then evaluated based on the plurality of records that are stored in the append-only sequence.

Claims

1. A method for determining policy compliance of a stream of events, the method comprising the computer-implemented steps of:

storing one or more policies that govern the stream of events;

defining and storing a set of base events according to a particular syntax specification, wherein each base event is a definition that specifies one or more conditions for selecting particular events from a stream of events based on one or more attributes included in a stream of records that are generated to respectively represent the stream of events;

receiving the stream of records that respectively represent the stream of events;

selecting a plurality of events from the stream of events by evaluating the one or more conditions specified in each base event of the set of base events over a plurality of attributes included in the stream of records in order to select a plurality of records that respectively represent the plurality of events, wherein the values of the plurality of attributes in the plurality of records satisfy the one or more conditions specified in each base event of the set of base events;

processing the plurality of events, wherein processing the plurality of events comprises;

storing the plurality of records in an append-only sequence on persistent storage; and

forgoing storing, in the append-only sequence on persistent storage, any records that represent those events that have not been selected from the stream of events;

determining whether one or more particular events, from the plurality of events, comply with the one or more policies by performing steps comprising;

retrieving the plurality of records from the append-only sequence;

evaluating one or more expressions based on the plurality of records; and

storing one or more results of evaluating the one or more expressions in one or more of volatile memory and persistent storage;

wherein the steps of the method are performed by one or more computing devices.

2. The method of claim 1, wherein the append-only sequence is a storage representation of the plurality of events that allows only appending new records that represent new events but does not allow modifying and deleting existing records that represent already existing events.

3. The method of claim 2, wherein the plurality of records in the append-only sequence are stored as rows in one or more tables.

4. The method of claim 1, wherein determining whether the one or more particular events comply with the one or more policies further comprises determining that the one or more particular events violate at least one policy of the one or more policies.

5. The method of claim 1, wherein:

the one or more policies are one or more security policies;

the plurality of events are a plurality of security events; and

the plurality of records that represent the plurality of security events are generated as audit records by one or more computer systems.

6. The method of claim 5, wherein the plurality of security events includes at least one of:

creating a user account;

deleting a user account;

a user logging into a particular computer system;

a user logging out of a particular computer system;

changing a password associated a user account;

changing one or more permissions associated with a user account; and

a user accessing of a particular resource in a particular computer system.

7. The method of claim 5, wherein the one or more computer systems include a database system.

8. The method of claim 1, wherein each record of the plurality of records includes a set of attributes that are associated with a particular event, of the plurality of events, that is represented by said each record, wherein the set of attributes includes a timestamp attribute.

9. The method of claim 1, wherein:

the method further comprises defining a set of compound events according to the particular syntax specification, wherein each compound event specifies correlation conditions between multiple base events of the set of base events; and

evaluating the one or more expressions further comprises evaluating one or more compound events, of the set of compound events, that are associated with the one or more expressions.

10. The method of claim 1, wherein:

the method further comprises defining a logical update event according to the particular syntax specification, wherein;

the logical update event is defined based on at least two base events of the set of base events; and

the logical update event indicates first one or more events of the plurality of events that change the effect of second one or more events of the plurality of events, wherein the second one or more events have occurred earlier than the first one or more events; and

evaluating the one or more expressions further comprises evaluating at least one of the one or more expressions based on the logical update event.

11. The method of claim 10, wherein storing the plurality of records in the append-only sequence further comprises:

creating and maintaining a base event index that uniquely identifies each record of the plurality of records; and

creating and maintaining, for the logical update event, a modification list of index values from the base event index, wherein the modification list of index values identifies all records of the plurality of records that represent the first one or more events and the second one or more events.

12. The method of claim 1, wherein:

the one or more expressions comprise a snapshot query that needs to be evaluated as of a particular point in time in the past; and

the method further comprises;

selecting, from the append-only sequence, a set of records that have occurred prior to the particular point in time; and

evaluating the snapshot query over the selected set of records.

13. A computer-readable volatile or non-volatile medium storing one or more sequences of instructions for determining policy compliance of a stream of events, which instructions, when executed by one or more processors, cause performance of steps comprising:

storing one or more policies that govern the stream of events;

defining and storing a set of base events according to a particular syntax specification, wherein each base event is a definition that specifies one or more conditions for selecting particular events from a stream of events based on one or more attributes included in a stream of records that are generated to respectively represent the stream of events;

receiving the stream of records that respectively represent the stream of events;

selecting a plurality of events from the stream of events by evaluating the one or more conditions specified in each base event of the set of base events over a plurality of attributes included in the stream of records in order to select a plurality of records that respectively represent the plurality of events, wherein the values of the plurality of attributes in the plurality of records satisfy the one or more conditions specified in each base event of the set of base events;

processing the plurality of events, wherein processing the plurality of events comprises;

storing the plurality of records in an append-only sequence on persistent storage; and

forgoing storing, in the append-only sequence on persistent storage, any records that represent those events that have not been selected from the stream of events;

determining whether one or more particular events, from the plurality of events, comply with the one or more policies by performing steps comprising;

retrieving the plurality of records from the append-only sequence;

evaluating one or more expressions based on the plurality of records; and

storing one or more results of evaluating the one or more expressions in one or more of volatile memory and persistent storage.

14. The computer-readable volatile or non-volatile medium of claim 13, wherein the append-only sequence is a storage representation of the plurality of events that allows only appending new records that represent new events but does not allow modifying and deleting existing records that represent already existing events.

15. The computer-readable volatile or non-volatile medium claim 14, wherein the plurality of records in the append-only sequence are stored as rows in one or more tables.

16. The computer-readable volatile or non-volatile medium of claim 13, wherein the instructions that cause determining whether the one or more particular events comply with the one or more policies further comprise instructions which, when executed by the one or more processors, cause determining that the one or more particular events violate at least one policy of the one or more policies.

17. The computer-readable volatile or non-volatile medium of claim 13, wherein:

the one or more policies are one or more security policies;

the plurality of events are a plurality of security events; and

the plurality of records that represent the plurality of security events are generated as audit records by one or more computer systems.

18. The computer-readable volatile or non-volatile medium of claim 17, wherein the plurality of security events includes at least one of:

creating a user account;

deleting a user account;

a user logging into a particular computer system;

a user logging out of a particular computer system;

changing a password associated a user account;

changing one or more permissions associated with a user account; and

a user accessing of a particular resource in a particular computer system.

19. The computer-readable volatile or non-volatile medium of claim 17, wherein the one or more computer systems include a database system.

20. The computer-readable volatile or non-volatile medium of claim 13, wherein each record of the plurality of records includes a set of attributes that are associated with a particular event, of the plurality of events, that is represented by said each record, wherein the set of attributes includes a timestamp attribute.

21. The computer-readable volatile or non-volatile medium of claim 13, wherein:

the one or more sequences of instructions further comprise instructions which, when executed by the one or more processors, cause defining a set of compound events according to the particular syntax specification, wherein each compound event specifies correlation conditions between multiple base events of the set of base events; and

the instructions that cause evaluating the one or more expressions further comprise instructions which, when executed by the one or more processors, cause evaluating one or more compound events, of the set of compound events, that are associated with the one or more expressions.

22. The computer-readable volatile or non-volatile medium of claim 13, wherein:

the one or more sequences of instructions further comprise instructions which, when executed by the one or more processors, cause defining a logical update event according to the particular syntax specification, wherein;

the logical update event is defined based on at least two base events of the set of base events; and

the logical update event indicates first one or more events of the plurality of events that change the effect of second one or more events of the plurality of events, wherein the second one or more events have occurred earlier than the first one or more events; and

the instructions that cause evaluating the one or more expressions further comprise instructions which, when executed by the one or more processors, cause evaluating at least one of the one or more expressions based on the logical update event.

23. The computer-readable volatile or non-volatile medium of claim 22, wherein the instructions that cause storing the plurality of records in the append-only sequence further comprise instructions which, when executed by the one or more processors, cause:

creating and maintaining a base event index that uniquely identifies each record of the plurality of records; and

creating and maintaining, for the logical update event, a modification list of index values from the base event index, wherein the modification list of index values identifies all records of the plurality of records that represent the first one or more events and the second one or more events.

24. The computer-readable volatile or non-volatile medium of claim 13, wherein:

the one or more expressions comprise a snapshot query that needs to be evaluated as of a particular point in time in the past; and

the one or more sequences of instructions further comprise instructions which, when executed by the one or more processors, cause;

selecting, from the append-only sequence, a set of records that have occurred prior to the particular point in time; and

evaluating the snapshot query over the selected set of records.