Techniques for performing regular expression-based pattern matching in data streams

• US 8,589,436 B2
• Filed: 08/26/2009
• Issued: 11/19/2013
• Est. Priority Date: 08/29/2008
• Status: Active Grant

• Current Assignee: Oracle International Corporation (Oracle Corporation)
• Original Assignee: Oracle International Corporation (Oracle Corporation)
• Inventors: Srinivasan, Anand, Jain, Namit, Mishra, Shailendra, Komuravelli, Rakesh
• Primary Examiner(s): Corrielus, Jean M
• Assistant Examiner(s): ASPINWALL, EVAN S
• Application Number:
  US12/548,222
• Publication Number:
  US 20100057736A1
• Time in Patent Office: 1,546 Days
• Field of Search: 707/713, 707/758, 707/759, 707/760, 707/761, 707/762, 707/763, 707/765, 707/769, 707/771, 707/772, 707/779, 707/E17.014, 707/E17.017, 707/E17.061, 707/E17.062, 707/E17.063, 707/E17.069, 707/999.003, 707/780, 719/318
• US Class Current: 707/780
• CPC Class Codes: G06F 18/00 Pattern recognition
• CPC Class Codes: G06F 18/20 Analysing

Abstract

Techniques for detecting patterns in one or more data streams. A pattern to be detected may be specified using a regular expression. Events received in a data stream are processed during runtime to detect occurrences of the specified pattern in the data stream.

Claims

1. A method of detecting a pattern in a data stream comprising events, the method comprising:

receiving, at a processing system, including a processor, predicate information, the predicate information specifying a predicate associated with each symbol in a set of symbols, the predicate information including a first predicate associated with a first symbol in the set of symbols, and including a second predicate associated with a second symbol in the set of symbols;

receiving, at the processing system, a regular expression specifying a pattern, the regular expression comprising one or more symbols from the set of symbols;

constructing an automaton for the pattern;

upon receiving an event in the data stream;

determining, based upon the received event, states of the automaton and associated bindings, and

determining if the pattern is matched due to the received event based upon the states and associated bindings;

in response to determining that the pattern is matched by a particular sequence of events in the data stream, continuing to determine whether subsequent events following the particular sequence of events in the data stream produce a match that is longer than a match produced by the particular sequence of events; and

outputting the match produced by the particular sequence of events only after determining that no match longer than the match produced by the particular sequence of events is possible based on the states and associated bindings.

2. The method of claim 1 further comprising:

upon receiving an event in the data stream, determining one or more symbols from the set of symbols that are matched due to the received event, wherein a symbol is matched if the predicate associated with the symbol is satisfied due to the received event; and

wherein determining the states upon receiving an event in the data stream comprises determining the states based upon the one or more symbols and set of states and associated bindings stored prior to receiving the input event.

3. The method of claim 1 wherein the predicate information specifies a predicate that is associated with the first symbol in the set of symbols and is dependent upon the second symbol included in the set of symbols.

4. The method of claim 1 wherein the regular expression comprises at least one of an alternation operator, a grouping operator, or an aggregation operator.

5. The method of claim 1 wherein at least one symbol in the regular expression has an associated lazy quantifier.

6. The method of claim 1, wherein constructing the automaton for the pattern comprises:

determining whether the pattern satisfies specified restrictions;

selecting, based on whether the pattern satisfies the specified restrictions, and from a plurality of types of automatons, a particular type of automaton; and

constructing an automaton of the particular type of automaton.

7. A computer-readable non-transitory storage medium storing a plurality of instructions for controlling a processor to detect a pattern in a data stream comprising events, the plurality of instructions comprising:

instructions that cause the processor to receive predicate information, the predicate information specifying a predicate associated with each symbol in a set of symbols, the predicate information including a first predicate associated with a first symbol in the set of symbols, and including a second predicate associated with a second symbol in the set of symbols;

instructions that cause the processor to receive a regular expression specifying a pattern, the regular expression comprising one or more symbols from the set of symbols;

instructions that cause the processor to construct an automaton for the pattern; and

instructions that cause the processor to, upon receiving an event in the data stream,

determine, based upon the received event, states of the automaton and associated bindings, and

determine if the pattern is matched due to the received event based upon the states and associated bindings;

instructions that cause the processor to continue to determine, in response to determining that the pattern is matched by a particular sequence of events in the data stream, whether subsequent events following the particular sequence of events in the data stream produce a match that is longer than a match produced by the particular sequence of events; and

instructions that cause the processor to output the match produced by the particular sequence of events only after determining that no match longer than the match produced by the particular sequence of events is possible based on the states and associated bindings.

8. The computer-readable non-transitory storage medium of claim 7 wherein the plurality of instructions further comprises:

instructions that cause the processor to, upon receiving an event in the data stream, determine one or more symbols from the set of symbols that are matched due to the received event, wherein a symbol is matched if the predicate associated with the symbol is satisfied due to the received event; and

wherein the instructions that cause the processor to determine the states upon receiving an event in the data stream comprise instructions that cause the processor to determine the states based upon the one or more symbols and set of states and associated bindings stored prior to receiving the input event.

9. The computer-readable non-transitory storage medium of claim 7 wherein the predicate information specifies a predicate that is associated with the first symbol in the set of symbols and is dependent upon the second symbol included in the set of symbols.

10. The computer-readable non-transitory storage medium of claim 7 wherein the regular expression comprises at least one of an alternation operator, a grouping operator, or an aggregation operator.

11. The computer-readable non-transitory storage medium of claim 7 wherein at least one symbol in the regular expression has an associated lazy quantifier.

12. A system for detecting a pattern in a data stream comprising events, the system comprising:

a memory storing a plurality of instructions; and

a processor coupled to the memory, the processor configured to execute the plurality of instructions to;

receive predicate information, the predicate information specifying a predicate associated with each symbol in a set of symbols, the predicate information including a first predicate associated with a first symbol in the set of symbols, and including a second predicate associated with a second symbol in the set of symbols;

receive a regular expression specifying a pattern, the regular expression comprising one or more symbols from the set of symbols;

construct an automaton for the pattern;

upon receiving an event in the data stream,

determine, based upon the received event, states of the automaton and associated bindings, and

determine if the pattern is matched due to the received event based upon the states and associated bindings;

continue to determine, in response to determining that the pattern is matched by a particular sequence of events in the data stream, whether subsequent events following the particular sequence of events in the data stream produce a match that is longer than a match produced by the particular sequence of events; and

output the match produced by the particular sequence of events only after determining that no match longer than the match produced by the particular sequence of events is possible based on the states and associated bindings.

13. The system of claim 12 wherein the processor is configured to:

upon receiving an event in the data stream, determine one or more symbols from the set of symbols that are matched due to the received event, wherein a symbol is matched if the predicate associated with the symbol is satisfied due to the received event; and

upon receiving an event in the data stream, determine the states based upon the one or more symbols and set of states and associated bindings stored prior to receiving the input event.

14. The system of claim 12 wherein the predicate information specifies a predicate that is associated with the first symbol in the set of symbols and is dependent upon the second symbol included in the set of symbols.

15. The system of claim 12 wherein the regular expression comprises at least one of an alternation operator, a grouping operator, or an aggregation operator.

16. The system of claim 12 wherein at least one symbol in the regular expression has an associated lazy quantifier.