Pattern recognition using transition table templates

• US 8,572,014 B2
• Filed: 10/16/2009
• Issued: 10/29/2013
• Est. Priority Date: 10/16/2009
• Status: Active Grant

• Current Assignee: McAfee, LLC
• Original Assignee: McAfee, Inc. (McAfee, LLC)
• Inventors: Nedbal, Manuel, Kajekar, Sachin
• Primary Examiner(s): Chaki, Kakali
• Assistant Examiner(s): Bharadwaj, Kalpana
• Application Number:
  US12/580,438
• Publication Number:
  US 20110093694A1
• Time in Patent Office: 1,474 Days
• Field of Search: 706/45
• US Class Current: 706/45
• CPC Class Codes: H04L 63/0227 Filtering policies mail message filtering H04L51/212
• CPC Class Codes: H04L 63/1416 Event detection, e.g. attack signature detection

Abstract

Methods, systems, and apparatus, including computer programs encoded on computer storage media, for using transition table templates. In one aspect, a method includes receiving a transition table for a current state of a finite automaton and determining whether the transition table for the current state is similar to a transition table template in a set of transition table templates. The method further includes generating a condensed representation of the transition table if the transition table is similar to a transition table template and otherwise adding the transition table to the set of transition table templates. In another aspect, a method includes receiving an input element and determining whether a next state corresponding to the input element is in the difference region of a condensed transition table. The method further includes retrieving the next state from the difference region, or a transition table template, based on the determination.

Claims

1. A computer-implemented method, comprising:

receiving, in a data processing apparatus, a transition table for a current state of a finite automaton, the finite automaton configured to match patterns in input data, the transition table storing, for each possible next element in the input data, a corresponding next state of the finite automaton;

determining, with the data processing apparatus, whether the transition table for the current state is similar to any transition table template in a set of transition table templates, each transition table template being a transition table for a respective different state of the finite automaton, where the transition table is similar to a transition table template when a difference region for the transition table and the transition table template has a size satisfying a threshold, the difference region being a contiguous region in the transition table containing all next states that are different from the corresponding next states in the transition table template;

in response to determining that the transition table is similar to a transition table template in the set of transition table templates, generating, with the data processing apparatus, a condensed representation of the transition table, the condensed representation including a reference to the similar transition table template, an identification of the difference region, and the next states in the difference region of the transition table; and

in response to determining that the transition table is not similar to any transition table template in the set of transition table templates, adding, in the data processing apparatus, the transition table to the set of transition table templates.

2. The method of claim 1, wherein determining whether the transition table for the current state is similar to any transition table template further comprises:

selecting a transition table template from the plurality of transition table templates, the selected transition table template being the transition table template for which the transition table for the current state has the smallest difference region; and

determining whether the size of the difference region for the transition table and the selected transition table template satisfies the threshold.

3. The method of claim 1, wherein the transition table for the current state is indexed according to a deterministic indexing scheme, where next states are indexed according to indices associated with input data elements, and determining whether the transition table for the current state is similar to any transition table in the set of transition table templates comprises, for a particular transition table template:

identifying a lowest index in the transition table having a corresponding next state that is different from a next state stored at the lowest index in the particular transition table template;

identifying a highest index in the transition table having a corresponding next state that is different from a next state stored at the highest index in the particular transition table template; and

determining that the transition table is similar to the particular transition table template if the difference between the highest index and the lowest index satisfies a threshold.

4. The method of claim 3, wherein the identification of the region of differences comprises an identification of the lowest index and the highest index.

5. The method of claim 1, wherein the input data corresponds to network traffic.

6. The method of claim 5, wherein the patterns are character strings indicative of a prohibited process.

7. The method of claim 5, wherein the patterns are associated with network protocols.

8. The method of claim 1, wherein the input data corresponds to a stream of characters generated while scanning a file.

9. A system, comprising:

a processor; and

a computer storage medium coupled to the processor and including instructions, which, when executed by the processor, causes the processor to perform operations comprising;

receiving a transition table for a current state of a finite automaton, the finite automaton configured to match patterns in input data, the transition table storing, for each possible next element in the input data, a corresponding next state of the finite automaton;

determining whether the transition table for the current state is similar to any transition table template in a set of transition table templates, each transition table template being a transition table for a respective different state of the finite automaton, where the transition table is similar to a transition table template when a difference region for the transition table and the transition table template has a size satisfying a threshold, the difference region being a contiguous region in the transition table containing all next states that are different from the corresponding next states in the transition table template;

in response to determining that the transition table is similar to a transition table template in the set of transition table templates, generating a condensed representation of the transition table, the condensed representation including a reference to the similar transition table template, an identification of the difference region, and the next states in the difference region of the transition table; and

in response to determining that the transition table is not similar to any transition table template in the set of transition table templates, adding the transition table to the set of transition table templates.

10. A non-transitory computer storage medium encoded with a computer program, the computer program comprising instructions that when executed by data processing apparatus cause the data processing apparatus to perform actions comprising:

receiving a transition table for a current state of a finite automaton, the finite automaton configured to match patterns in input data, the transition table storing, for each possible next element in the input data, a corresponding next state of the finite automaton;

determining whether the transition table for the current state is similar to any transition table template in a set of transition table templates, each transition table template being a transition table for a respective different state of the finite automaton, where the transition table is similar to a transition table template when a difference region for the transition table and the transition table template has a size satisfying a threshold, the difference region being a contiguous region in the transition table containing all next states that are different from the corresponding next states in the transition table template;

in response to determining that the transition table is similar to a transition table template in the set of transition table templates, generating a condensed representation of the transition table, the condensed representation including a reference to the similar transition table template, an identification of the difference region, and the next states in the difference region of the transition table; and

in response to determining that the transition table is not similar to any transition table template in the set of transition table templates, adding the transition table to the set of transition table templates.

11. A computer-implemented method, comprising:

storing, in a data processing apparatus, a current state of a finite automaton and a condensed transition table for the current state, the finite automaton configured to match patterns in input data, the condensed transition table indicating, for each possible next element in the input data, a corresponding next state of the finite automaton, the condensed representation including a reference to a transition table template, an identification of a difference region for the transition table and the transition table template, and a difference table storing the next states of the difference region of the transition table, where the difference region is a contiguous region in the transition table containing all next states that are different from the corresponding next states in the transition table template;

receiving, in the data processing apparatus, an input element;

determining, with the data processing apparatus, a next state for the finite automaton, the determining including;

determining whether a next state corresponding to the input element is in the difference region;

in response to determining that the next state corresponding to the input element is not in the difference region, retrieving the next state from the transition table template; and

in response to determining that the next state corresponding to the input element is in the difference region, retrieving the next state from the difference table; and

updating the current state of the finite automaton to be the next state.

12. The method of claim 11, wherein:

the transition table, the transition table template, and the difference table are each indexed according to a deterministic indexing scheme, where next states are indexed according to indices associated with input elements, and

the identification of the difference region includes a lowest index in the transition table having a corresponding next state that is different from a next state stored at the lowest index in the transition table template, and a highest index in the transition table having a corresponding next state that is different from a next state stored at the highest index in the transition table template;

determining whether a next state corresponding to the input element is in the difference region comprises determining whether an index corresponding to the input element is between the lowest index and the highest index;

retrieving the next state from the transition table template comprises retrieving the next state from the transition table template at the index corresponding to the input element; and

retrieving the next state from the difference table comprises retrieving the next state from the difference table at an index corresponding to the difference between the index corresponding to the input element and the lowest index.

13. The method of claim 11, further comprising:

receiving input data, the input data compressed according to a compression scheme;

decompressing the input data; and

identifying the input element from the decompressed input data.

14. The method of claim 11, further comprising:

receiving input data encrypted according to an encryption scheme;

decrypting the input data; and

identifying the input element from the decrypted input data.

15. The method of claim 11, further comprising:

determining that the next state indicates that the input data matches a predefined pattern; and

in response to the determination, initiating a protection process on a source of the input data.

16. The method of claim 15, further comprising terminating processing of the input data in response to the determination.

17. The method of claim 11, further comprising:

determining that the next state indicates that the input data matches a predefined pattern; and

modifying an analysis of network traffic in response to the determination.

18. The method of claim 11, further comprising:

determining from the next state that the input data does not match any of a group of predefined patterns represented by the finite automaton; and

in response to the determination, terminating processing of the input data.

19. A system, comprising:

a processor; and

a computer storage medium coupled to the processor and including instructions, which, when executed by the processor, causes the processor to perform operations comprising;

storing a current state of a finite automaton and a condensed transition table for the current state, the finite automaton configured to match patterns in input data, the condensed transition table indicating, for each possible next element in the input data, a corresponding next state of the finite automaton, the condensed representation including a reference to a transition table template, an identification of a difference region for the transition table and the transition table template, and a difference table storing the next states of the difference region of the transition table, where the difference region is a contiguous region in the transition table containing all next states that are different from the corresponding next states in the transition table template;

receiving an input element;

determining a next state for the finite automaton, the determining including;

determining whether a next state corresponding to the input element is in the difference region;

in response to determining that the next state corresponding to the input element is not in the difference region, retrieving the next state from the transition table template; and

in response to determining that the next state corresponding to the input element is in the difference region, retrieving the next state from the difference table; and

updating the current state of the finite automaton to be the next state.

20. A non-transitory computer storage medium encoded with a computer program, the computer program comprising instructions that when executed by data processing apparatus cause the data processing apparatus to perform actions comprising:

storing a current state of a finite automaton and a condensed transition table for the current state, the finite automaton configured to match patterns in input data, the condensed transition table indicating, for each possible next element in the input data, a corresponding next state of the finite automaton, the condensed representation including a reference to a transition table template, an identification of a difference region for the transition table and the transition table template, and a difference table storing the next states of the difference region of the transition table, where the difference region is a contiguous region in the transition table containing all next states that are different from the corresponding next states in the transition table template;

receiving an input element;

determining a next state for the finite automaton, the determining including;

determining whether a next state corresponding to the input element is in the difference region;

in response to determining that the next state corresponding to the input element is not in the difference region, retrieving the next state from the transition table template; and

in response to determining that the next state corresponding to the input element is in the difference region, retrieving the next state from the difference table; and

updating the current state of the finite automaton to be the next state.

21. An article of manufacture, comprising:

a non-transitory computer readable medium; and

information stored in the computer readable medium that, when processed by a computer, defines a data structure storing a condensed representation of a transition table for a current state of a finite automaton, the finite automaton configured to match patterns in input data, the transition table including, for each possible next element in the input data, a corresponding next state of the finite automaton, the data structure comprising;

a reference to a transition table template for the transition table;

an identification of a difference region for the transition table and the transition table template, the difference region being a contiguous region in the transition table containing all next states that are different from the corresponding next states in the transition table template; and

the next states corresponding to the difference region in the transition table.

22. The article of manufacture of claim 21, wherein:

the transition table for the current state is indexed according to a deterministic indexing scheme, where next states are indexed according to indices associated with input data elements; and

the identification of the difference region includes;

a lowest index in the transition table having a corresponding next state that is different from a next state stored at the lowest index in the transition table template; and

a highest index in the transition table having a corresponding next state that is different from a next state stored at the highest index in the transition table template.