SYSTEM AND METHOD FOR PROTECTING SPECIFIED DATA COMBINATIONS

• US 20120114119A1
• Filed: 11/04/2010
• Published: 05/10/2012
• Est. Priority Date: 11/04/2010
• Status: Active Grant

• Current Assignee: McAfee, LLC
• Original Assignee: McAfee, Inc. (McAfee, LLC)
• Inventors: Ahuja, Ratinder Paul Singh, Deninger, William J.
• Granted Patent: US 8,806,615 B2
• Time in Patent Office: Days
• US Class Current: 380/44
• CPC Class Codes: H04L 63/0853 using an additional device, e.g. smartcard, SIM or a different communication terminal cryptographic mechanisms or cryptographic arrangements for entity authentication involving additional secure or trusted devices H04L9/3234
• CPC Class Codes: H04L 63/1416 Event detection, e.g. attack signature detection

Abstract

A method in one example implementation includes extracting a plurality of data elements from a record of a data file, tokenizing the data elements into tokens, and storing the tokens in a first tuple of a registration list. The method further includes selecting one of the tokens as a token key for the first tuple, where the token is selected because it occurs less frequently in the registration list than each of the other tokens in the first tuple. In specific embodiments, at least one data element is an expression element having a character pattern matching a predefined expression pattern that represents at least two words and a separator between the words. In other embodiments, at least one data element is a word defined by a character pattern of one or more consecutive essential characters. Other specific embodiments include determining an end of the record by recognizing a predefined delimiter.

Claims

1. A method, comprising:

extracting a plurality of data elements from a record of a data file;

tokenizing the plurality of data elements into a plurality of tokens;

storing the plurality of tokens in a first tuple of a registration list;

selecting one of the plurality of tokens as a token key for the first tuple, wherein the token key occurs less frequently in the registration list than each other token of the plurality of tokens in the first tuple.

2. The method of claim 1, wherein at least one of the plurality of data elements is a word defined by a character pattern of one or more consecutive essential characters.

3. The method of claim 1, wherein at least one data element is an expression element defined by a character pattern matching a predefined expression pattern, the predefined expression pattern representing at least two words and a separator between the words.

4. The method of claim 1, wherein each of the tokens of the plurality of tokens is a numerical representation of a corresponding data element.

5. The method of claim 1, further comprising:

determining an end of the record by recognizing a predefined delimiter indicating the end of the record.

6. The method of claim 1, wherein the registration list includes a plurality of tuples, each tuple of the registration list being associated with only one record of one or more data files.

7. The method of claim 1, wherein the token key is stored in an index of an index table, the index including a first offset indicating a location of the first tuple in the registration list.

8. The method of claim 7, wherein the index includes a document identifier associated with the first offset, the document identifier representing the data file containing the record.

9. The method of claim 7, wherein the index includes a second offset corresponding to a second tuple with another plurality of tokens in the registration list, the second offset indicating a location of the second tuple in the registration list, wherein one of the other plurality of tokens of the second tuple is equivalent to the token key.

10. The method of claim 1, wherein the selecting one of the plurality of tokens as a token key for the first tuple further includes:

generating a count table including a sum of occurrences of each token in the registration list; and

searching the count table to identify the one of the plurality of tokens having a lowest sum of occurrences relative to each other token in the first tuple.

11. An apparatus, comprising:

one or more registration modules operable to generate a registration list having a plurality of tuples, each tuple representing a specified combination of data elements; and

a processor operable to execute instructions associated with the one or more registration modules, including;

extracting a plurality of data elements from a record of a data file;

tokenizing the plurality of data elements into a plurality of tokens;

storing the plurality of tokens in a first tuple of the registration list;

selecting one of the plurality of tokens as a token key for the first tuple, wherein the token key occurs less frequently in the registration list than each other token of the plurality of tokens in the first tuple.

12. The apparatus of claim 11, wherein at least one data element is an expression element defined by a character pattern matching a predefined expression pattern, the predefined expression pattern representing at least two words and a separator between the words.

13. The apparatus of claim 11, wherein the processor is operable to perform further instructions, comprising:

determining an end of the record by recognizing a predefined delimiter indicating the end of the record.

14. The apparatus of claim 13, wherein each tuple of the registration list is associated with only one record of one or more data files.

15. The apparatus of claim 11, wherein the token key is stored in an index of an index table, the index including a first offset indicating a location of the first tuple in the registration list.

16. The apparatus of claim 15, wherein the index includes a second offset corresponding to a second tuple with another plurality of tokens in the registration list, the second offset indicating a location of the second tuple in the registration list, wherein one of the other plurality of tokens of the second tuple is equivalent to the token key.

17. The apparatus of claim 11, wherein the selecting one of the plurality of tokens as a token key for the first tuple further includes:

generating a count table including a sum of occurrences for each token in the registration list; and

searching the count table to identify the one of the plurality of tokens having a lowest sum of occurrences relative to each other token in the first tuple.

18. A method, comprising:

identifying a start of a first data element in a record of a data file;

determining the first data element is an expression element if a first string of characters beginning at the start of the first data element matches a predefined expression pattern, the predefined expression pattern representing at least two words and a separator between the two words;

extracting the expression element;

tokenizing the expression element into a first token; and

storing the first token in a first tuple of a registration list.

19. The method of claim 18, further comprising:

normalizing the expression element prior to the tokenizing the expression element.

20. The method of claim 19, wherein the normalizing includes eliminating the separator from the expression element.

21. The method of claim 18, wherein the predefined expression pattern is a longest one of a plurality of predefined expression patterns that matches any string of characters beginning at the start of the first data element.

22. The method of claim 18, further comprising:

identifying a start of a second data element in the record;

determining the second data element is a word, wherein none of a plurality of predefined expression patterns matches a second string of characters beginning at the start of the second data element;

extracting the word;

tokenizing the word into a second token; and

storing the second token in the first tuple of the registration list.

23. The method of claim 18, wherein a predefined delimiter separates each consecutive pair of records in the data file.

24. The method of claim 18, wherein a predefined delimiter is located at an end of the record in the data file, the predefined delimiter selected from a group consisting of a semi-colon, a line feed, a carriage return, a colon, and a period.

25. The method of claim 18, wherein each tuple of the registration list is associated with only one record of the data file.

26. The method of claim 18, wherein each of the at least two words is defined by a character pattern of one or more consecutive essential characters.

27. The method of claim 26, wherein the essential characters are selected from a group consisting of numbers, letters of a written language, and symbols representing speech segments of a written language.

28. The method of claim 18, wherein the separator is not equivalent to an essential character, and wherein the separator is differentiated from a predefined delimiter if the separator is equivalent to the predefined delimiter, the predefined delimiter indicating an end of the record.

29. An apparatus, comprising:

a registration list module; and

a processor operable to execute instructions associated with the registration list module, including;

identifying a start of a first data element in a record of a data file;

determining the first data element is an expression element if a first string of characters beginning at the start of the first data element matches a predefined expression pattern, the predefined expression pattern representing at least two words and a separator between the two words;

extracting the expression element;

tokenizing the expression element into a first token; and

storing the first token in a first tuple of the registration list.

30. The apparatus of claim 29, wherein the processor is operable to perform further instructions, comprising:

normalizing the expression element prior to the tokenizing the expression element, wherein the normalizing includes eliminating the separator from the expression element.

31. The apparatus of claim 29, wherein the predefined expression pattern is a longest one of a plurality of predefined expression patterns that matches any string of characters beginning at the start of the first data element.

32. The apparatus of claim 29, wherein the processor is operable to perform further instructions, comprising:

identifying a start of a second data element in the record;

determining the second data element is a word, wherein none of a plurality of predefined expression patterns matches a second string of characters beginning at the start of the second data element;

extracting the word;

tokenizing the word into a second token; and

storing the second token in the first tuple of the registration list.

33. The apparatus of claim 29, wherein a predefined delimiter separates each consecutive pair of records in the data file.

34. The apparatus of claim 29, wherein each tuple of the registration list is associated with only one record of the data file.

35. A method, comprising:

extracting a plurality of data elements from an object;

tokenizing the plurality of data elements into a plurality of object tokens;

identifying a first tuple of a plurality of tuples in a registration list, the first tuple including a plurality of associated tokens;

determining if each one of the plurality of associated tokens corresponds to at least one of the plurality of object tokens; and

validating an event if an amount of correspondence between the plurality of associated tokens and the plurality of object tokens meets a predetermined threshold.

36. The method of claim 35, wherein the identifying the first tuple further includes:

identifying a pending key from the plurality of object tokens; and

searching an index table to find a token key corresponding to the pending key, wherein an offset is linked to the token key, the offset indicating a location of the first tuple in the registration list.

37. The method of claim 36, wherein the pending key is included in a pending key list, the pending key list generated by searching the index table to determine if any of the plurality of object tokens corresponds to one of a plurality of token keys in the index table, wherein each object token that corresponds to one of the plurality of token keys is stored in the pending key list.

38. The method of claim 36, wherein the token key occurs less frequently in the registration list than each other associated token of the plurality of associated tokens in the first tuple.

39. The method of claim 35, wherein the predetermined threshold is met when each of the plurality of associated tokens in the first tuple corresponds to at least one of the plurality of object tokens.

40. The method of claim 35, further comprising:

representing the plurality of object tokens in a bit hash table by setting a bit for each of the plurality of object tokens, the bit hash table including a plurality of bits each having a separate bit position in the bit hash table.

41. The method of claim 40, wherein the determining further includes evaluating the bit hash table to determine whether a bit is set in each bit position of the bit hash table corresponding to one of the plurality of associated tokens.

42. The method of claim 35, wherein at least one of the plurality of data elements is a word defined by a character pattern of one or more consecutive essential characters.

43. The method of claim 35, wherein at least one of the plurality of data elements is an expression element having a character pattern matching a predefined expression pattern, the predefined expression pattern representing at least two words and a separator between the words.

44. An apparatus, comprising:

one or more detection modules operable to receive an object including a plurality of data elements for comparison to a registration list with a plurality of tuples, each of the tuples representing a specified combination of data elements; and

a processor operable to execute instructions associated with one or more detection modules, including;

extracting the plurality of data elements from the object;

tokenizing the plurality of data elements into a plurality of object tokens;

identifying a first tuple of the plurality of tuples in the registration list, the first tuple including a plurality of associated tokens;

determining if each one of the plurality of associated tokens corresponds to at least one of the plurality of object tokens; and

validating an event if an amount of correspondence between the plurality of associated tokens and the plurality of object tokens meets a predetermined threshold.

45. The apparatus of claim 44, wherein the identifying the first tuple further includes:

identifying a pending key from the plurality of object tokens; and

searching an index table to find a token key corresponding to the pending key, wherein an offset is linked to the token key, the offset pointing to a location of the first tuple in the registration list.

46. The apparatus of claim 45, wherein the pending key is included in a pending key list, the pending key list generated by searching the index table to determine if any of the plurality of object tokens corresponds to one of a plurality of token keys in the index table, wherein each object token that corresponds to one of the plurality of token keys in the index table is stored in the pending key list.

47. The apparatus of claim 44, wherein the token key occurs less frequently in the registration list than each other associated token of the plurality of associated tokens in the first tuple.

48. The apparatus of claim 44, wherein the predetermined threshold is met when each of the plurality of associated tokens in the first tuple corresponds to at least one of the plurality of object tokens.

49. The apparatus of claim 44, wherein at least one of the plurality of data elements is a word defined by a character pattern of one or more consecutive essential characters.

50. The apparatus of claim 44, wherein at least one of the plurality of data elements is an expression element having a character pattern matching a predefined expression pattern, the predefined expression pattern representing at least two words and a separator between the words.