Method and a system for information identification

• US 7,991,751 B2
• Filed: 04/02/2004
• Issued: 08/02/2011
• Est. Priority Date: 04/02/2003
• Status: Active Grant

• Current Assignee: Forcepoint LLC (Francisco Partners Management LLC)
• Original Assignee: PortAuthority Technologies, Inc. (Francisco Partners Management LLC)
• Inventors: Troyansky, Lidror, Carny, Ofir, Peled, Ariel
• Primary Examiner(s): Ali; Mohammad
• Assistant Examiner(s): Ahluwalia; Navneet K
• Application Number:
  US10/815,764
• Publication Number:
  US 20040199490A1
• Time in Patent Office: 2,678 Days
• Field of Search: 707 1- 10, 707100-1041, 707200-205, 707/602, 707/698, 713100-250
• US Class Current: 707/698
• CPC Class Codes: G06F 7/02 Comparing digital values G06F7/06, G06F7/22, G06F7/38 take precedence

Abstract

A method for detecting an information item within an information sequence obtained from a digital medium, the information item comprising any one of a specified set of prestored information items, comprises: transforming each of the set of prestored information items into a respective representation, in accordance with a predetermined transformation format; transforming the information sequence obtained from the digital medium, in accordance with the transformation format; and determining the presence of one or more of the prestored information items within the transformed information sequence, utilizing the respective representation.

Claims

1. A processor-implemented method for detecting distribution of an information item included within passing information sequences obtained from a digital medium, said information item comprising any one of a specified set of prestored information items whose distribution it is desired to control, comprising:

transforming each item of said set of prestored information items whose distribution it is desired to control from a first representation format into a respective format facilitating a first comparison, said first comparison being fast in relation to a second relatively slower textual comparison, in accordance with a predetermined transformation format, said predetermined transformation format being preservative of meaning;

transforming passing information sequences, obtained from said digital medium, into said format facilitating said first relatively fast comparison in accordance with said transformation format;

determining the presence of one or more of said prestored information items within said transformed information sequence, said determining comprising;

comparing respective information sequences in said format facilitating said relatively fast comparison with said prestored information items in said format facilitating said relatively fast comparison;

when a match is found between said formats facilitating said relatively fast comparison then carrying out said second relatively slower textual comparison between said respective prestored information item in said first representation format and a respective information sequence obtained from said digital medium, and

when a match is found using said second relatively slower textual comparison, applying a policy to control distribution of said respective information sequence.

2. A processor-implemented method according to claim 1, further comprising storing said representations in a database.

3. A processor-implemented method according to claim 1, further comprising sorting said representations into a sorted list.

4. A processor-implemented method according to claim 3, wherein said sorting is in accordance with a tree sorting algorithm.

5. A processor-implemented method according to claim 1, wherein said information item comprises a single word.

6. A processor-implemented method according to claim 1, wherein said information item comprises a sequence of words.

7. A processor-implemented method according to claim 1, wherein said information item comprises a delimited sequence of sub-items.

8. A processor-implemented method according to claim 7, wherein each of said sub-items comprises a sequence of alphanumeric characters.

9. A processor-implemented method according to claim 1, wherein a type of said information item comprises one of a group of types comprising:

a word, a phrase, a number, a credit-card number, a social security number, a name, an address, an email address, and an account number.

10. A processor-implemented method according to claim 1, wherein said information sequence is provided over a digital traffic channel.

11. A processor-implemented method according to claim 10, wherein said digital traffic channel comprises one of a group of channels comprising:

email, instant messaging, peer-to-peer network, fax, and a local area network.

12. A processor-implemented method according to claim 1, wherein said information sequence comprises the body of an email.

13. A processor-implemented method according to claim 1, wherein said information sequence comprises an email attachment.

14. A processor-implemented method according to claim 1, further comprising retrieving said information sequence from a digital storage medium.

15. A processor-implemented method according to claim 14, wherein said digital storage medium comprises a digital cache memory.

16. A processor-implemented method according to claim 1, wherein said representation depends only on the textual and numeric content of the information item.

17. A processor-implemented method according to claim 1, wherein said transforming into a format that facilitates fast comparison comprises Unicode encoding.

18. A processor-implemented method according to claim 1, wherein said transforming into a format that facilitates fast comparison comprises converting all characters to upper-case characters or to lower-case characters.

19. A processor-implemented method according to claim 1, wherein said transforming into a format that facilitates fast comparison comprises encoding an information item into a numeric representation.

20. A processor-implemented method according to claim 1, wherein said transforming into a format that facilitates fast comparison comprises applying a first hashing function to said representations.

21. A processor-implemented method according to claim 1, wherein said information sequence comprises sub-sequences.

22. A processor-implemented method according to claim 21, wherein said sub-sequences are separated by delimiters.

23. A processor-implemented method according to claim 22 wherein said sub-sequences separated by delimiters are any of:

words;

names, and numbers.

24. A processor-implemented method according to claim 23, further comprising scanning said information sequence to identify said sub-sequences.

25. A processor-implemented method according to claim 24, and said determining is performed by matching said information item to an ordered series of said sub-sequences.

26. A processor-implemented method according to claim 1, wherein said policy is a security policy, said security policy comprises at least one of the following group of security policies:

blocking said transmission, logging a record of said detection and detection details, and reporting said detection and detection details.

27. A processor-implemented method according to claim 26, wherein said information items are divided into sets, and wherein said security policy depends on the number of detected information items that belong to the same set.

28. A processor-implemented method according to claim 27 wherein each of said sets comprises information items associated with a single individual.

29. A processor-implemented method according to claim 1, wherein said information item comprises a sequence of sub-items.

30. A processor-implemented method according to claim 29, wherein said sub-items are separated by delimiters.

31. A processor-implemented method according to claim 29, wherein a sub-item comprises one of a group comprising:

a word, a number, and a character string.

32. A processor-implemented method according to claim 29, wherein said determining comprises using a state machine operable to detect said sequence of delimited sub-items within said information sequence.

33. A processor-implemented method according to claim 29, wherein said transforming into a format facilitating fast comparison comprises:

applying a first hashing function to assign a respective preliminary hash value to each sub-item within said information item; and

applying a second hashing function to assigning a global hash value to said information item based on said preliminary hash values of said sub-items.

34. A processor-implemented method according to claim 33, wherein said information sequence comprises sub-sequences, and wherein said determining comprises:

applying said first hashing function to assign a respective preliminary hash value to each of said sub-sequences;

applying said second hashing function to at least one of said preliminary hash values to assign a global hash value to said at least one of said sub-sequences; and

comparing said global hash value to hash values of said sub-sequences.

35. A processor-implemented method according to claim 34, wherein said sub-sequences comprise one of a group comprising:

a word, a number, and a character string.

36. A processor-implemented method according to claim 34, wherein said sub-sequences comprise a plurality of ordered combinations of sub-sequences within said data sequence.

37. A processor-implemented method according to claim 35, wherein said sub-sequences comprise a plurality of combinations of sub-sequences within said data sequence.

38. A processor-implemented method according to claim 37, wherein said second hash function is invariant to reordering of at least two of said sub-sequences.

39. A processor-implemented method according to claim 38, further comprising checking whether a delimited segment was previously stored, and continuing said detection process only if a current delimited segment was previously stored.

49. A processor-implemented method according to claim 2, wherein said transforming said representation and storage of said information items comprises:

a) assigning a hash value to each delimited segment within said information item;

b) assigning a hash value for said information item based on said hashes assigned to delimited segments within said information item;

c) storing said hash values evaluated in step a) and step b) above;

and wherein detecting said information items within said digital medium comprises;

d) assigning a hash value to each delimited segment within said digital medium utilizing the same hash function used in step a) above;

e) assigning a hash value for sequences of delimited segments utilizing the same hash function used in step b) above, said sequences being of pluralities of possible numbers of delimited segments within said information items;

f) comparing the hashes values evaluated in step e) above with said hash values stored in step e) above.

40. An apparatus for detecting an information item included within passing information sequences obtained from a digital medium, said information item comprising any one of a specified set of prestored data items whose distribution it is desired to control, comprising:

a preprocessor, configured for transforming each item of said prestored information items from a first representation format into a respective format facilitating a first comparison, said first comparison being fast in relation to a second relatively slower textual comparison, in accordance with a predetermined transformation format, said predetermined transformation format being preservative of meaning;

a scanner, configured for transforming passing information sequences, obtained from said digital medium, into said format facilitating said first relatively fast comparison in accordance with said transformation format; and

a comparator associated with said preprocessor and said scanner, configured for determining the presence of one or more of said prestored information items within said transformed information sequence by;

comparing respective information sequences in said format facilitating said relatively fast comparison with said prestored information items in said format facilitating said relatively fast comparison, and

wherever a match is found between said formats facilitating said relatively fast comparison, then carrying out said second relatively slower textual comparison between said respective prestored information item in said first representation format and a respective information sequence obtained from said digital medium, to make a second determination of a match,

the apparatus being configured to apply a policy for controlling distribution of said respective information sequence when said second match is detected.

41. An apparatus for detecting a specified information item within an information sequence according to claim 40, further comprising a user interface configured for inputting said information items.

42. An apparatus for detecting a specified information item within an information sequence according to claim 40, further comprising a sorter configured for forming a sorted list of the respective representations of set of data items.

43. An apparatus for detecting a specified information item within an information sequence according to claim 40, further comprising a database configured for storing a representation of each data item of said set.

44. An apparatus for detecting a specified information item within an information sequence according to claim 40, wherein a type of said information item comprises one of a group of types comprising:

a word, a phrase, a number, a credit-card number, a social security number, a name, an address, an email address, and an account number.

45. An apparatus for detecting a specified information item within an information sequence according to claim 40, wherein said information sequence is provided over a digital traffic channel.

46. An apparatus for detecting a specified information item within an information sequence according to claim 40, wherein said scanner is further configured for retrieving said information sequence from a digital storage medium.

47. An apparatus for detecting a specified information item within an information sequence according to claim 46, wherein said digital storage medium comprises digital storage medium within a proxy server.

48. An apparatus for detecting a specified information item within an information sequence according to claim 40, wherein said encoding function comprises a hashing function.