Automatic generation of embedded signatures for duplicate detection on a public network

• US 20090299994A1
• Filed: 05/30/2008
• Published: 12/03/2009
• Est. Priority Date: 05/30/2008
• Status: Active Grant

• Current Assignee: AT&T Intellectual Property I LP (AT&T, Inc.), New York University
• Original Assignee: New York University
• Inventors: Krishnamurthy, Balachander, Kumar, Saurabh, Subramanian, Lakshminarayanan
• Granted Patent: US 7,979,413 B2
• Time in Patent Office: Days
• US Class Current: 1/1
• CPC Class Codes: G06F 16/3334 Selection or weighting of terms from queries, including natural language queries
• CPC Class Codes: G06F 21/552 involving long-term monitoring or reporting

Abstract

In accordance with an aspect of the invention, a method and system are disclosed for constructing an embedded signature in order to facilitate post-facto detection of leakage of sensitive data. The leakage detection mechanism involves: 1) identifying at least one set of words in an electronic document containing sensitive data, the set of words having a low frequency of occurrence in a first collection of electronic documents; and, 2) transmitting a query to search a second collection of electronic documents for any electronic document that contains the set of words having a low frequency of occurrence. This leakage detection mechanism has at least the following advantages: a) it is tamper-resistant; b) it avoids the need to add a watermark to the sensitive data, c) it can be used to locate the sensitive data even if the leakage occurred before the embedded signature was ever identified; and, d) it can be used to detect an embedded signature regardless of whether the data is being presented statically or dynamically.

Claims

1. A method comprising:

identifying at least one set of words in a first electronic document, said set of words having a frequency of occurrence in a first collection of electronic documents that is below a predetermined threshold; and

transmitting a query to search a second collection of electronic documents for any electronic documents that contain the said set of words.

2. The method of claim 1 wherein a first party transmits the query to search the second collection of electronic documents to a second party that has access to the second collection of electronic documents.

3. The method of claim 2 wherein the second party is an enterprise maintaining an Internet search engine.

4. The method of claim 1 wherein said set of words comprises a set of consecutive words.

5. The method of claim 4 wherein the set of consecutive words consists of more than one word and less than six words.

6. The method of claim 1 wherein:

at least two sets of consecutive words are identified in the first electronic document, each set of consecutive words having a frequency of occurrence in the said first collection of electronic documents that is below a predetermined threshold;

at least one of the said at least two sets of consecutive words is selected randomly; and

a query is transmitted to search the said second collection of electronic documents for any document that contains the said randomly selected at least one set of consecutive words.

7. The method of claim 1 wherein:

at least three sets of consecutive words are identified in the first electronic document, each set of consecutive words having a frequency of occurrence in the said first collection of electronic documents that is below a predetermined threshold;

at least two of the said at least three sets of consecutive words are selected randomly; and

a query is transmitted to search the second collection of electronic documents for any document that contains the said randomly selected at least two sets of consecutive words.

8. The method of claim 1 wherein the first collection of electronic documents is different than the second collection of electronic documents.

9. The method of claim 1 wherein the first collection of electronic documents is the same as the second collection of electronic documents.

10. The method of claim 1 wherein the first collection of electronic documents is a collection of Internet web pages searchable by a repository of occurrences of word frequency on the public internet.

11. The method of claim 1 wherein the set of words having a low frequency of occurrence in the first collection of electronic documents appears at a frequency of less than one in five hundred documents.

12. The method of claim 1 wherein the set of words having a low frequency of occurrence in the first collection of electronic documents appears at a frequency of less than one in a thousand documents.

13. The method of claim 1 wherein the set of second documents comprises a web page.

14. A method comprising:

transmitting a query to search a second collection of electronic documents for any electronic documents that contain a set of words having a frequency of occurrence in a first collection of electronic documents that is below a predetermined threshold.

15. The method of claim 14 wherein said set of words comprises a set of consecutive words.

16. The method of claim 15 wherein a first party transmits the query to search the second collection of electronic documents to a second party that has access to the second collection of electronic documents.

17. The method of claim 14 wherein:

a plurality of set of words is identified, each set having a frequency of occurrence in the first collection of electronic documents below a predetermined threshold; and

the query includes at least one of the identified sets of words having a frequency of occurrence in the first collection of electronic documents below a predetermined threshold.

18. An apparatus comprising:

means for identifying at least one set of words in a first electronic document, said set of words having a frequency of occurrence in a first collection of electronic documents that is below a predetermined threshold; and

means for transmitting a query to search a second collection of electronic documents for any electronic documents that contain the said set of words.

19. A computer readable medium encoded with computer executable instructions defining steps comprising:

identifying at least one set of words in a first electronic document, said set of words having a frequency of occurrence in a first collection of electronic documents that is below a predetermined threshold; and

transmitting a query to search a second collection of electronic documents for any electronic documents that contain the said set of words.