Methods and systems to fingerprint textual information using word runs

• US 8,286,171 B2
• Filed: 07/21/2008
• Issued: 10/09/2012
• Est. Priority Date: 07/21/2008
• Status: Active Grant

• Current Assignee: Freedom Solutions Group, LLC
• Original Assignee: Workshare Technology Incorporated
• Inventors: More, Scott, Beyer, Ilya
• Primary Examiner(s): Augustin, Evens J
• Application Number:
  US12/177,043
• Publication Number:
  US 20100017850A1
• Time in Patent Office: 1,541 Days
• Field of Search: 718/102, 718/103
• US Class Current: 718/102
• CPC Class Codes: G06F 21/554 involving event detection and direct action
• CPC Class Codes: G06F 21/608 Secure printing
• CPC Class Codes: G06F 21/62 Protecting access to data via a platform, e.g. using keys or access control rules
• CPC Class Codes: G06F 21/6218 to a system of files or objects, e.g. local or distributed file system or database
• CPC Class Codes: H04L 63/0245 Filtering by information in the payload
• CPC Class Codes: H04L 63/04 for providing a confidential data exchange among entities communicating through data packet networks
• CPC Class Codes: H04L 63/08 for authentication of entities cryptographic mechanisms or cryptographic arrangements for entity authentication H04L9/32
• CPC Class Codes: H04L 63/12 Applying verification of the received information cryptographic mechanisms or cryptographic arrangements for data integrity or data verification H04L9/32
• CPC Class Codes: H04L 63/1408 by monitoring network traffic monitoring network traffic per se H04L43/00

Abstract

The present invention provides methods and systems to enable fast, efficient, and scalable means for fingerprinting textual information using word runs. The present system receives textual information and provides algorithms to convert the information into representative fingerprints. In one embodiment, the fingerprints are recorded in a repository to maintain a database of an organization'"'"'s secure data. In another embodiment, textual information entered by a user is verified against the repository of fingerprints to prevent unauthorized disclosure of secure data. This invention provides approaches to allow derivative works (e.g., different ordering of words, substitution of words with synonyms, etc.) of the original information to be detected at the sentence level or even at the paragraph level. This invention also provides means for enhancing storage and resource efficiencies by providing approaches to optimize the number of fingerprints generated for the textual information.

Claims

1. A computer implemented method for preventing unauthorized disclosure of secure information, the computer implemented method comprising:

receiving information including a first text, by a computer system having at least a processor for executing instructions, said first text including a plurality of words;

normalizing, by said computer system, said first text into a first canonical text expression, said first canonical text expression including a plurality of normalized words;

generating, at said computer system, a first word hash list for said first canonical text expression, where said first word hash list is generated at a word level;

generating, at said computer system, a first set of fingerprints for said first word hash list;

wherein generating said first word hash list includes converting said plurality of normalized words into a plurality of word-value hashes, each specific one of said word-value hashes representing a specific normalized word; and

wherein said generating said first set of fingerprints includes;

assigning a sliding window of size W, wherein said sliding window is used for reading a W number of said word-value hashes from said first word hash list;

using said sliding window to read said W number of said word-level hashes from said first word hash list;

designating said word-value hash with a distinct value within said sliding window as an anchor; and

generating a fingerprint using a fingerprint hash function, wherein said fingerprint hash function is applied over all said word-value hashes contained within a start of said sliding window to where said anchor resides in said sliding window.

2. A computer implemented method for preventing unauthorized disclosure of secure information as recited in claim 1, wherein each of said plurality of words are defined as a combination of one or more text characters not separated by a specific predefined character, and each of said plurality of words are separated from a previous word and a subsequent word by at least one of said specific predefined character.

3. A computer implemented method for preventing unauthorized disclosure of secure information as recited in claim 2, wherein said specific predefined character is a space.

4. A computer implemented method for preventing unauthorized disclosure of secure information as recited in claim 2, wherein said specific predefined character is a punctuation character.

5. A computer implemented method for preventing unauthorized disclosure of secure information as recited in claim 2, wherein said specific predefined character includes one or more character types including a space, a period, a comma, a semi-colon, a colon, an exclamation point, a dash, a parenthesis, and a quote.

6. A computer implemented method for preventing unauthorized disclosure of secure information as recited in claim 1, wherein said plurality of words includes detecting separation of a specific word from a following or preceding word using a word boundary detector.

7. A computer implemented method for preventing unauthorized disclosure of secure information as recited in claim 6, wherein said word boundary detector is language independent.

8. A computer implemented method for preventing unauthorized disclosure of secure information as recited in claim 1, wherein said receiving information includes receiving secure information from a local database.

9. A computer implemented method for preventing unauthorized disclosure of secure information as recited in claim 1, wherein said normalizing said first text into a first canonical text expression includes converting said first text into Unicode.

10. A computer implemented method for preventing unauthorized disclosure of secure information as recited in claim 9, wherein said Unicode is Unicode Transformation Format 16 (“

UTF16”

).

11. A computer implemented method for preventing unauthorized disclosure of secure information as recited in claim 1, wherein said first text is case-folded before generating said first word hash list, such that a specific word-value hash assigned to a specific normalized word does not depend upon whether the characters in said specific normalized word are upper case or lower case characters.

12. A computer implemented method for preventing unauthorized disclosure of secure information as recited in claim 1, wherein said first word hash list includes representing said word-value hash as an integer.

13. A computer implemented method for preventing unauthorized disclosure of secure information as recited in claim 12, wherein said integer includes a 32-bit unsigned integer.

14. A computer implemented method for preventing unauthorized disclosure of secure information as recited in claim 1, wherein converting said plurality of normalized words into said plurality of word-value hashes includes applying a hash function over all characters of each normalized word.

15. A computer implemented method for preventing unauthorized disclosure of secure information as recited in claim 1, where an ordering of said plurality of word-value hashes in said word hash list corresponds to an original ordering of said plurality of normalized words.

16. A computer implemented method for preventing unauthorized disclosure of secure information as recited in claim 1, wherein said converting said plurality of normalized words includes excluding word-value hashes for each of said plurality of normalized words which are stop words.

17. A computer implemented method for preventing unauthorized disclosure of secure information as recited in claim 16, wherein said stop words include a predefined set of words that occur frequently and do not add substantive content.

18. A computer implemented method for preventing unauthorized disclosure of secure information as recited in claim 1, wherein said converting said plurality of normalized words includes mapping a predefined set of common words to a unique word-value hash.

19. A computer implemented method for preventing unauthorized disclosure of secure information as recited in claim 1, wherein said converting said plurality of normalized words includes mapping a predefined set of synonyms to a unique word-value hash.

20. A computer implemented method for preventing unauthorized disclosure of secure information as recited in claim 1, wherein said converting said plurality of normalized words includes mapping a predefined set of words in a particular category to a unique word-value hash.

21. A computer implemented method for preventing unauthorized disclosure of secure information as recited in claim 1, wherein said generating said first set of fingerprints further comprises:

moving said sliding window one said word-value hash to the right at a time within said first word hash list;

reading said W number of said word-value hashes from said sliding window at said time;

designating a second word-value hash with a new distinct value within said sliding window as a new anchor at said time; and

generating a new fingerprint at said time using said fingerprint hash function, wherein said new fingerprint is generated only when said new anchor from said sliding window at said time is not identical to said anchor from said sliding window at the immediately previous said time.

22. A computer implemented method for preventing unauthorized disclosure of secure information as recited in claim 1, wherein said fingerprint hash function includes any hash function that allows said fingerprint to be independent of the order of said words in said first word hash list.

23. A computer implemented method for preventing unauthorized disclosure of secure information as recited in claim 22, wherein said fingerprint hash function includes a symmetric hash function.

24. A computer implemented method for preventing unauthorized disclosure of secure information as recited in claim 23, wherein said symmetric hash function uses an addition hash function.

25. A computer implemented method for preventing unauthorized disclosure of secure information as recited in claim 23, wherein said symmetric hash function uses a multiplication hash function.

26. A computer implemented method for preventing unauthorized disclosure of secure information as recited in claim 23, wherein said symmetric hash function uses an exclusive-or hash function.

27. A computer implemented method for preventing unauthorized disclosure of secure information as recited in claim 21, wherein said first set of fingerprints are recorded in a fingerprint database.

28. A computer implemented method for preventing unauthorized disclosure of secure information as recited in claim 27, wherein said fingerprint database serves as a repository for said first set of fingerprints generated for all said secure information.

29. A computer implemented method for preventing unauthorized disclosure of secure information as recited in claim 27, wherein said first set of fingerprints are matched against any repository that maintains a database of said first set of fingerprints that were previously generated for all said secure information.

30. A computer implemented method for preventing unauthorized disclosure of secure information as recited in claim 1, wherein said first set of fingerprints are impervious to derivative works of said first text.

31. A computer implemented method for preventing unauthorized disclosure of secure information as recited in claim 30, wherein said derivative work includes at least one of:

change in the position at which a specific word of said plurality of words appears in said first text;

change in the order in which said plurality of words appear in said first text;

addition of one or more stop words to said first text;

deletion of one or more stop words from said first text;

addition of one or more punctuation marks to said first text;

deletion of one or more punctuation marks from said first text;

substitution of a specific word in said first text with a synonym of said specific word;

substitution of a specific word in said first text with a hypernym of said specific word;

or

substitution of a specific word in said first text with a different word, wherein said different word has the same stem as said specific word.