Method and System for Adjudicating Text Against a Defined Policy

• US 20100287182A1
• Filed: 05/08/2009
• Published: 11/11/2010
• Est. Priority Date: 05/08/2009
• Status: Active Grant

• Current Assignee: Forcepoint Federal Holdings LLC (Forcepoint LLC)
• Original Assignee: Raytheon Company (Rtx Corporation)
• Inventors: Brooks, Randall S., Traxler, Sylvia A., Rodriguez, Ricardo J.
• Granted Patent: US 8,234,259 B2
• Time in Patent Office: Days
• US Class Current: 707/769
• CPC Class Codes: G06F 16/325 Hash tables
• CPC Class Codes: H04L 51/212 using filtering or selective blocking

Abstract

A computerized method of adjudicating text against a policy includes receiving one or more system policies, creating a system datastructure for each received system policy, receiving an input message comprising a text to be adjudicated, selecting a system policy from the one or more received system policies based on the input message, and processing the text to be adjudicated and the system datastructure corresponding to the selected system policy to determine if a prohibited word is present in the text to be adjudicated. The one or more system policies include one or more prohibited words and a first hit value corresponding to each prohibited word. The system datastructure includes a plurality of linked lists corresponding the letters of the alphabet and a head linked list operable to store one or more found prohibited words.

Claims

1. A computerized method of adjudicating text against a policy comprising:

receiving one or more system policies comprising one or more prohibited words and a first hit value corresponding to each prohibited word, the first hit value indicating the significance of the prohibited word;

creating a system datastructure for each received system policy, each system datastructure further comprising;

a hash table comprising a plurality of linked lists corresponding the letters of the alphabet; and

a head linked list operable to store one or more found prohibited words;

receiving an input message comprising text to be adjudicated;

selecting a system policy from the one or more received system policies based on the input message; and

determining whether a prohibited word is present in the text to be adjudicated by processing the text to be adjudicated and the system datastructure corresponding to the selected system policy.

2. The computerized method of adjudicating text against a policy of claim 1, wherein each linked list corresponding to a letter of the alphabet comprises the prohibited words from the system policy that begin with a corresponding letter of the alphabet and the first hit value associated with each prohibited word.

3. The computerized method of adjudicating text against a policy of claim 1, wherein the input message comprises a Simple Object Access Protocol message.

4. The computerized method of adjudicating text against a policy of claim 2, wherein the determining whether a prohibited word is present in the text to be adjudicated by processing the text to be adjudicated and the system datastructure corresponding to the selected system policy comprises:

creating a temporary datastructure by duplicating the system datastructure corresponding to the selected system policy;

selecting a word from the text to be adjudicated;

identifying a linked list in the temporary datastructure that corresponds to the first letter of the selected word;

searching the identified linked list for the selected word;

recording the selected word in the head linked list of the temporary datastructure if the selected word is found in the identified linked list; and

repeating the selecting a word, identifying a linked list, searching the identified linked list for the selected word, and recording the selected word in the head linked list of the temporary datastructure if the selected word is found in the identified linked list until every word in the text to be adjudicated has been searched.

5. The computerized method of adjudicating text against a policy of claim 1 wherein the one or more system policies further comprise a group of associated prohibited words, the group comprising:

a threshold value corresponding to the group of associated prohibited words;

an associated prohibited word, the associated prohibited word being associated to one of the one or more prohibited words;

a second hit value corresponding to the associated prohibited word; and

a proximity value corresponding to the associated prohibited word.

6. The computerized method of adjudicating text against a policy of claim 5 wherein each system datastructure further comprises the associated prohibited word, the second hit value, and the threshold value.

7. The computerized method of adjudicating text against a policy of claim 6 further comprising determining whether the text contains the associated prohibited word by processing the text and the system datastructure corresponding to the selected system policy, the processing comprising:

selecting a found prohibited word from the head linked list having an associated prohibited word;

determining a proximity string of text associated with the selected found prohibited word;

searching the proximity string of text for the associated prohibited word; and

determining whether the proximity string of text comprises the associated prohibited word.

8. The computerized method of adjudicating text against a policy of claim 7 further comprising:

determining whether the threshold value has been exceeded by adding the first hit value corresponding to the found prohibited word and the second hit value corresponding to the associated prohibited word to determine a total hit value; and

determining whether the total hit value is greater than the threshold value.

9. The computerized method of adjudicating text against a policy of claim 1 further comprising normalizing the text to be adjudicated, the normalizing comprising:

identifying an attribute of the text as an attempt to conceal information; and

removing the attribute;

wherein the attribute comprises a special character, formatting, and encoding.

10. The computerized method of adjudicating text against a policy of claim 1, wherein the determining whether a prohibited word is present in the text to be adjudicated by processing the text to be adjudicated and the system datastructure corresponding to the selected system policy comprises:

creating a temporary datastructure by duplicating the system datastructure corresponding to the selected system policy;

selecting a group of words from the text to be adjudicated, the number of words in the group corresponding to a number of processing threads of a processor;

simultaneously processing the selected group of words, the processing further comprising;

spawning a processor thread for each word in the selected groups of words;

identifying a linked list in the temporary datastructure that corresponds to the first letter of each word in the group of words;

searching the identified linked list for the word;

recording the word in the head linked list of the temporary datastructure if the word is found in the identified linked list; and

repeating the selecting a group of words and simultaneously processing the selected group of words until every word in the text to be adjudicated has been processed.

11. The computerized method of adjudicating text against a policy of claim 1 further comprising sending a response message corresponding to whether a prohibited word is present in the text to be adjudicated, the response message comprising a Simple Object Access Protocol message.

12. The computerized method of adjudicating text against a policy of claim 8 further comprising sending a response message corresponding to whether the threshold value has been exceeded, the response message comprising a Simple Object Access Protocol message.

13. A computerized method of adjudicating text against a policy comprising:

receiving one or more system policies, each system policy comprising;

one or more prohibited words;

one or more groups of associated prohibited words;

a first hit value for each prohibited word, the hit value indicating the significance of the prohibited word;

a second hit value for each associated prohibited word, the hit value indicating the significance of the associated prohibited word;

a proximity value corresponding to each associated prohibited word; and

a threshold value for each group of associated prohibited words, the threshold value indicating a limit on the hit values for the associated prohibited words in the group;

creating a system datastructure for each received system policy, each system datastructure comprising a hash table, the hash table comprising a plurality of linked lists corresponding the letters of the alphabet;

receiving an input message comprising a text to be adjudicated;

selecting a system policy from the one or more received system policies based on the input message; and

determining whether the text contains any prohibited associated words from the one or more prohibited associated words by processing the text and the system datastructure corresponding to the selected system policy.

14. The computerized method of adjudicating text against a policy of claim 13, wherein the input message comprises a Simple Object Access Protocol message.

15. The computerized method of adjudicating text against a policy of claim 13, wherein:

each linked list in the system datastructure comprises the prohibited words from the system policy that begin with a corresponding letter of the alphabet and the first hit value associated with each prohibited word; and

each system datastructure further comprises the associated prohibited words, the second hit values, and the threshold values.

16. The computerized method of adjudicating text against a policy of claim 13, wherein the determining whether the text contains any prohibited associated words from the one or more prohibited associated words by processing the text and the system datastructure corresponding to the selected system policy comprises:

creating a temporary datastructure by duplicating the system datastructure corresponding to the selected system policy;

selecting a word from the text to be adjudicated;

identifying a linked list in the temporary datastructure that corresponds to the first letter of the selected word;

searching the identified linked list for the selected word;

recording the selected word as a found prohibited word in a head linked list of the temporary datastructure if the selected word is found in the identified linked list;

recording a proximity string of text associated with each prohibited word in the head linked list of the temporary datastructure if the selected word is found in the identified linked list;

searching the proximity string of text for the associated prohibited word; and

determining whether the proximity string of text comprises the associated prohibited word.

17. The computerized method of adjudicating text against a policy of claim 16 further comprising determining whether the threshold value has been exceeded, the determining comprising:

adding the first hit value corresponding to the found prohibited word and the second hit value corresponding to the associated prohibited word to determine a total hit value; and

determining whether the total hit value is greater than the threshold value.

18. The computerized method of adjudicating text against a policy of claim 13 further comprising normalizing the text to be adjudicated, the normalizing comprising:

identifying an attribute of the text as an attempt to conceal information; and

removing the attribute;

wherein the attribute comprises a special character, formatting, and encoding.

19. The computerized method of adjudicating text against a policy of claim 13 further comprising sending a response message corresponding to whether a prohibited associated word is present in the text to be adjudicated, the response message comprising a Simple Object Access Protocol message.

20. The computerized method of adjudicating text against a policy of claim 17 further comprising sending a response message corresponding to whether the threshold value has been exceeded, the response message comprising a Simple Object Access Protocol message.