Filtering Electronic Messages

• US 20150295869A1
• Filed: 04/14/2014
• Published: 10/15/2015
• Est. Priority Date: 04/14/2014
• Status: Abandoned Application

• Current Assignee: Microsoft Technology Licensing LLC (Microsoft Corporation)
• Original Assignee: Microsoft Corporation
• Inventors: Li, Weisheng, Chan, Kok Wai, Chen, Rui
• Application Number:
  US14/252,249
• Publication Number:
  US 20150295869A1
• Time in Patent Office: Days
• US Class Current: 1/1
• CPC Class Codes: G06F 40/30 Semantic analysis
• CPC Class Codes: H04L 51/212 using filtering or selective blocking

Abstract

Technologies are described herein for filtering of electronic messages. A method for filtering messages includes receiving an electronic message for transmission to a recipient, generating a fingerprint for the electronic message, determining if the electronic message is associated with a known cluster of previously transmitted electronic messages, and filtering the electronic message based on the determining. The fingerprint is a fixed length of appended bits selected from hash values determined by applying hash functions to separate textual words included in the electronic message.

Claims

1. A computer-implemented method for filtering electronic messages, the method comprising:

receiving an electronic message for transmission to a recipient;

generating a fingerprint for the electronic message, the fingerprint being a fixed length of appended bits selected from hash values determined from a plurality of hash functions applied to separate textual words included in the electronic message;

determining if the electronic message is associated with a known cluster of previously transmitted electronic messages; and

filtering the electronic message based on the determining.

2. The method of claim 1, wherein generating the fingerprint comprises:

removing noisy characters from the message;

dividing the message into a plurality of shingles absent the noisy characters;

performing the plurality of hash functions on each shingle of the plurality of shingles to create a plurality of hash values associated with each shingle; and

generating the fingerprint based on the plurality of hash functions.

3. The method of claim 2, wherein generating the fingerprint further comprises:

determining a final hash value for each hash value across all shingles of the plurality of shingles; and

selecting a predetermined number of bits from each final hash value as bits for the fingerprint.

4. The method of claim 3, wherein determining the final hash value comprises determining a minimum hash value associated with each hash function across all shingles of the plurality of shingles.

5. The method of claim 1, wherein determining if the electronic message is associated with a known cluster comprises:

dividing the fingerprint into a plurality of bit sequences; and

comparing each bit sequence of the plurality of bit sequences to an associated bin of bit sequences for the known clusters.

6. The method of claim 5, wherein the plurality of bit sequences are each a first length, and wherein each associated bin of bit sequences includes bit sequences of the first length.

7. The method of claim 1, further comprising classifying the known cluster based on message features of the known cluster if the electronic message is associated with a known cluster of previously transmitted electronic messages;

and

publishing an electronic mail filter configured to filter future messages received based on the classifying and the known cluster.

8. The method of claim 7, wherein the classifying the known cluster comprises:

counting the message features for the known cluster;

determining if an existing message classification exists based on the counting; and

if an existing message classification exists, publishing the classification and an associated fingerprint for the known cluster.

9. The method of claim 7, wherein the message features comprise origin and destination information associated with the known cluster.

10. The method of claim 7, the message classification comprises at least a classification that messages associated with the known cluster are noisy messages.

11. A computer-readable storage medium having computer executable instructions stored thereon which, when executed by a computer, cause the computer to:

receive an electronic message for transmission to a recipient;

generate a fingerprint for the electronic message, the fingerprint being a fixed length of appended bits selected from hash values determined from a plurality of hash functions applied to separate textual words included in the electronic message;

determine if the electronic message is associated with a known cluster of previously transmitted electronic messages;

classify the known cluster based on message features of the known cluster in response to determining the electronic message is associated with the known cluster; and

publish an electronic mail filter configured to filter future messages received based on the classification and the known cluster.

12. The computer-readable storage medium of claim 11, wherein generate the fingerprint comprises:

remove noisy characters from the message;

divide the message into a plurality of shingles absent the noisy characters;

perform the plurality of hash functions on each shingle of the plurality of shingles to create a plurality of hash values associated with each shingle; and

generate the fingerprint based on the plurality of hash functions.

13. The computer-readable storage medium of claim 12, wherein generate the fingerprint further comprises:

determine a final hash value for each hash value across all shingles of the plurality of shingles; and

select a predetermined number of bits from each final hash value as bits for the fingerprint.

14. The computer-readable storage medium of claim 13, wherein determine the final hash value comprises determining a minimum hash value associated with each hash function across all shingles of the plurality of shingles.

15. The computer-readable storage medium of claim 11, wherein determine if the electronic message is associated with a known cluster comprises:

divide the fingerprint into a plurality of bit sequences; and

compare each bit sequence of the plurality of bit sequences to an associated bin of bit sequences for the known clusters.

16. The computer-readable storage medium of claim 15, wherein the plurality of bit sequences are each a first length, and wherein each associated bin of bit sequences includes bit sequences of the first length.

17. The computer-readable storage medium of claim 11, wherein the electronic mail filter includes at least a portion of the fingerprint of the electronic message.

18. The computer-readable storage medium of claim 11, wherein classify the known cluster comprises:

count the message features for the known cluster;

determine if an existing message classification exists based on the counting; and

if an existing message classification exists, publish the classification and an associated fingerprint for the known cluster.

19. The computer-readable storage medium of claim 17, wherein the message features comprise origin and destination information associated with the known cluster and wherein the message classification comprises at least a classification that messages associated with the known cluster are noisy messages.

20. A mail processing system configured to distribute electronic messages from a plurality of client computers to a plurality of recipients, the system comprising:

at least one computer executing an electronic messaging service configured to receive the electronic messages from the plurality of client computers, the electronic messaging service further configured to

divide each message into a plurality of shingles absent noisy characters,

perform a plurality of hash functions on each shingle of the plurality of shingles to create a plurality of hash values associated with each shingle, and

generate a message fingerprint for each message based on the plurality of hash functions;

at least one computer executing a clustering service configured to receive each message fingerprint from the electronic messaging service, the clustering service further configured to,

divide each fingerprint into a plurality of bit sequences,

compare each bit sequence of the plurality of bit sequences to an associated bin of bit sequences for known clusters of previously transmitted electronic messages, and

determine if a similarity threshold between each fingerprint and the known clusters has been met; and

at least one computer executing a filtering agent configured to filter the electronic messages based on filter information received from the clustering service.