HASH-BASED SYSTEMS AND METHODS FOR DETECTING AND PREVENTING TRANSMISSION OF UNWANTED E-MAIL

• US 20090132669A1
• Filed: 10/09/2008
• Published: 05/21/2009
• Est. Priority Date: 06/19/2000
• Status: Active Grant

• Current Assignee: Stragent, LLC (Oso IP, LLC)
• Original Assignee: Stragent, LLC (Oso IP, LLC)
• Inventors: Milliken, Walter Clark, Milligan, Stephen Douglas, Strayer, William Timothy
• Granted Patent: US 8,204,945 B2
• Time in Patent Office: Days
• US Class Current: 709/206
• CPC Class Codes: G06F 21/562 Static detection
• CPC Class Codes: H04L 51/212 using filtering or selective blocking
• CPC Class Codes: H04L 63/145 the attack involving the propagation of malware through the network, e.g. viruses, trojans or worms

Abstract

A system (120) detects transmission of potentially unwanted e-mail messages. The system (120) may receive e-mail messages and generate hash values based on one or more portions of the e-mail messages. The system (120) may then determine whether the generated hash values match hash values associated with prior e-mail messages. The system (120) may determine that one of the e-mail messages is a potentially unwanted e-mail message when one or more of the generated hash values associated with the e-mail message match one or more of the hash values associated with the prior e-mail messages.

Claims

1. A method for detecting transmission of potentially unwanted e-mail messages, comprising:

receiving a plurality of e-mail messages;

generating hash values, as generated hash values, based on one or more portions of the plurality of e-mail messages;

determining whether the generated hash values match hash values associated with prior e-mail messages; and

determining that one of the plurality of e-mail messages is a potentially unwanted e-mail message when one or more of the generated hash values associated with the one of the plurality of e-mail messages match one or more of the hash values associated with the prior e-mail messages.

2. The method of claim 1, wherein the generating hash values includes:

performing a plurality of hashes on a plurality of variable-sized blocks of a main text of the plurality of e-mail messages.

3. The method of claim 1, wherein the generating hash values includes:

performing a plurality of hashes on a plurality of fixed-sized blocks of a main text of the plurality of e-mail messages.

4. The method of claim 1, wherein the generating hash values includes:

performing a plurality of hashes on a main text of the plurality of e-mail messages using a plurality of different hash functions.

5. The method of claim 1, wherein the generating hash values includes:

performing a plurality of hashes on a main text of the plurality of e-mail messages using a same hash function.

6. The method of claim 1, wherein the generating hash values includes:

attempting to expand an attachment of the plurality of e-mail messages, and

hashing the attachment after attempting to expand the attachment.

7. The method of claim 1, wherein the generating hash values includes:

performing a plurality of hashes on a plurality of variable-sized blocks of an attachment of the plurality of e-mail messages.

8. The method of claim 1, wherein the generating hash values includes:

performing a plurality of hashes on a plurality of fixed-sized blocks of an attachment of the plurality of e-mail messages.

9. The method of claim 1, wherein the generating hash values includes:

performing a plurality of hashes on an attachment of the plurality of e-mail messages using a plurality of different hash functions.

10. The method of claim 1, wherein the generating hash values includes:

performing a plurality of hashes on an attachment of the plurality of e-mail messages using a same hash function.

11. The method of claim 1, further comprising:

comparing the generated hash values to hash values corresponding to known unwanted e-mails.

12. The method of claim 11, wherein the known unwanted e-mails include at least one of e-mails containing a virus, e-mails containing a worm, and unsolicited commercial e-mails.

13. The method of claim 1, wherein the generating hash values includes:

hashing at least one of a main text and an attachment to generate one or more first hash values, and

hashing a concatenation of first and second header fields to generate a second hash value.

14. The method of claim 13, wherein the first and second header fields include a From header field and a To header field.

15. The method of claim 13, wherein the determining whether the generated hash values match hash values associated with prior e-mail messages includes:

determining a first suspicion count based on a number of the hash values associated with the prior e-mail messages that match the one or more first hash values, and

determining a second suspicion count based on a number of the hash values associated with the prior e-mail messages that match the second hash value.

16. The method of claim 15, wherein the determining that one of the plurality of e-mail messages is a potentially unwanted e-mail message includes:

determining that the one of the plurality of e-mail messages is a potentially unwanted e-mail message when the first suspicion count is significantly higher than the second suspicion count.

17. The method of claim 1, further comprising:

taking remedial action when the one of the plurality of e-mail messages is a potentially unwanted e-mail message, the taking remedial action including at least one of;

discarding the one of the plurality of e-mail messages,

bouncing the one of the plurality of e-mail messages,

marking the one of the plurality of e-mail messages with a warning,

subjecting the one of the plurality of e-mail messages to a virus or worm detection process,

creating a notification message, and

generating a suspicion score for the one of the plurality of e-mail messages and using the suspicion score to identify further processing for the one of the plurality of e-mail messages.

18. The method of claim 1, further comprising:

generating a suspicion score for the plurality of e-mail messages based on a result of the determination of whether the generated hash values match hash values associated with prior e-mail messages; and

taking remedial action when the one of the plurality of e-mail messages is a potentially unwanted e-mail message, the taking remedial action including;

determining whether a newly received e-mail message exceeds a mail quota,

identifying an earlier-received e-mail message with a highest suspicion score,

determining whether the suspicion score of the newly received e-mail message is lower than the suspicion score of the earlier-received e-mail message when the newly received e-mail message exceeds the mail quota,

deleting the earlier-received e-mail message when the suspicion score of the newly received e-mail message is lower than the suspicion score of the earlier-received e-mail message, and

storing the newly received e-mail message.

19. The method of claim 1, wherein the generating hash values and the determining whether the hash values match hash values associated with prior e-mail messages are performed incrementally as the plurality of e-mail messages are being received.

20. The method of claim 19, further comprising:

generating a suspicion score for the plurality of e-mail messages based on a result of the determination of whether the generated hash values match hash values associated with prior e-mail messages; and

taking remedial action when the suspicion score of an e-mail message of the plurality of e-mail messages is above a threshold, the taking remedial action including rejecting the e-mail message.

21. The method of claim 20, wherein the rejecting occurs before the e-mail message is completely received.

22. The method of claim 1, further comprising:

comparing the generated hash values to known legitimate mailing lists; and

passing the plurality of e-mail messages without further examination when the generated hash values match one or more of the known legitimate mailing lists.

23. The method of claim 22, wherein the comparing the generated hash values includes:

determining whether the plurality of e-mail messages originated from the known legitimate mailing lists.

24. The method of claim 1, wherein the generating hash values includes:

hashing a main text to generate a first hash value, and

hashing sender-related header fields to generate one or more second hash values.

25. The method of claim 24, wherein the sender-related header fields include at least one of a From header field, a Sender header field, and a Reply-To header field.

26. The method of claim 24, wherein the determining whether the generated hash values match hash values associated with prior e-mail messages includes:

determining a first suspicion count based on a number of the hash values associated with the prior e-mail messages that match the first hash value, and

determining one or more second suspicion counts based on a number of the hash values associated with the prior e-mail messages that match the one or more second hash values.

27. The method of claim 26, wherein the determining that one of the plurality of e-mail messages is a potentially unwanted e-mail message includes:

determining that the one of the plurality of e-mail messages is a potentially unwanted e-mail message when the first suspicion count is higher than the one or more second suspicion counts.

28. The method of claim 1, wherein the generating hash values includes:

hashing a main text of the plurality of e-mail messages to generate a main text hash, and

hashing at least one header field of the plurality of e-mail messages to generate at least one header hash.

29. The method of claim 28, wherein the determining whether the generated hash values match hash values associated with prior e-mail messages includes:

determining whether the main text hash matches a substantially higher number of the hash values associated with the prior e-mail messages than the at least one header hash; and

wherein the determining that one of the plurality of e-mail messages is a potentially unwanted e-mail message includes;

determining that the one of the plurality of e-mail messages is a potentially unwanted e-mail message when the main text hash matches a substantially higher number of the hash values associated with the prior e-mail messages than the at least one header hash.