Systems and methods for detecting email spam and variants thereof

• US 8,495,737 B2
• Filed: 03/01/2011
• Issued: 07/23/2013
• Est. Priority Date: 03/01/2011
• Status: Active Grant

• Current Assignee: Zscaler Incorporated
• Original Assignee: Zscaler Incorporated
• Inventors: Sinha, Amit, Voit, Robert Louis, Kailash, Kailash, Raphel, Jose
• Primary Examiner(s): Bruckart, Benjamin R
• Assistant Examiner(s): Abedin, Normin
• Application Number:
  US13/038,144
• Publication Number:
  US 20120227104A1
• Time in Patent Office: 875 Days
• Field of Search: 709/22
• US Class Current: 726/22
• CPC Class Codes: H04L 51/212 using filtering or selective blocking
• CPC Class Codes: H04L 63/0227 Filtering policies mail message filtering H04L51/212
• CPC Class Codes: H04L 63/1441 Countermeasures against malicious traffic countermeasures against attacks on cryptographic mechanisms H04L9/002

Abstract

The present disclosure provides systems and methods for detecting email spam and variants thereof. The systems and methods are configured to detect spam messages and variations thereof for different senders and with slight differences within the message body. In an exemplary embodiment, an incoming message body (m) is converted to a sequence of successive word lengths (S_m): m->S_m, a comparison is performed between the sequence, S_m, and a plurality of stored sequences (S_k) of known spam messages, and the incoming message is flagged as spam based on the comparison. Further, the plurality of stored sequences, S_k, may be continually updated based on user feedback and other spam detection techniques. The systems and methods of the present invention may be implemented through a computer, such as a mail server, through a cloud-based security system, through a user'"'"'s computer via a software agent, and the like.

Claims

1. A computer-implemented spam detection method, comprising:

receiving an electronic mail message;

converting the electronic mail message into a sequence;

comparing the sequence with a plurality of sequences stored in a spam table;

determining if the electronic mail message is spam responsive to the comparing step;

determining the email message is not spam responsive to the comparing step, receiving an indication of spam related to the email message, and adding the sequence to the plurality of sequences in the spam table;

creating an initial spam table comprising the plurality of sequences and an aging factor associated with each of the plurality of sequences;

if the sequence is within a threshold of an edit distance of one of the plurality of sequences, resetting the aging factor for the one of the plurality of sequences; and

periodically removing aged sequences in the plurality of sequences based on the aging factor.

2. The computer-implemented spam detection method of claim 1, wherein the comparing step comprises:

determining an edit distance for the sequence to each of the plurality of sequences.

3. The computer-implemented spam detection method of claim 2, wherein the edit distance comprises a Levenshtein Distance.

4. The computer-implemented spam detection method of claim 3, wherein the sequence comprises a series of word lengths representing a number of characters in each word of the email message.

5. The computer-implemented spam detection method of claim 4, wherein the word lengths are determined based on delimiters comprising any of whitespaces, linefeeds, tabs, and other user specified delimiters.

6. The computer-implemented spam detection method of claim 1, wherein the indication of spam is based on any of flagging as spam by a user, sender reputation analysis, and message analysis comprising keywords, rule-based filtering, and statistical content filtering.

7. The computer-implemented spam detection method of claim 1, further comprising:

receiving updates for the spam table based upon remote spam detections.

8. The computer-implemented spam detection method of claim 1, further comprising:

determining a size of the sequence; and

limiting the comparing step to sequences in the spam table within a size ranged of the size of the sequence.

9. A spam detection system, comprising:

a data store;

a network interface communicatively coupled to a network;

a processor, wherein the data store, the network interface, and the processor are communicatively coupled therebetween, and wherein the processor is configured to;

convert an email message body to a sequence of successive word lengths;

compute an edit distance between the sequence and each of a plurality of known spam sequences; and

flag the email message as spam based upon one of the edit distances being within a threshold;

determine the email message is not spam responsive to the comparing step, receive an indication of spam related to the email message, and add the sequence of successive word lengths to the plurality of known spam sequences;

receive or create an initial spam table comprising the plurality of known spam sequences and an aging factor associated with each of the plurality of known spam sequences;

if the sequence of successive word lengths is within a threshold of an edit distance of one of the plurality of sequences, reset the aging factor for the one of the plurality of known spam sequences; and

periodically remove aged sequences in the plurality of known spam sequences based on the aging factor.

10. The spam detection system of claim 9, wherein the spam detection system comprises an email server communicatively coupled to a plurality of users and receiving email messages therefor.

11. The spam detection system of claim 9, wherein the sequence comprises a series of word lengths representing a number of characters in each word of the email message, and wherein the word lengths are determined based on delimiters comprising any of whitespaces, linefeeds, tabs and other user specified delimiters.

12. The spam detection system of claim 9, wherein the processor is configured to:

receive updates for the plurality of known spam sequences based upon remote spam detections; and

provide updates for the plurality of known spam sequences based upon local spam detections.

13. A network security system, comprising:

a processing node communicatively coupled to a user and to an external network, wherein the processing node comprises a data store storing security policy data for the user, data inspection engines configured to perform threat detection classification on content to the user from the external network, and a manager communicatively coupled to the data store and the data inspection engines; and

an authority node communicatively coupled to the processing node, wherein the authority node comprises a data store storing security policy data comprising a spam table;

wherein the processing node and the authority node are configured to detect email spam and update the spam table based thereon;

wherein the processing node is configured to;

receive an electronic mail message;

convert the electronic mail message into a sequence;

compare the sequence with a plurality of sequences stored in a spam table;

determine if the electronic mail message is spam responsive to the compare step;

determine the email message is not spam responsive to the comparing step, receive an indication of spam related to the email message, and add the sequence to the plurality of sequences in the spam table;

receive an initial spam table comprising the plurality of sequences and an aging factor associated with each of the plurality of sequences from the authority node;

if the sequence is within a threshold of an edit distance of one of the plurality of sequences, reset the aging factor for the one of the plurality of sequences; and

periodically remove aged sequences in the plurality of sequences based on the aging factor.

14. The network security system of claim 13, wherein the spam detection algorithm comprises:

converting each of the monitored email messages into a sequence of successive word lengths;

comparing the sequence of successive word lengths with sequences in the spam table; and

determining if each of the monitored email messages is spam responsive to the comparing step.

15. The network security system of claim 13, wherein the spam detection algorithm is configured to provide continual updates between the processing node and the authority node.