Message stream analysis for spam detection and filtering

• US 7,716,297 B1
• Filed: 01/30/2007
• Issued: 05/11/2010
• Est. Priority Date: 01/30/2007
• Status: Active Grant

• Current Assignee: Proofpoint Incorporated
• Original Assignee: Proofpoint Incorporated
• Inventors: Lewis, Steven, Yang, Yanyan, Panzer, Scott, Wittel, Greg
• Primary Examiner(s): Duong; Oanh
• Application Number:
  US11/700,473
• Time in Patent Office: 1,197 Days
• Field of Search: 709/206, 709/207, 709/223, 726/22, 726/26
• US Class Current: 709/206
• CPC Class Codes: H04L 51/212 using filtering or selective blocking

Abstract

A system and method are described for performing a correlative statistical analysis on a stream of email messages to identify new spam campaigns. For example, a method according to one embodiment of the invention comprises: extracting a series of patterns from a stream of incoming email messages; performing a correlation between the patterns to identify recurring patterns within the stream of email messages over a specified time period; dynamically updating a spam filtering engine to include a particular recurring pattern if the number of times the particular recurring pattern is detected within the specified time period is above a first specified threshold value.

Claims

1. A computer-implemented method for dynamically identifying a new spam campaign comprising:

identifying a series of patterns from a stream of incoming email messages;

performing a correlation between the series of patterns to identify recurring patterns within the stream of email messages over a specified time period;

dynamically updating a spam filtering engine by including a new feature related to a particular recurring pattern to the spam filtering engine if a number of times the particular recurring pattern is detected within the specified time period is above a first specified threshold value;

associating a spam weight with the new feature;

firing the new feature in response to detecting a new email message having the particular recurring pattern;

combining the spam weight of the new feature with spam weights of other features fired in response to receipt of the new email message;

generating a spam score based on the combined spam weights; and

making a determination as to whether the new email message is spam based on the spam score,

wherein the scam weight associated with the new feature is increased if the number of times the particular recurring pattern is detected in the stream of the incoming email messages within the specified time period is above a second specified threshold value.

2. The method as in claim 1 wherein one of the series of patterns comprises uniform resource locators (URLs) embedded within the email messages.

3. The method as in claim 1 wherein one of the series of patterns comprises source IP address or IP address blocks from which the messages originated.

4. The method as in claim 1 wherein dynamically updating the spam filtering engine comprises firing a rule associated with recurring patterns, the rule having a particular weight associated therewith.

5. A computer system having a memory for storing program code and a processor for processing the program code to perform the operations of:

identifying a series of patterns from a stream of incoming email messages;

performing a correlation between the series of patterns to identify recurring patterns within the stream of email messages over a specified time period;

dynamically updating a spam filtering engine by including a new feature related to a particular recurring pattern to the spam filtering engine if a number of times the particular recurring pattern is detected within the specified time period is above a first specified threshold value;

associating a spam weight with the new feature;

firing the new feature in response to detecting a new email message having the particular recurring pattern;

combining the spam weight of the new feature with spam weights of other features fired in response to receipt of the new email message;

generating a spam score based on the combined scam weights; and

making a determination as to whether the new email message is spam based on the spam score,

wherein the spam weight associated with the new feature is increased if the number of times the particular recurring pattern is detected in the stream of the incoming email messages within the specified time period is above a second specified threshold value.

6. The system as in claim 5 wherein one of the series of patterns comprises uniform resource locators (URLs) embedded within the email messages.

7. The system as in claim 5 wherein one of the series of patterns comprises source IP address or IP address blocks from which the messages originated.

8. The system as in claim 5 wherein dynamically updating the spam filtering engine comprises firing a rule associated with recurring patterns, the rule having a particular weight associated therewith.