Sender reputations for spam prevention

• US 7,610,344 B2
• Filed: 12/13/2004
• Issued: 10/27/2009
• Est. Priority Date: 12/13/2004
• Status: Active Grant

• Current Assignee: Microsoft Technology Licensing LLC (Microsoft Corporation)
• Original Assignee: Microsoft Corporation
• Inventors: Howell, Nathan D, Mehr, John D., Rehfuss, Paul S
• Primary Examiner(s): Jaroenchonwanit; Bunjob
• Assistant Examiner(s): Fan; Hua
• Application Number:
  US11/011,462
• Publication Number:
  US 20060168024A1
• Time in Patent Office: 1,779 Days
• Field of Search: 709206-207, 709/202, 706/16, 707/10, 715/205
• US Class Current: 709/206
• CPC Class Codes: H04L 51/212 using filtering or selective blocking
• CPC Class Codes: H04L 63/14 for detecting or protecting against malicious traffic
• CPC Class Codes: H04L 63/1491 using deception as countermeasure, e.g. honeypots, honeynets, decoys or entrapment
• CPC Class Codes: H04L 67/025 for remote control or remote monitoring of applications
• CPC Class Codes: H04L 67/306 User profiles

Abstract

Techniques are presented for assigning reputations to email senders. In one implementation, real-time statistics and heuristics are constructed, stored, analyzed, and used to formulate a sender reputation level for use in evaluating and controlling a given sender'"'"'s connection to an message transfer agent or email recipient. A sender with an unfavorable reputation may be denied a connection before resources are spent receiving and processing email messages from the sender. A sender with a favorable reputation may be rewarded by having safeguards removed from the connection, which also saves system resources. The statistics and heuristics may include real-time analysis of traffic patterns and delivery characteristics used by an email sender, analysis of content, and historical or time-sliced views of all of the above.

Claims

1. A method, comprising:

evaluating, by a mail transfer agent (MTA) independent of a mail recipient, a sender of an email, using multiple characteristics of an email delivery to establish a reputation for the sender of the email, wherein the sender of the email is connecting to the MTA,

wherein evaluating comprises;

monitoring, real-time, traffic patterns between the sender of the email and the MTA,

collecting sender-specific information and heuristics from the email delivery, wherein the collecting occurs real-time at a conclusion of a Simple Mail Transfer Protocol (SMTP) session, and wherein the sender-specific information and heuristics include;

whether a domain name provided includes one of .edu, .gov, or .mil;

or whether the domain appears to point to a private computer,

applying, in combination with the sender-specific information and heuristics, a machine learning process to generate an integer, the integer representative of a probabilistic reputation for the sender of the email, wherein the machine learning process classifies results of the evaluation of the delivery characteristics to establish the reputation,

establishing a baseline reputation for the sender, comprising;

evaluating a content of each email message from the sender;

evaluating a ratio of emails that include favorable content to emails that include unfavorable content, per unit of time; and

evaluating changes in the ratio over multiple units of time,

comparing a first group of the evaluated delivery characteristics evaluated during a first time period with a second group of the evaluated delivery characteristics evaluated during a second time period to detect a change in a delivery behavior of the sender, wherein detecting a sudden change in the delivery behavior of the sender is an indication of malicious activity, malicious activity including a machine or a mail server being compromised,

wherein the sudden change in the delivery behavior of the sender comprises;

an abrupt onset or an abrupt abandonment of malicious spamming behavior; and

using a trainable filter to perform the evaluating multiple characteristics of an email delivery to establish the reputation for the sender;

training the trainable filter by analyzing email delivery used by multiple senders, the training occurring offline, outside of a system using the filter; and

controlling a connection with the sender based on the reputation.

2. The method as recited in claim 1, wherein said evaluating multiple characteristics includes determining a probability that the delivery is similar to a delivery of unsolicited commercial email resulting in an unfavorable reputation.

3. The method as recited in claim 1, wherein establishing a reputation includes:

evaluating a message content in addition to said evaluating multiple characteristics of a delivery; and

determining a similarity between the message content and an unsolicited commercial email content.

4. The method as recited in claim 1, wherein said evaluating multiple characteristics includes:

evaluating multiple characteristics of multiple delivery processes associated with multiple email messages from the sender;

counting instances of similarity between the evaluated characteristics and characteristics of unsolicited commercial email delivery; and

basing the reputation on the counted instances.

5. The method as recited in claim 1, wherein said evaluating multiple characteristics includes:

evaluating multiple delivery characteristics of multiple emails from the sender; and

establishing the reputation based on a statistical comparison between the evaluated multiple delivery characteristics and delivery characteristics of email sent by at least one different sender.

6. The method as recited in claim 1, wherein said evaluating multiple characteristics includes:

evaluating multiple delivery characteristics of multiple emails from the sender; and

comparing a quantity of the evaluated delivery characteristics with a threshold to determine a reputation.

7. The method as recited in claim 1, wherein said evaluating multiple characteristics includes:

evaluating multiple delivery characteristics of multiple emails from the sender; and

comparing a statistical distribution of the evaluated characteristics with a distribution profile of delivery characteristics associated with a sender of unsolicited commercial email.

8. The method as recited in claim 1, wherein said evaluating multiple characteristics includes:

evaluating multiple delivery characteristics of multiple emails from the sender; and

comparing a statistical distribution of the evaluated characteristics with a norm distribution profile of delivery characteristics associated with a mixture of different email senders.

9. The method as recited in claim 1, wherein controlling a connection with the sender comprises proactively filtering email from a sender with an unfavorable reputation.

10. The method as recited in claim 1, wherein controlling a connection with the sender comprises denying an SMTP simple mail transfer protocol connection with the sender if the sender has an unfavorable reputation.

11. The method as recited in claim 1, wherein controlling a connection with the sender comprises terminating an SMTP simple mail transfer protocol connection with the sender if the sender has an unfavorable reputation.

12. The method as recited in claim 1, wherein controlling a connection with the sender comprises removing a safeguard from the connection if the sender has a favorable reputation.

13. The method as recited in claim 12, wherein the safeguard comprises a spam filter.

14. The method as recited in claim 1, wherein one of the characteristics of email delivery to be used as a heuristic for said establishing a reputation comprises an open proxy status of the sender.

15. The method as recited in claim 1, wherein one of the characteristics of email delivery to be used as a heuristic for said establishing a reputation comprises a number of unique values provided by the sender in HELO and EHLO commands of SMTP simple mail transfer protocol.

16. The method as recited in claim 1, wherein one of the characteristics of email delivery to be used as a heuristic for said establishing a reputation comprises a number of times the sender submits a HELO or a EHLO command of SMTP simple mail transfer protocol that includes an Internet protocol address that does not match the originating Internet protocol address of the SMTP session.

17. The method as recited in claim 1, wherein one of the characteristics of email delivery to be used as a heuristic for said establishing a reputation comprises a number of times the sender submits a HELO or a EHLO command of SMTP simple mail transfer protocol that includes a domain name that is included in a list of locally supported domains on a receiving message transfer agent.

18. The method as recited in claim 1, wherein one of the characteristics of email delivery to be used as a heuristic for said establishing a reputation comprises a number of times the sender submits a DATA command of SMTP simple mail transfer protocol followed by no subsequent information before being terminated.

19. The method as recited in claim 18, further comprising determining if said no information followed the DATA command by measuring a size of a received header and subtracting the size from an overall size of the of information included in the DATA command.

20. The method as recited in claim 1, further comprising assigning a reputation based on a single email message received from the sender, wherein the trainable filter compares delivery characteristics of the single email message to delivery characteristics of the multiple senders.

21. A sender reputation level engine, comprising:

a traffic monitor to connect to an email network and monitor delivery of email;

a sender analysis engine to gather heuristic indications associated with the email delivery process used by each sender of email, wherein each heuristic indication relates a probability that a sender sends malicious email or unsolicited commercial email, the gathering occurring real-time at a conclusion of a Simple Mail Transfer Protocol (SMTP) session,

wherein the heuristic indications include;

whether a domain name provided includes one of .edu, .gov, or .mil;

or whether the domain appears to point to a private computer,

the sender analysis engine further configured to compare a first group of delivery characteristics evaluated during a first time period with a second group of delivery characteristics evaluated during a second time period to detect a change in a delivery behavior of the sender, wherein detecting a sudden change in the delivery behavior of the sender is an indication of malicious activity, malicious activity including a machine or a mail server being compromised,

wherein the sudden change in the delivery behavior of the sender comprises;

an abrupt onset or an abrupt abandonment of malicious spamming behavior; and

a statistics engine to determine a reputation level for each sender from statistical analysis of the gathered heuristic indications, the statistics engine comprising a machine learning process to generate an integer, the integer representative of a probabilistic reputation for a sender of an email, wherein a sender of malicious email or unsolicited commercial email is allotted an unfavorable reputation level.

22. The sender reputation level engine as recited in claim 21, further comprising a mail blocker to block communication from senders with unfavorable reputations.

23. The sender reputation level engine as recited in claim 21, further comprising a sender reputation database to store reputations and reputation statistics of multiple senders of email.

24. A computing device comprising the sender reputation level engine as recited in claim 21.

25. A system, comprising:

memory;

one or more processors operatively coupled to the memory;

means for evaluating, by a mail transfer agent (MTA) independent of a mail recipient, a sender of an email, using multiple characteristics of an email delivery to establish a reputation for the sender of the email based on the evaluated characteristics;

means for monitoring, real-time, traffic patterns between the sender of the email and the MTA,

means for collecting sender-specific information and heuristics from the email delivery, wherein the collecting occurs real-time at a conclusion of a Simple Mail Transfer Protocol (SMTP) session, and wherein the sender-specific information and heuristics include;

whether a domain name provided includes one of .edu, .gov, or .mil;

or whether the domain appears to point to a private computer,

means for comparing a first group of delivery characteristics evaluated during a first time period with a second group of delivery characteristics evaluated during a second time period to detect a change in a delivery behavior of the sender, wherein detecting a sudden change in the delivery behavior of the sender is an indication of malicious activity, malicious activity including a machine or a mail server being compromised, wherein the sudden change in the delivery behavior of the sender comprises;

an abrupt onset or an abrupt abandonment of malicious spamming behavior;

means for applying, in combination with the said sender-specific information and heuristics, a machine learning process to generate an integer, the integer representative of a probabilistic reputation for the sender of the email; and

means for controlling a connection with the sender based on the reputation.

26. The system as recited in claim 25, wherein the means for establishing a reputation includes means for comparing a quantity of the evaluated delivery characteristics with a threshold to determine a reputation.

27. The system as recited in claim 26, wherein the means for establishing a reputation includes means for comparing a statistical distribution of the evaluated characteristics with a distribution profile of delivery characteristics associated with a sender of unsolicited commercial email.

28. The system as recited in claim 25, wherein the means for establishing a reputation includes means for comparing a statistical distribution of the evaluated characteristics with a norm of delivery characteristics associated with a mixture of different email senders.

29. The system as recited in claim 25, wherein the means for controlling a connection includes means for proactively filtering email from a sender with an unfavorable reputation.

30. The system as recited in claim 25, wherein the means for controlling a connection includes means for denying an SMTP simple mail transfer protocol connection with the sender if the sender has an unfavorable reputation.

31. The system as recited in claim 25, wherein the machine learning process classifies results of the evaluation of the delivery characteristics.

32. The system as recited in claim 25, farther comprising means for comparing a first group of the evaluated delivery characteristics evaluated during a first time period with a second group of the evaluated delivery characteristics evaluated during a second time period to detect a change in a delivery behavior of the sender.

33. A computer-readable storage medium including instructions capable of being read by a computing device to execute actions, including:

evaluating, by a mail transfer agent (MTA) independent of a mail recipient, a sender of email messages, using aspects of email delivery used by the sender of email messages;

monitoring, real-time, traffic patterns between the sender of the email messages and the MTA,

collecting sender-specific information and heuristics from the email delivery, wherein the collecting occurs real-time at a conclusion of a Simple Mail Transfer Protocol (SMTP) session,

counting, by a message counter, the number of messages received from the sender;

once a first administrator-specified number of messages has been counted by the message counter, applying, in combination with the said sender-specific information and heuristics, a machine learning process to generate an integer, the integer representative of a probabilistic reputation for the sender of the email messages;

comparing a statistical distribution of the evaluated aspects with a profile of delivery characteristics associated with a sender of unsolicited commercial email;

comparing a first group of delivery characteristics evaluated during a first time period with a second group of delivery characteristics evaluated during a second time period to detect a change in a delivery behavior of the sender, wherein detecting a sudden change in the delivery behavior of the sender is an indication of malicious activity, malicious activity including a machine or a mail server being compromised,

wherein the sudden change in the delivery behavior of the sender comprises;

an abrupt onset or an abrupt abandonment of malicious spamming behavior;

establishing a reputation for the sender based on said applying and said comparing; and

once a second administrator-specified number of messages has been counted by the message counter, controlling a connection with the sender based on the reputation.

34. The readable medium as recited in claim 33, further including instructions to use an open proxy status of the sender as one of the aspects of email delivery to be used as a heuristic for establishing the reputation.

35. The readable medium as recited in claim 33, further including instructions to use a quantity of unique values provided by the sender in HELO and EHLO commands of SMTP simple mail transfer protocol as one of the aspects of email delivery to be used as a heuristic for establishing the reputation.

36. The readable medium as recited in claim 33, further including instructions to use a number of times the sender submits a HELO or a EHLO command of SMTP simple mail transfer protocol that includes an Internet protocol address that does not match the originating Internet protocol address of the SMTP session as one of the aspects of email delivery to be used as a heuristic for establishing the reputation.

37. The readable medium as recited in claim 33, further including instructions to use a number of times the sender submits a HELO or a EHLO command of SMTP simple mail transfer protocol that includes a domain name that is included in a list of locally supported domains on a receiving message transfer agent as one of the aspects of email delivery to be used as a heuristic for establishing the reputation.

38. The readable medium as recited in claim 33, further including instructions to use a number of times the sender submits a DATA command of SMTP simple mail transfer protocol followed by no subsequent information before being terminated as one of the aspects of email delivery to be used as a heuristic for establishing the reputation.

39. The readable medium as recited in claim 33, further including instructions to establish a baseline reputation for the sender, comprising:

evaluating a content of each email message from the sender;

evaluating a ratio of emails that include favorable content to emails that include unfavorable content, per unit of time; and

evaluating changes in the ratio over multiple units of time.

40. The readable medium as recited in claim 33, further including instructions to use a trainable filter to perform said evaluating multiple aspects of an email delivery to establish a reputation for the sender.

41. The readable medium as recited in claim 33, further including instructions to train the trainable filter by analyzing email delivery used by multiple senders.

42. The readable medium as recited in claim 41, further including instructions to assign a reputation based on a single email message received from the sender, wherein the trainable filter compares delivery characteristics of the single email message to delivery characteristics of the multiple senders.