Recognizing spam email

• US 8,549,081 B2
• Filed: 12/15/2008
• Issued: 10/01/2013
• Est. Priority Date: 02/03/2006
• Status: Expired due to Fees

• Current Assignee: International Business Machines Corporation
• Original Assignee: International Business Machines Corporation
• Inventors: Leiba, Barry, Ossher, Joel, Rajan, Vadakkedathu Thomas, Segal, Richard, Wegman, Mark N.
• Primary Examiner(s): Ibrahim, Mohamed
• Assistant Examiner(s): RICHARDSON, THOMAS W
• Application Number:
  US12/335,339
• Publication Number:
  US 20090094342A1
• Time in Patent Office: 1,751 Days
• Field of Search: 709204-207, 370/93.01
• US Class Current: 709/206
• CPC Class Codes: G06Q 10/107 Computer-aided management of electronic mailing [e-mailing]

Abstract

A system includes at least one router for routing email messages from a sender node to a destination node; a system memory; a network interface; a database; and a processor configured for: extracting the delivery path information from the email message; determining a network path for the email message using delivery path information; comparing the delivery path information with the plurality of prior delivery paths; determining a measure of similarity between the network path of the received email message and one or more of the plurality of prior email paths; and determining a spam score for the email received, based on the measure of similarity.

Claims

1. A method for estimating a probability of spamminess for email messages, comprising:

using a processor device, performing steps of;

reviewing a sequence of IP addresses used in transmission of an email message, beginning with a last IP address and proceeding backward to an originating IP address, with intermediate IP address in between;

comparing a current IP address in the sequence of IP addresses with an IP address in a training set;

when the current IP address does not match any IP address in the training set, combining statistics of nearby IP addresses by;

building a tree of known IP addresses, wherein a root of said tree has up to 256 first level sub trees, each first level sub tree corresponding to various possible first bytes of an IP address;

wherein each node n in the tree represents an IP address; and

at each node n in the tree;

storing a count Sn of spam messages in which the IP address the node n represents has appeared;

storing a count NSn of non-spam messages in which the IP address the node n represents has appeared; and

computing a ratio that is a measure of spaminess s of the node n;

wherein the ratio of the measure of spaminess s of the node n is computed by dividing the total number of messages that have come through the address, which is Sn/(Sn+NSn);

determining an overall score for the message based on the ratio of spaminess of each node along the message path;

wherein the overall score is calculated by calculating a weighted average of the spaminess s of the nodes, with the weight equal to 1/(s*(1−

s); and

determining a probability that the message is spam based on the overall score being greater than a defined spam threshold.

2. The method of claim 1 wherein each node n in the tree represents a range of IP addresses.

3. The method of claim 1 wherein building the tree of known IP addresses further comprises:

building up to 256 second level sub trees for each first level sub tree, corresponding to various possible second bytes of an IP address;

building up to 256 third level sub trees for each second level sub tree, corresponding to various possible third bytes of an IP address; and

building up to 256 fourth level sub trees for each third level sub tree, corresponding to various possible fourth bytes of an IP address.

4. The method of claim 3 wherein building the tree of known IP addresses further comprises:

not including nodes n in the first level sub trees for any first bytes corresponding to unknown IP addresses;

not including nodes n in the second level sub trees for any second bytes corresponding to unknown IP addresses;

not including nodes n in the third level sub trees for any third bytes corresponding to unknown IP addresses; and

not including nodes n in the fourth level sub trees for any fourth bytes corresponding to unknown IP addresses.

5. The method of claim 1 wherein computing the ratio further comprises:

adding an artificial new root with a score of 0.5;

beginning from the sub tree that contains the last IP address, computing a score between 0 and 1 for each IP address and then combining that score with a score for the next IP address until reaching the artificial new root.

6. The method of claim 5 further comprising:

establishing a credibility value for each intermediate address; and

ignoring remaining addresses in the sequence when an intermediate address has a low credibility value.

7. The method claim 6 further comprising:

keeping separate statistics for originating addresses and intermediate addresses.

8. An information processing system for estimating a probability of spamminess for email messages, comprising:

a system memory;

a processor device operably coupled with the system memory and performing steps of;

reviewing a sequence of IP addresses used in transmission of an email message, beginning with a last IP address and proceeding backward to an originating IP address, with intermediate IP address in between;

comparing a current IP address in the sequence of IP addresses with an IP address in a training set;

when the current IP address does not match any IP address in the training set, combining statistics of nearby IP addresses by;

building a tree of known IP addresses, wherein a root of said tree has up to 256 first level sub trees, each first level sub tree corresponding to various possible first bytes of an IP address;

wherein each node n in the tree represents an IP address; and

at each node n in the tree;

storing a count Sn of spam messages in which the IP address the node n represents has appeared;

storing a count NSn of non-spam messages in which the IP address the node n represents has appeared; and

computing a ratio that is a measure of spamminess s of the node n;

wherein the ratio of the measure of seaminess s of the node n is computed by dividing the total number of messages that have come through the address, which is Sn/(Sn+NSn);

determining an overall score for the message based on the ratio of seaminess of each node along the message path;

wherein the overall score is calculated by calculating a weighted average of the seaminess s of the nodes, with the weight equal to 1/(s*(1−

s); and

determining a probability that the message is spam based on the overall score being greater than a defined spam threshold.

9. The information processing system of claim 8 wherein each node n in the tree represents a range of IP addresses.

10. The information processing system of claim 8 wherein the step of building the tree of known IP addresses further comprises:

building up to 256 second level sub trees for each first level sub tree, corresponding to various possible second bytes of an IP address;

building up to 256 third level sub trees for each second level sub tree, corresponding to various possible third bytes of an IP address; and

building up to 256 fourth level sub trees for each third level sub tree, corresponding to various possible fourth bytes of an IP address.

11. The information processing system of claim 10 wherein the step of building the tree of known IP addresses further comprises:

not including nodes n in the first level sub trees for any first bytes corresponding to unknown IP addresses;

not including nodes n in the second level sub trees for any second bytes corresponding to unknown IP addresses;

not including nodes n in the third level sub trees for any third bytes corresponding to unknown IP addresses; and

not including nodes n in the fourth level sub trees for any fourth bytes corresponding to unknown IP addresses.

12. The information processing system of claim 8 wherein the step of computing the ratio further comprises:

adding an artificial new root with a score of 0.5;

beginning from the sub tree that contains the last IP address, computing a score between 0 and 1 for each IP address and then combining that score with a score for the next IP address until reaching the artificial new root.

13. The information processing system of claim 12 wherein the processor device further performs steps of:

keeping separate statistics for originating addresses and intermediate addresses;

establishing a credibility value for each intermediate address; and

ignoring remaining addresses in the sequence when an intermediate address has a low credibility value.

14. A non-transitory computer readable storage medium comprising program code for estimating a probability of spamminess for email messages that, when executed, enables a computer to perform steps of:

reviewing a sequence of IP addresses used in transmission of an email message, beginning with a last IP address and proceeding backward to an originating IP address, with intermediate IP address in between;

comparing a current IP address in the sequence of IP addresses with an IP address in a training set;

when the current IP address does not match any IP address in the training set, combining statistics of nearby IP addresses by;

building a tree of known IP addresses, wherein a root of said tree has up to 256 first level sub trees, each first level sub tree corresponding to various possible first bytes of an IP address;

wherein each node n in the tree represents an IP address; and

at each node n in the tree;

storing a count Sn of spam messages in which the IP address the node n represents has appeared;

storing a count NSn of non-spam messages in which the IP address the node n represents has appeared; and

computing a ratio that is a measure of spaminess s of the node n;

wherein the ratio of the measure of spaminess s of the node n is computed by dividing the total number of messages that have come through the address, which is Sn/(Sn+NSn);

determining an overall score for the message based on the ratio of spaminess of each node along the message path;

wherein the overall score is calculated by calculating a weighted average of the seaminess s of the nodes, with the weight equal to 1/(s*(1−

s); and

determining a probability that the message is spam based on the overall score being greater than a defined spam threshold.

15. The non-transitory computer readable storage medium of claim 14 wherein building the tree of known IP addresses further comprises:

building up to 256 second level sub trees for each first level sub tree, corresponding to various possible second bytes of an IP address;

building up to 256 third level sub trees for each second level sub tree, corresponding to various possible third bytes of an IP address; and

building up to 256 fourth level sub trees for each third level sub tree, corresponding to various possible fourth bytes of an IP address.

16. The non-transitory computer readable storage medium of claim 15 wherein building the tree of known IP addresses further comprises:

not including nodes n in the first level sub trees for any first bytes corresponding to unknown IP addresses;

not including nodes n in the second level sub trees for any second bytes corresponding to unknown IP addresses;

not including nodes n in the third level sub trees for any third bytes corresponding to unknown IP addresses; and

not including nodes n in the fourth level sub trees for any fourth bytes corresponding to unknown IP addresses.

17. The non-transitory computer readable storage medium of claim 14 wherein computing the ratio further comprises:

adding an artificial new root with a score of 0.5;

beginning from the sub tree that contains the last IP address, computing a score between 0 and 1 for each IP address and then combining that score with a score for the next IP address until reaching the artificial new root.