Evaluating electronic mail messages based on probabilistic analysis

• US 7,921,063 B1
• Filed: 05/15/2007
• Issued: 04/05/2011
• Est. Priority Date: 05/17/2006
• Status: Active Grant

• Current Assignee: Ironport Systems (Cisco Systems, Inc.)
• Original Assignee: Ironport Systems (Cisco Systems, Inc.)
• Inventors: Quinlan, Daniel
• Primary Examiner(s): Sparks; Donald
• Assistant Examiner(s): Bharadwaj; Kalpana
• Application Number:
  US11/803,846
• Time in Patent Office: 1,421 Days
• Field of Search: 709/242
• US Class Current: 706/12
• CPC Class Codes: G06F 40/00 Handling natural language data speech analysis or synthesis, speech recognition G10L
• CPC Class Codes: G06F 40/44 Statistical methods, e.g. probability models
• CPC Class Codes: G06N 7/01 Probabilistic graphical models, e.g. probabilistic networks
• CPC Class Codes: G06Q 20/3821 Electronic credentials
• CPC Class Codes: H04L 51/00 User-to-user messaging in packet-switching networks, transmitted according to store-and-forward or real-time protocols, e.g. e-mail
• CPC Class Codes: H04L 51/212 using filtering or selective blocking
• CPC Class Codes: H04L 51/48 Message addressing, e.g. address format or anonymous messages, aliases

Abstract

A computer system can evaluate electronic messages based on probabilistic analysis, including Bayesian analysis. In one embodiment, a data processing system comprises logic configured for perform training a probabilistic filter using first properties of one or more first network resource identifiers obtained from a whitelist; training the probabilistic filter using second properties of one or more second network resource identifiers obtained from a blocklist; testing third properties of a third network resource identifier using the probabilistic filter, resulting in creating a probability output; and adding the third network resource identifier to the blocklist when the probability output is greater than a specified threshold. The blocklist can be communicated to messaging gateways in the field for local use in evaluating messages that contain spam or other threats.

Claims

1. A data processing system, comprising:

one or more processors;

logic encoded in one or more media for execution and when executed operable to cause the one or more processors to perform;

training a probabilistic filter using first properties of one or more first network resource identifiers obtained from a whitelist;

wherein at least one of the first properties is obtained from any of;

information obtained from DNS queries based, at least in part, on the one or more first work resource identifiers;

server software information;

or

information obtained from “

whois”

queries based, at least in part, on information contained in the network resource identifier;

training the probabilistic filter using second properties of one or more second network resource identifiers obtained from a blocklist;

testing third properties of a third network resource identifier using the probabilistic filter, resulting in creating a probability output;

adding the third network resource identifier to the blocklist when the probability output is greater than a specified threshold.

2. The invention of claim 1, wherein the whitelist comprises network resource identifiers that are not associated with spam or threats, and wherein the blocklist comprises network resource identifiers that are associated with spam or threats.

3. The invention of claim 1, wherein the specified threshold indicates spam or threats.

4. The invention of claim 1, wherein the training comprises generating lists of one or more of the properties from the network resource identifiers.

5. The invention of claim 1, wherein the logic when executed is further operable to perform:

extracting a domain name portion of the third network resource identifier;

retrieving one or more MX records for the domain name portion from a domain name system (DNS) server;

retrieving network address records for each mail exchange that is identified in the one or more MX records;

determining an average reputation score for the network address records;

adding the third network resource identifier to the blocklist when the average reputation score is less than a specified threshold.

6. The invention of claim 5, wherein the logic when executed is further operable to perform determining a blocklist status for the network address records, and adding the third network resource identifier to the blocklist when the blocklist status indicates that network addresses in the network address records are blocked.

7. The invention of claim 1, wherein the logic when executed is further operable to perform

extracting a domain name portion of the third network resource identifier;

retrieving one or more NS records for the domain name portion from a domain name system (DNS) server;

retrieving network address records for each mail exchange that is identified in the one or more NS records;

determining an average reputation score for the network address records;

adding the third network resource identifier to the blocklist when the average reputation score is less than a specified threshold.

8. The invention of claim 7, wherein the logic when executed is further operable to perform determining a blocklist status for the network address records, and adding the third network resource identifier to the blocklist when the blocklist status indicates that network addresses in the network address records are blocked.

9. The invention of claim 1, wherein the logic when executed is further operable to communicate the blocklist to one or more messaging gateways over a data network.

10. The invention of claim 1, further comprising:

one or more messaging gateways coupled to the processor over a data network;

wherein the logic when executed is further operable to communicate the blocklist to the one or more messaging gateways over the data network;

in each of the one or more messaging gateways, second logic encoded in one or more media for execution and when executed operable to perform;

receiving a message containing one or more hyperlinks or network resource identifiers;

searching the received blocklist to determine whether the one or more hyperlinks or network resource identifiers are in the blocklist;

modifying a spam score value of the message based on whether the one or more hyperlinks or network resource identifiers are in the blocklist.

29. The invention of claim 1, wherein the instructions for adding comprise instructions for adding the third network resource identifier to a blocklist other than the blocklist.

11. A data processing system, comprising:

means for training a probabilistic filter using first properties of one or more first network resource identifiers obtained from a whitelist;

wherein at least one of the first properties is obtained from any of;

information obtained from DNS queries based, at least in part, on the one or more first work resource identifiers;

server software information;

or

information obtained from “

whois”

queries based, at least in part, on information contained in the network resource identifier;

means for training the probabilistic filter using second properties of one or more second network resource identifiers obtained from a blocklist;

means for testing third properties of a third network resource identifier using the probabilistic filter, resulting in creating a probability output;

means for adding the third network resource identifier to the blocklist when the probability output is greater than a specified threshold.

12. The invention of claim 11, wherein the whitelist comprises network resource identifiers that are not associated with spam or threats, and wherein the blocklist comprises network resource identifiers that are associated with spam or threats.

13. The invention of claim 11, further comprising:

means for extracting a domain name portion of the third network resource identifier;

means for retrieving one or more MX records for the domain name portion from a domain name system (DNS) server;

means for retrieving network address records for each mail exchange that is identified in the one or more MX records;

means for determining an average reputation score for the network address records;

means for adding the third network resource identifier to the blocklist when the average reputation score is less than a specified threshold.

14. The invention of claim 13, further comprising means for determining a blocklist status for the network address records, and adding the third network resource identifier to the blocklist when the blocklist status indicates that network addresses in the network address records are blocked.

15. The invention of claim 11, further comprising:

means for extracting a domain name portion of the third network resource identifier;

means for retrieving one or more NS records for the domain name portion from a domain name system (DNS) server;

means for retrieving network address records for each mail exchange that is identified in the one or more NS records;

means for determining an average reputation score for the network address records;

means for adding the third network resource identifier to the blocklist when the average reputation score is less than a specified threshold.

16. The invention of claim 15, further comprising means for determining a blocklist status for the network address records, and adding the third network resource identifier to the blocklist when the blocklist status indicates that network addresses in the network address records are blocked.

17. The invention of claim 11, further comprising means for communicating the blocklist to one or more messaging gateways over a data network.

18. The invention of claim 11, further comprising:

one or more messaging gateways coupled to the processor over a data network;

means for communicating the blocklist to the one or more messaging gateways over the data network;

means for receiving a message containing one or more hyperlinks or network resource identifiers;

means for searching the received blocklist to determine whether the one or more hyperlinks or network resource identifiers are in the blocklist;

means for modifying a spam score value of the message based on whether the one or more hyperlinks or network resource identifiers are in the blocklist.

30. The invention of claim 11, wherein the means for adding comprises adding the third network resource identifier to a blocklist other than the blocklist.

19. A computer-implemented method of evaluating an electronic mail message based on probabilistic analysis, comprising:

training a probabilistic filter using first properties of one or more first network resource identifiers obtained from a whitelist;

wherein at least one of the first properties is obtained from any of;

information obtained from DNS queries based, at least in part, on the one or more first work resource identifiers;

server software information;

or

information obtained from “

whois”

queries based, at least in part, on information contained in the network resource identifier;

training the probabilistic filter using second properties of one or more second network resource identifiers obtained from a blocklist;

testing third properties of a third network resource identifier using the probabilistic filter, resulting in creating a probability output;

adding the third network resource identifier to the blocklist when the probability output is greater than a specified threshold.

20. The invention of claim 19, wherein the whitelist comprises network resource identifiers that are not associated with spam or threats, and wherein the blocklist comprises network resource identifiers that are associated with spam or threats.

21. The invention of claim 19, further comprising:

extracting a domain name portion of the third network resource identifier;

retrieving one or more MX records for the domain name portion from a domain name system (DNS) server;

retrieving network address records for each mail exchange that is identified in the one or more MX records;

determining an average reputation score for the network address records;

adding the third network resource identifier to the blocklist when the average reputation score is less than a specified threshold.

22. The invention of claim 21, further comprising determining a blocklist status for the network address records, and adding the third network resource identifier to the blocklist when the blocklist status indicates that network addresses in the network address records are blocked.

23. The invention of claim 19, further comprising:

extracting a domain name portion of the third network resource identifier;

retrieving one or more NS records for the domain name portion from a domain name system (DNS) server;

retrieving network address records for each mail exchange that is identified in the one or more NS records;

determining an average reputation score for the network address records;

adding the third network resource identifier to the blocklist when the average reputation score is less than a specified threshold.

24. The invention of claim 23, further comprising determining a blocklist status for the network address records, and adding the third network resource identifier to the blocklist when the blocklist status indicates that network addresses in the network address records are blocked.

31. The invention of claim 19, wherein the step of adding comprises adding the third network resource identifier to a blocklist other than the blocklist.

25. A non-transitory computer-readable tangible storage medium storing one or more sequences of instructions which, when executed by one or more processors, cause the one or more processors to perform the steps of:

training a probabilistic filter using first properties of one or more first network resource identifiers obtained from a whitelist;

wherein at least one of the first properties is obtained from any of;

information obtained from DNS queries based, at least in part, on the one or more first work resource identifiers;

server software information;

or

information obtained from “

whois”

queries based, at least in part, on information contained in the network resource identifier;

training the probabilistic filter using second properties of one or more second network resource identifiers obtained from a blocklist;

testing third properties of a third network resource identifier using the probabilistic filter, resulting in creating a probability output;

adding the third network resource identifier to the blocklist when the probability output is greater than a specified threshold.

26. The invention of claim 25, wherein the whitelist comprises network resource identifiers that are not associated with spam or threats, and wherein the blocklist comprises network resource identifiers that are associated with spam or threats.

27. The invention of claim 25, wherein the logic when executed is further operable to perform:

extracting a domain name portion of the third network resource identifier;

retrieving one or more MX records for the domain name portion from a domain name system (DNS) server;

retrieving network address records for each mail exchange that is identified in the one or more MX records;

determining an average reputation score for the network address records;

adding the third network resource identifier to the blocklist when the average reputation score is less than a specified threshold.

28. The invention of claim 27, wherein the logic when executed is further operable to perform determining a blocklist status for the network address records, and adding the third network resource identifier to the blocklist when the blocklist status indicates that network addresses in the network address records are blocked.

32. The invention of claim 25, wherein the logic for adding comprises logic for adding the third network resource identifier to a blocklist other than said blocklist.