System and method for intelligent SPAM detection using statistical analysis

• US 7,209,954 B1
• Filed: 03/30/2005
• Issued: 04/24/2007
• Est. Priority Date: 07/26/2001
• Status: Expired due to Term

• Current Assignee: McAfee, LLC
• Original Assignee: McAfee, Inc. (McAfee, LLC)
• Inventors: Jagger, Luke D., Rothwell, Anton C., Dennis, William R., Clarke, David R.
• Primary Examiner(s): Barot; Bharat
• Application Number:
  US11/095,146
• Time in Patent Office: 755 Days
• Field of Search: 709206-207
• US Class Current: 709/206
• CPC Class Codes: H04L 51/212 using filtering or selective blocking

Abstract

A system, method and computer program product are provided for detecting an unwanted message. First, an electronic mail message is received. Text in the electronic mail message is decomposed. Statistics associated with the text are gathered using a statistical analyzer. The statistics are analyzed for determining whether the electronic mail message is an unwanted message.

Claims

1. A method for detecting an unwanted message, comprising:

(a) receiving an electronic mail message;

(b) decomposing text in the electronic mail message;

(c) gathering statistics associated with the text using a statistical analyzer; and

(d) analyzing the statistics for determining whether the electronic mail message is an unwanted message;

wherein the statistics gathered using the statistical analyzer include results of an analysis of a uniform resource locator (URL) in the electronic mail message text;

wherein the statistics gathered using the statistical analyzer include results of an analysis of e-mail addresses in the electronic mail message text;

wherein the statistics gathered using the statistical analyzer include results of a message header field analysis.

2. The method as recited in claim 1, wherein the statistics gathered using the statistical analyzer include a ratio of words capitalized to total number of words.

3. The method as recited in claim 1, wherein the statistics gathered using the statistical analyzer include a punctuation to word ratio.

4. The method as recited in claim 1, wherein the statistics gathered using the statistical analyzer include a number of uniform resource locators (URLs) in the text.

5. The method as recited in claim 1, wherein the statistics gathered using the statistical analyzer include at least one telephone number in the text.

6. The method as recited in claim 1, wherein the statistics gathered using the statistical analyzer include results of an analysis of character type.

7. The method as recited in claim 1, wherein the statistics gathered using the statistical analyzer include a ratio of words capitalized to total number of words, a punctuation to word ratio, a number of uniform resource locators (URLs) in the text, and a number of telephone numbers in the text.

8. The method as recited in claim 1, wherein the statistics are placed in a results table, wherein entries in the table are passed as inputs to a neural network engine.

9. The method as recited in claim 1, wherein the neural network engine analyzes previous user input for determining whether the message is unwanted.

10. The method as recited in claim 1, wherein the statistics are sent to a neural network engine, wherein the neural network engine compares the statistics to predetermined weights for determining whether the electronic mail message is an unwanted message.

11. The method as recited in claim 10, wherein the neural network engine is taught to recognize unwanted messages.

12. The method as recited in claim 11, wherein examples are provided to the neural network engine, wherein the examples are of wanted messages and unwanted messages, and each of the examples is associated with a desired output.

13. The method as recited in claim 12, wherein each of the examples are processed with statistics by the neural network engine for generating weights for the statistics, wherein each of the weights is used to denote wanted and unwanted messages.

14. The method as recited in claim 13, wherein logic associated with the neural network engine is updated based on the processing by the neural network engine.

15. The method as recited in claim 14, wherein the neural network engine is updated to recognize an unwanted message, the message is identified as an unwanted message, the features of the message that make the message unwanted are identified, and the identified features are stored and used by the neural network to identify subsequent unwanted messages.

16. A method for detecting an unwanted message, comprising:

(a) receiving an electronic mail message;

(b) decomposing text in the electronic mail message;

(c) gathering statistics associated with the text using a statistical analyzer; and

(d) analyzing the statistics for determining whether the electronic mail message is an unwanted message;

wherein the statistics gathered using the statistical analyzer include results of an analysis of a uniform resource locator (URL) in the electronic mail message text;

wherein the statistics gathered using the statistical analyzer include results of an analysis of e-mail addresses in the electronic mail message text;

wherein the statistics gathered using the statistical analyzer include results of a message header field analysis;

wherein the statistics are sent to a neural network engine, wherein the neural network engine compares the statistics to predetermined weights for determining whether the electronic mail message is an unwanted message;

wherein the neural network engine is taught to recognize unwanted messages;

wherein examples are provided to the neural network engine, wherein the examples are of wanted messages and unwanted messages, and each of the examples is associated with a desired output;

wherein each of the examples are processed with statistics by the neural network engine for generating weights for the statistics, wherein each of the weights is used to denote wanted and unwanted messages;

wherein the neural network engine utilizes adaptive linear combination for adjusting the weights.

17. A computer program product having computer-executable codes embodied in a computer-readable medium for detecting an unwanted message, comprising:

(a) computer code for receiving an electronic mail message;

(b) computer code for decomposing text in the electronic mail message;

(c) computer code for gathering statistics associated with the text using a statistical analyzer; and

(d) computer code for analyzing the statistics for determining whether the electronic mail message is an unwanted message;

wherein the statistics gathered using the statistical analyzer include results of an analysis of a uniform resource locator (URL) in the electronic mail message text;

wherein the statistics gathered using the statistical analyzer include results of an analysis of e-mail addresses in the electronic mail message text;

wherein the statistics gathered using the statistical analyzer include results of a message header field analysis.

18. A system for detecting an unwanted message, comprising:

(a) a statistical analyzer for gathering statistics associated with text retrieved from an electronic mail message; and

(b) a neural network engine coupled to the statistical analyzer for analyzing the statistics;

(c) wherein the neural network engine determines whether the electronic mail message is an unwanted message;

wherein the statistics gathered using the statistical analyzer include results of an analysis of a uniform resource locator (URL) in the electronic mail message text;

wherein the statistics gathered using the statistical analyzer include results of an analysis of e-mail addresses in the electronic mail message text;

wherein the statistics gathered using the statistical analyzer include results of a message header field analysis.

19. A method for detecting an unwanted message, comprising:

(a) receiving an electronic mail message;

(b) decomposing text in the electronic mail message;

(c) gathering statistics associated with the text using a statistical analyzer, wherein the statistics gathered using the statistical analyzer include at least three of a ratio of words capitalized to total number of words, a punctuation to word ratio, a number of uniform resource locators (URLs) in the text, a telephone number in the text, results of an analysis of a URL in the electronic mail message text, results of an analysis of e-mail addresses in the electronic mail message text, results of an analysis of character type, and results of a message header field analysis; and

(d) analyzing the statistics for determining whether the electronic mail message is an unwanted message.

20. The method as recited in claim 19, wherein the statistics gathered using the statistical analyzer include at least four of a ratio of words capitalized to total number of words, a punctuation to word ratio, a number of uniform resource locators (URLs) in the text, a telephone number in the text, results of an analysis of a URL in the electronic mail message text, results of an analysis of e-mail addresses in the electronic mail message text, results of an analysis of character type, and results of a message header field analysis.

21. The method as recited in claim 19, wherein the statistics gathered using the statistical analyzer include at least five of a ratio of words capitalized to total number of words, a punctuation to word ratio, a number of uniform resource locators (URLs) in the text, a telephone number in the text, results of an analysis of a URL in the electronic mail message text, results of an analysis of e-mail addresses in the electronic mail message text, results of an analysis of character type, and results of a message header field analysis.

22. The method as recited in claim 19, wherein the statistics gathered using the statistical analyzer include at least six of a ratio of words capitalized to total number of words, a punctuation to word ratio, a number of uniform resource locators (URLs) in the text, a telephone number in the text, results of an analysis of a URL in the electronic mail message text, results of an analysis of e-mail addresses in the electronic mail message text, results of an analysis of character type, and results of a message header field analysis.

23. A method for detecting an unwanted message, comprising:

decomposing text in an electronic mail message;

gathering statistics associated with the text using a statistical analyzer; and

analyzing the statistics for determining whether the electronic mail message is an unwanted message;

wherein the statistics gathered using the statistical analyzer are selected from the group consisting of results of an analysis of a uniform resource locator (URL) in the electronic mail message text, results of an analysis of e-mail addresses in the electronic mail message text, and results of a message header field analysis.