Method and apparatus for filtering email spam using email noise reduction

• US 20050108339A1
• Filed: 05/13/2004
• Published: 05/19/2005
• Est. Priority Date: 05/15/2003
• Status: Active Grant

• Current Assignee: CA, Inc. (d/b/a CA Technologies) (Broadcom, Inc.)
• Original Assignee: Symantec Corporation (NortonLifeLock Inc.)
• Inventors: Gleeson, Matt, Schneider, Ken, Mantel, Eli, Medlar, Art, Jensen, Sandy, Hoogstrate, David
• Granted Patent: US 7,831,667 B2
• Time in Patent Office: Days
• US Class Current: 709/206
• CPC Class Codes: H04L 51/063 Content adaptation, e.g. replacement of unsuitable content
• CPC Class Codes: H04L 51/212 using filtering or selective blocking

Abstract

A method and system for filtering email spam using email noise reduction are described. In one embodiment, the method includes detecting, in an email message, data indicative of noise added to the email message to avoid spam filtering. The method further includes modifying the content of the email message to reduce the noise, and comparing the modified content of the email message with the content of a spam message.

Claims

1. A method comprising:

detecting, in an email message, data indicative of noise added to the email message to avoid spam filtering;

modifying content of the email message to reduce the noise; and

comparing the modified content of the email message with content of a spam message.

2. The method of claim 1 wherein the data indicative of the noise added to the email message is selected from the group consisting of formatting data, one or more numeric character references, one or more character entity references, and URL data of predefined categories.

3. The method of claim 2 wherein modifying the content of the email message comprises:

extracting from the email message formatting data that does not qualify as an exception.

4. The method of claim 3 wherein the formatting data is qualified as an exception of the formatting data is required for processing of the email message.

5. The method of claim 2 wherein modifying the content of the email message comprises:

converting each numeric character reference and each character entity reference into a corresponding ASCII character;

determining that at least one of converted numeric character references and character entity references contains any of a numeric character reference and a character entity reference; and

converting said any of the numeric character reference and the character entity reference into a corresponding ASCII character.

6. The method of claim 2 wherein modifying the content of the email message comprises:

converting each numeric character and each character entity reference within a URL into a corresponding ASCII character.

7. The method of claim 2 wherein modifying the content of the email message comprises:

removing, from a URL, unique identifier data of a predefined category.

8. The method of claim 2 wherein modifying the content of the email message comprises:

removing, from a URL, query data of a predefined category.

9. The method of claim 1 wherein comparing the modified content of the email message with the content of the spam message comprises:

determining whether the modified content of the email message is similar to the content of the spam message.

10. The method of claim 1 further comprising:

prior to comparing the modified content of the email message with the content of the spam message, modifying the content of the spam message to reduce noise.

11. A system comprising:

a message cleaning algorithm to detect, in an email message, data indicative of noise added to the email message to avoid spam filtering, and to modify content of the email message to reduce the noise; and

a resemblance identifier to compare the modified content of the email message with content of a spam message.

12. The system of claim 11 wherein the data indicative of the noise added to the email message is selected from the group consisting of formatting data, one or more numeric character references, one or more character entity references, and URL data of predefined categories.

13. The system of claim 12 wherein the message cleaning algorithm is to modify the content of the email message by extracting from the email message formatting data that does not qualify as an exception.

14. The system of claim 13 wherein the formatting data is qualified as an exception of the formatting data is required for processing of the email message.

15. The system of claim 12 wherein the message cleaning algorithm is to modify the content of the email message by converting each numeric character reference and each character entity reference into a corresponding ASCII character, determining that at least one of converted numeric character references and character entity references contains any of a numeric character reference and a character entity reference, and converting said any of the numeric character reference and the character entity reference into a corresponding ASCII character.

16. The system of claim 12 wherein the message cleaning algorithm is to modify the content of the email message by converting each numeric character and each character entity reference within a URL into a corresponding ASCII character.

17. The system of claim 12 wherein the message cleaning algorithm is to modify the content of the email message by removing, from a URL, unique identifier data of a predefined category.

18. The system of claim 12 wherein the message cleaning algorithm is to modify the content of the email message by removing, from a URL, query data of a predefined category.

19. The system of claim 11 wherein the resemblance identifier is to compare the modified content of the email message with the content of the spam message by determining whether the modified content of the email message is similar to the content of the spam message.

20. An apparatus comprising:

means for detecting, in an email message, data indicative of noise added to the email message to avoid spam filtering;

means for modifying content of the email message to reduce the noise; and

means for comparing the modified content of the email message with content of a spam message.

21. The apparatus of claim 20 wherein the data indicative of the noise added to the email message is selected from the group consisting of formatting data, one or more numeric character references, one or more character entity references, and URL data of predefined categories.

22. A computer readable medium comprising executable instructions which when executed on a processing system cause said processing system to perform a method comprising:

detecting, in an email message, data indicative of noise added to the email message to avoid spam filtering;

modifying content of the email message to reduce the noise; and

comparing the modified content of the email message with content of a spam message.

23. The computer readable medium of claim 22 wherein the data indicative of the noise added to the email message is selected from the group consisting of formatting data, one or more numeric character references, one or more character entity references, and URL data of predefined categories.