System and method for filtering spam messages utilizing URL filtering module

• US 20090070872A1
• Filed: 06/17/2004
• Published: 03/12/2009
• Est. Priority Date: 06/18/2003
• Status: Active Grant

• Current Assignee: Symantec Corporation (NortonLifeLock Inc.)
• Original Assignee: Brightmail, Inc. (Gen Digital Inc.)
• Inventors: Jenson, Sandy, Schneider, Ken, Medlar, Art, Hoogstrate, David, Cowings, David
• Granted Patent: US 8,145,710 B2
• Time in Patent Office: Days
• US Class Current: 726/23
• CPC Class Codes: G06Q 10/107 Computer-aided management of electronic mailing [e-mailing]
• CPC Class Codes: H04L 51/212 using filtering or selective blocking

Abstract

Systems and methods for filtering spam messages utilizing a URL filtering module are described. In one embodiment, the method includes detecting, in an incoming message, data indicative of a URL and comparing the URL from the incoming message with URLs characterizing spam. The method further includes determining whether the incoming message is spam based on the comparison of the URL from the incoming message with the URLs characterizing spam.

Claims

1. A method, comprising:

detecting, in an incoming message, data indicative of a uniform resource locator (URL);

creating hash data for the URL from the incoming message;

comparing the hash data for the URL from the incoming message with a plurality of URL filtering rules created using URLs extracted from messages identified as spam; and

determining whether the incoming message is spam based on the comparison of the hash data for the URL from the incoming message with the plurality of URL filtering rules.

2. The method of claim 1, further comprising reducing noise in the data indicative of the URL to identify the URL.

4. The method of claim 1, wherein comparing the hash data for the URL from the incoming message with a plurality of URL filtering rules comprises:

determining that the hash data for the URL from the incoming message matches hash data from one of the plurality of URL filtering rules;

determining that the matching hash data from one of the plurality of URL filtering rules is associated with an inclusion URL; and

determining whether the incoming message contains a second URL matching the inclusion URL.

5. The method of claim 1, wherein comparing the hash data for the URL from the incoming message with a plurality of URL filtering rules comprises:

determining that the hash data for the URL from the incoming message matches hash data from one of the plurality of URL filtering rules;

determining that the matching hash data from one of the plurality of URL filtering rules is associated with an exclusion URL; and

determining whether the incoming message contains any URLs matching the exclusion URL.

6. The method of claim 1, wherein comparing the hash data for the URL from the incoming message with a plurality of URL filtering rules comprises:

determining that the URL from the incoming message includes a path component;

creating a first hash value for a path-level URL associated with the URL from the incoming message and a second hash value for a host-level URL associated with the URL from the incoming message;

determining whether the first hash value for the path-level URL matches hash data in any of the plurality of URL filtering rules; and

if the the first hash value for the path-level URL does not match hash data in any of the plurality of URL filtering rules, determining whether the second hash value for the host-level URL matches hash data in any of the plurality of URL filtering rules.

7. The method of claim 1, wherein comparing the hash data for the URL from the incoming message with a plurality of URL filtering rules:

determining that the URL from the incoming message includes one or more sub-domains; and

determining whether a hash value of a URL string of any sub-domain level matches hash data in any of the plurality of URL filtering rules.

8. The method of claim 1, wherein comparing the hash data for the URL from the incoming message with a plurality of URL filtering rules:

determining that the URL from the incoming message includes a redirection to a target URL; and

determining whether a hash value of the target URL matches hash data in any of the plurality of URL filtering rules.

9. The method of claim 2, wherein reducing noise comprises:

converting each numeric character reference and each character entity reference in the URL from the incoming message into a corresponding ASCII character.

10. The method of claim 1, wherein determining whether the email message is spam comprises:

determining whether a resemblance between the URL from the incoming message and a URL from the plurality of URL filtering rules exceeds a threshold.

11. The method of claim 1, wherein determining whether the incoming message is spam comprises:

determining that the hash data for the URL from the incoming message matches hash data in any of the plurality of URL filtering rules;

determining whether a weight associated with the matching URL exceeds a threshold; and

if the weight associated with the matching URL exceeds a threshold, determining that the incoming message is spam.

3. (canceled)

12. A method, comprising:

receiving a spam message sent to a probe email address;

extracting data indicative of a spam uniform resource locator (URL) from the spam message;

identifying the spam URL based on the data indicative of the spam URL; and

storing hash data associated with the spam URL in a database, the hash data associated with the spam URL being subsequently used to detect incoming spam messages.

13. The method of claim 12, further comprising:

transferring the hash data associated with the spam URL to a client.

14. The method of claim 12, further comprising:

modifying the data indicative of the spam URL to reduce noise.

15. The method of claim 12, wherein storing the hash data associated with the spam URL in the database comprises:

creating a hash value of the spam URL.

16. The method of claim 12, further comprising assigning to the spam URL a weight characterizing an effectiveness of the URL for indicating spam.

17. The method of claim 12, further comprising classifying the spam URL according to content of a website associated with the URL.

18. The method of claim 12, further comprising associating the spam URL with at least one of an inclusion URL and an exclusion URL.

19. The method of claim 12, further comprising creating for the spam URL at least one of a host-level URL, a path-level URL, a sub-domain URL, and a redirect URL.

20. A system comprising:

an incoming message parser to detect, in an incoming message, data indicative of a uniform resource locator (URL);

a URL data generator to create hash data for the URL from the incoming message; and

a resemblance identifier to compare the hash data for the URL from the incoming message with a plurality of URL filtering rules created using URLs extracted from messages identified as spam, and to determine whether the incoming message is spam based on the comparison of the hash data for the URL from the incoming message with the plurality of URL filtering rules.

21. The system of claim 20, further comprising a URL normalizer to reduce noise in the data indicative of the URL.

23. The system of claim 20, further comprising:

a spam URL receiver to receive the plurality of URL filtering rules; and

a spam URL database to store the plurality of URL filtering rules.

22. (canceled)

24. A system comprising:

a spam receiver to receive a spam message sent to a probe email address and to extract data indicative of a spam uniform resource locator (URL) from the spam message;

a noise reduction algorithm to identify the spam URL based on the data indicative of the spam URL; and

a database to store hash data associated with the spam URL, the hash data associated with the spam URL being subsequently used to detect incoming spam messages.

25. The system of claim 24, further comprising a spam URL transmitter to transfer the hash data associated with the spam URL to a client.

26. The system of claim 24, wherein the noise reduction algorithm is further to modify the data indicative of the spam URL to reduce noise.

27. (canceled)

28. An apparatus comprising:

means for detecting, in an incoming message, data indicative of a uniform resource locator (URL);

means for creating hash data for the URL from the incoming message;

means for comparing the hash data for the URL from the incoming message with a plurality of URL filtering rules created using URLs extracted from messages identified as spam; and

means for determining whether the incoming message is spam based on the comparison of the hash data for the URL from the incoming message with the plurality of URL filtering rules.

29. An apparatus comprising:

means for receiving a spam message sent to a probe email address;

means for extracting data indicative of a spam uniform resource locator (URL) from the spam message;

means for identifying the spam URL based on the data indicative of the spam URL; and

means for storing hash data associated with the spam URL in a database, the hash data associated with the spam URL being subsequently used to detect incoming spam messages.

30. A computer readable medium comprising executable instructions which when executed on a processing system cause said processing system to perform a method comprising:

detecting, in an incoming message, data indicative of a uniform resource locator (URL);

creating hash data for the URL from the incoming message;

comparing the hash data for the URL from the incoming message with a plurality of URL filtering rules created using URLs extracted from messages identified as spam; and

determining whether the incoming message is spam based on the comparison of the hash data for the URL from the incoming message with the plurality of URL filtering rules.

31. A computer readable medium comprising executable instructions which when executed on a processing system cause said processing system to perform a method comprising:

receiving a spam message sent to a probe email address;

extracting data indicative of a spam uniform resource locator (URL) from the spam message;

identifying the spam URL based on the data indicative of the spam URL; and

storing hash data associated with the spam URL in a database, the hash data associated with the spam URL being subsequently used to detect incoming spam messages.