Training filters for detecting spasm based on IP addresses and text-related features

• US 7,464,264 B2
• Filed: 03/25/2004
• Issued: 12/09/2008
• Est. Priority Date: 06/04/2003
• Status: Expired due to Fees

• Current Assignee: Microsoft Technology Licensing LLC (Microsoft Corporation)
• Original Assignee: Microsoft Corporation
• Inventors: Rounthwaite, Robert L., Goodman, Joshua T., Hulten, Geoffrey J., Yih, Wen-tau
• Primary Examiner(s): Moazzami; Nasser G
• Assistant Examiner(s): Traore; Fatoumata
• Application Number:
  US10/809,163
• Publication Number:
  US 20040260922A1
• Time in Patent Office: 1,720 Days
• Field of Search: 713/150, 713/153, 713/154, 726/3, 726/11, 726/13
• US Class Current: 713/154
• CPC Class Codes: G06Q 10/107 Computer-aided management of electronic mailing [e-mailing]
• CPC Class Codes: H04L 51/212 using filtering or selective blocking

Abstract

The subject invention provides for an intelligent quarantining system and method that facilitates detecting and preventing spam. In particular, the invention employs a machine learning filter specifically trained using origination features such as an IP address as well as destination feature such as a URL. Moreover, the system and method involve training a plurality of filters using specific feature data for each filter. The filters are trained independently each other, thus one feature may not unduly influence another feature in determining whether a message is spam. Because multiple filters are trained and available to scan messages either individually or in combination (at least two filters), the filtering or spam detection process can be generalized to new messages having slightly modified features (e.g., IP address). The invention also involves locating the appropriate IP addresses or URLs in a message as well as guiding filters to weigh origination or destination features more than text-based features.

Claims

1. A machine-implemented system that facilitates spam detection comprising a processor executing:

a feature extraction component that receives an item and extracts a set of features associated with an origination of a message or part thereof and/or information that enables an intended recipient to contact or respond to the message;

a feature analysis component that analyzes a subset of the extracted features in connection with building and employing a plurality of feature-specific filters that are independently trained to mitigate undue influence of at least one feature type over another in the message, the subset of extracted features comprising of at least one of a Uniform Resource Locator (URL) and an Internet Protocol (IP) address, and the plurality of feature-specific filters comprising at least a first feature-specific filter; and

a machine learning component that determines last IP address external to the recipient'"'"'s system via a machine learning technique to facilitate spam detection, the machine learning component employs MX records to determine a true source of a message by way of tracing back through a received from list until an IP address is found that corresponds to a fully qualified domain which corresponds to an entry in the domain'"'"'s MX record and determines whether the IP address is external or internal by verifying if the IP address is in a form characteristic to internal IP addresses and performing at least one of an IP address lookup and a reverse IP address lookup to ascertain whether the IP address correlates with a sender'"'"'s domain name.

2. The system of claim 1, further comprising a plurality of training components that individually employ at least one of IP addresses or URLs and other features, respectively, in connection with building the plurality of feature-specific filters.

3. The system of claim 1, the first feature-specific filter is trained using IP addresses.

4. The system of claim 1, the first feature-specific filter is trained using URLs.

5. The system of claim 1, the plurality of feature specific filters comprising a second feature-specific filter that is trained using a subset of features extracted from the message other than a URL and an IP address.

6. The system of claim 1, further comprising a filter combining component that combines information collected from the first feature-specific filter and a second feature-specific filter.

7. The system of claim 6, the first feature-specific filter detects at least one of known IP addresses and at least one known URL in the message.

8. The system of claim 6, the second feature-specific filter detects non-IP address and non-URL data in the message.

9. The system of claim 6, the filter combining component combines the information by at least one of multiplying scores generated by the filters, adding scores generated by the filters, or training an additional filter to combine the scores.

10. The system of claim 1, the first feature-specific filter is trained independently of a second feature-specific filter to mitigate either filter influencing the other when filtering the message.

11. The system of claim 10, at least one of the feature-specific filters models dependencies.

12. The system of claim 1, the plurality of feature-specific filters is machine learning filters.

13. The system of claim 1, the machine learning component determines whether the IP address is external or internal comprises at least one of the following:

collecting user feedback related to user classification of messages as spam or good;

examining messages classified as good by a user to learn which servers are internal; and

finding a worst-scoring IP address in a message.

14. A machine-implemented system that facilitates spam detection comprising a processor executing:

a feature extraction component that receives an item and extracts a set of features associated with an origination of a message or part thereof and/or information that enables an intended recipient to contact or respond to the message;

at least one filter that is used when one of an Internet Protocol (IP) address of the message or at least some part of at least one of Uniform Resource Locator (URL) in the message is unknown; and

a machine learning component that determines last IP address external to the recipient'"'"'s system via a machine learning technique to facilitate spam detection, the machine learning component employs MX records to determine a true source of a message by way of tracing back through a received from list until an IP address is found that corresponds to a fully Qualified domain which corresponds to an entry in the domain'"'"'s MX record and determines whether the IP address is external or internal by verifying if the IP address is in a form characteristic to internal IP addresses and performing at least one of an IP address lookup and a reverse IP address lookup to ascertain whether the IP address correlates with a sender'"'"'s domain name.

15. The system of claim 14, the at least one filter is trained using some number of bits less than 32 bits of an IP address.

16. The system of claim 14, the at least one filter is trained using all bits of an IP address.

17. The system of claim 14, further comprising a filter selection component that selects and employs at least one feature-specific filter out of a plurality of feature-specific filters for which there is sufficient data extracted from the message.

18. A machine-implemented system that facilitates spam detection comprising a processor executing:

a feature extraction component that receives an item and extracts a set of features associated with an origination of a message or part thereof and/or information that enables an intended recipient to contact or respond to the message;

at least one filter that is used when one of the Internet Protocol (IP) address of the message or at least some part of at least one of the Uniform Resource Locators (URLs) in the message is known; and

a machine learning component that determines last IP address external to the recipient'"'"'s system via a machine learning technique to facilitate spam detection, the machine learning component employs MX records to determine a true source of a message by way of tracing back through a received from list until an IP address is found that corresponds to a fully Qualified domain which corresponds to an entry in the domain'"'"'s MX record and determines whether the IP address is external or internal by verifying if the IP address is in a form characteristic to internal IP addresses and performing at least one of an IP address lookup and a reverse IP address lookup to ascertain whether the IP address correlates with a sender'"'"'s domain name.

19. The system of claim 18, the at least one filter is trained on one of known IP addresses or known URLs together with text-based features.

20. The system of claim 18, further comprising at least one other filter that is used to examine text-based features in the message.

21. A machine learning method implemented on a machine that facilitates spam detection by optimizing an objective function of the form

OBJECTIVE(MAXSCORE(m1), MAXSCORE(m2), . . . , MAXSCORE(mk), w1 . . . wn) where MAXSCORE(mk) =MAX(SCORE(IPk,1), SCORE(IPk,2), . . . , SCORE(IPk,kI))

where mk =messages;

IPk,i represents the IP addresses of mk;

SCORE(IPk,i)=the sum of the weights of the IPk,i, and

wherein the machine learning method optimizes the weights associated with one feature at any given time and maximizes accuracy on a training data to facilitate improved detection of spam, the machine learning method employs MX records to determine a true source of a message by way of tracing back through a received from list until an IP address is found that corresponds to a fully qualified domain which corresponds to an entry in the domain'"'"'s MX record and determines whether the IP address is external or internal by verifying if the IP address is in a form characteristic to internal IP addresses and performing at least one of an IP address lookup and a reverse IP address lookup to ascertain whether the IP address correlates with a sender'"'"'s domain name and further determines last IP address external to a recipient'"'"'s system to facilitate spam detection.

22. The machine learning method of claim 21, the objective function depends in part on whether the messages are properly categorized as any one of spam or good.

23. The machine learning method of claim 21, further comprises learning the weights for each feature in turn.

24. The machine learning method of claim 23, learning the weight for a given feature comprises sorting training instances comprising a property, the property comprising a feature in order by the weight at which the score for that message varies with the weight for that feature.

25. The machine learning method of claim 24, the training instances comprise electronic messages.

26. The machine learning method of claim 21, the messages are training instances and the property and the properties comprise one or more IP addresses that the message originated from and any URLs in the message.

27. The machine learning method of claim 21, learning is performed using an approximation MAX(a₁, a₂, . . . , a_n) is approximately equal to SUM(a₁^x, a₂^x, . . . , a_n^x)^(1/x).

28. The machine learning method of claim 27, the objective function depends in part on whether the messages are properly categorized as spam or good.

29. A machine-implemented method that facilitates spam detection comprising:

providing a plurality of training data;

extracting a plurality of feature types from the training data, the feature types comprising at least one Internet Protocol (IP) address, at least one Uniform Resource Locator (URL) and text-based features;

training a plurality of feature-specific filters for the respective feature in an independent manner so that a first feature does not unduly influence a message score over a second feature type when determining whether a message is spam;

employing MX records to determine a true source of a message by way of tracing back through a received from list until an IP address is found that corresponds to a fully qualified domain which corresponds to an entry in the domain'"'"'s MX record;

verifying if the IP address is in a form characteristic to internal IP addresses;

performing at least one of an IP address lookup and a reverse IP address lookup to ascertain whether the IP address correlates with a sender'"'"'s domain name; and

determining last IP address external to the recipient'"'"'s system to facilitate spam detection.

30. The method of claim 29, the plurality of training data comprises messages.

31. The method of claim 29, the plurality of feature-specific filters comprises at least two of the following:

a known IP address filter;

an unknown IP address filter;

a known URL filter;

an unknown URL filter; and

a text-based filter.

32. The method of claim 31, the known IP address filter is trained using 32 bits of IP addresses.

33. The method of claim 31, the unknown IP address filter is trained using some number of bits of IP addresses less than 32 bits.

34. The method of claim 31, the unknown IP address filter is trained using other messages comprising unknown IP addresses.

35. The method of claim 31, the text-based filter is trained using words, phrases, character runs, character strings, and any other relevant non-IP address or non-URL data in the message.

36. The method of claim 31, employing at least one of the known IP address filter, the unknown IP address filter, the known URL filter, and the unknown URL filter together with the text-based filter to more accurately determine whether a new message is spam.

37. The method of claim 31, further comprising employing at least one of the feature-specific filters in connection with determining whether a new message is spam, such that the feature-specific filter is selected based in part on most relevant feature data observed in the new message.

38. The method of claim 31, the URL filter is trained on URL data comprising a fully qualified domain name and subdomains of the fully qualified domain name.

39. The method of claim 31 combined with a feedback loop mechanism whereby users provide their feedback regarding incoming messages by submitting message classifications to fine tune the one or more feature-specific filters.

40. The method of claim 29, further comprising combining message scores generated from at least two filters used to scan a new message to generate a total score that facilitates determining whether the message is spam.

41. The method of claim 40, combining message scores comprises at least one of the following:

multiplying the scores;

adding the scores; and

training a new model to combine the scores.

42. The method of claim 29, further comprising quarantining messages that satisfy at least one criterion for a period of time until additional information about the message can be collected to update one or more feature-specific filters to facilitate determining whether the messages are spam.

43. A machine-implemented method that facilitates spam detection comprising:

extracting data from a plurality of messages;

training at least one machine learning filter using at least a subset of the data, the training comprising employing a first smoothing for at least one of Internet Protocol (IP) address or Uniform Resource Locator (URL) features and at least a second smoothing for other non-IP address or non-URL features;

employing MX records to determine a true source of a message by way of tracing back through a received from list until an IP address is found that corresponds to a fully Qualified domain which corresponds to an entry in the domain'"'"'s MX record;

verifying that the IP address is in a form characteristic to internal IP addresses;

performing at least one of an IP address lookup and a reverse IP address lookup to ascertain whether the IP address correlates with a sender'"'"'s domain name; and

determining last IP address external to the recipient'"'"'s system to facilitate spam detection.

44. The method of claim 43, the smoothing differs in at least one of the following aspects:

the first smoothing comprises a different variance compared to the second smoothing with respect to a maximum entropy model; and

the first smoothing comprises a different value of weight decay compared to the second smoothing with respect to a Support Vector Machine (SVM) model.