(More) advanced spam detection features

• US 8,214,438 B2
• Filed: 03/01/2004
• Issued: 07/03/2012
• Est. Priority Date: 03/01/2004
• Status: Active Grant

• Current Assignee: Microsoft Technology Licensing LLC (Microsoft Corporation)
• Original Assignee: Microsoft Corporation
• Inventors: Mehr, John D., Howell, Nathan D., Rupersburg, Micah C.
• Primary Examiner(s): BIAGINI, CHRISTOPHER D
• Application Number:
  US10/790,574
• Publication Number:
  US 20050193073A1
• Time in Patent Office: 3,046 Days
• Field of Search: 709/206
• US Class Current: 709/206
• CPC Class Codes: H04L 51/212 using filtering or selective blocking

Abstract

The present invention involves a system and method that facilitate extracting data from messages for spam filtering. The extracted data can be in the form of features, which can be employed in connection with machine learning systems to build improved filters. Data associated with the subject line, timestamps, and the message body can be extracted and employed to generate one or more features. In particular, subject lines and message bodies can be examined for consecutive, repeating characters, blobs, the association or distance between such characters, blobs and non-blob portions of the message. The values or counts obtained can be broken down into one or more ranges corresponding to a degree of spaminess. Presence and type of attachments to messages, percentage of non-white-space and non-numeric characters of a message, and determining message delivery times can be used to identify spam. A time-based delta can be computed to facilitate determining the delivery time.

Claims

1. A system that facilitates extracting data in connection with spam processing, comprising:

a processing unit; and

a memory for storing computer-executable instructions that when executed by the processing unit executes;

a component comprising software that receives a message and extracts a set of features associated with some part, content or content type of a message; and

an analysis component comprising software that examines

(1) consecutiveness of characters within a subject line of the message, wherein the analysis component establishes ranges of consecutive, repeating characters, the ranges corresponding to varying degrees of spaminess, whereby messages can be sorted by their respective individual count of consecutive repeating characters and,

(2) a content type of the message for spam in connection with building a filter, wherein the content type describes a type of data contained within a body of the message, the content type being case-sensitive and comprising a primary content-type, a secondary-content type, or a combination thereof, the primary content-type and the secondary-content type comprising at least one of a text, a multipart, a message, an image, an audio, a video, or an application, wherein the analysis component compares the content type of the message to stored content types of a plurality of other messages to facilitate determining whether the message is spam.

2. The system of claim 1, the analysis component determines frequency of consecutive repeating characters within the subject line of the message.

3. The system of claim 2, the characters comprise letters, numbers, or punctuation.

4. The system of claim 1, the analysis component determines the frequency of white space characters within the subject line of the message.

5. The system of claim 1, the analysis component determines distance between at least one alpha-numeric character and a blob, wherein the blob comprises a random sequence of characters, numbers, punctuation, or a combination thereof.

6. The system of claim 1, the analysis component determines a maximum number of consecutive, repeating characters and stores this information.

7. The system of claim 1, the analysis component further determines time stamps associated with the message.

8. The system of claim 7, the analysis component determines a delta between time stamps.

9. The system of claim 8, the delta is between a first and a last time stamp.

10. The system of claim 1, the analysis component determines at least one of:

a percentage of white space to non-white space in the subject line of the message and a percentage of non-white space and non-numeric characters that are not letters in the subject line of the message.

11. The system of claim 1, the filter being a spam filter.

12. The system of claim 1, the filter being a parental control filter.

13. The system of claim 1, further comprising a machine learning system component that employs at least a subset of extracted features to learn at least one of spam and non-spam.

14. A method for evaluating spam as a function of message content, comprising:

employing a processor executing computer readable instructions stored on a computer readable storage medium to implement the following;

parsing a message to extract a set of features associated with a content type of the message, wherein the content type describes the type of data contained within a body of the message, the content type being case-sensitive and comprising a primary content-type, a secondary content-type, or a combination thereof;

examining the extracted set of features to identify a frequency of consecutiveness of repeating characters within a subject line of the message and to identify a distance of white-space characters between at least one alpha-numeric character and a blob comprising a random sequence of characters, numbers, punctuation, or a combination thereof to classify the message as spam or not spam;

establishing ranges of consecutive, repeating characters, the ranges correspond to various degrees of spaminess, wherein each range comprises a number range of frequencies of the consecutive, repeating characters within the subject line of the message;

employing the ranges to sort the message by the frequency of consecutive repeating characters within the subject line of the message;

comparing the content type of the message to stored content types of a plurality of other messages to facilitate determining whether the message is spam; and

processing the message as a function of the classification.

15. The method of claim 14, wherein the repeating characters comprise letters, numbers, punctuation, or white space.

16. One or more computer-readable storage devices having computer-executable instructions embodied thereon that, when executed, perform a method for facilitating extracting data in connection with spam processing, comprising:

receiving a message;

determining a particular portion of a body of the message to analyze;

extracting a set of features associated with some part, content or content type of the message;

examining consecutiveness of characters within a subject line of the message and identifying a distance comprising a number of white-space characters between at least one alpha-numeric character and a blob comprising a random sequence of characters, numbers, punctuation, or a combination thereof;

examining a content type of the message for spam in connection with building a filter, wherein the content type describes data contained within the body of the message, the content type being case-sensitive to capture a variation of a primary content-type, a secondary-content type, or a combination thereof, each of the primary content-type and the secondary-content type comprising one of a text, a multipart, a message, an image, an audio, a video, or an application;

comparing the content type of the message to stored content types of a plurality of other messages to facilitate determining whether the message is spam;

determining a percentage of white space to non-white space in the message and a percentage of non-white space and nonnumeric characters that are not letters in the message;

calculating a delivery time for the message using a first timestamp associated with origination of the message and a second timestamp associated with receipt of the message; and

categorizing the delivery time into one of a plurality of ranges comprising a range of amounts of time for delivering messages, the ranges corresponding to various degrees of spaminess.