Statistical message classifier

• US 9,386,046 B2
• Filed: 06/23/2014
• Issued: 07/05/2016
• Est. Priority Date: 07/22/2003
• Status: Expired due to Fees

• Current Assignee: Quest Software, Inc.
• Original Assignee: Dell Software, Inc. (Dell Technologies Inc.)
• Inventors: Oliver, Jonathan J., Roy, Scott, Eikenberry, Scott D., Kim, Bryan, Koblas, David A., Wilson, Brian K.
• Primary Examiner(s): Rahman, Shawnchoy
• Application Number:
  US14/312,645
• Publication Number:
  US 20140304829A1
• Time in Patent Office: 743 Days
• US Class Current: 1/1
• CPC Class Codes: G06N 20/00 Machine learning
• CPC Class Codes: H04L 51/212 using filtering or selective blocking
• CPC Class Codes: H04L 63/308 retaining data, e.g. retaining successful, unsuccessful communication attempts, internet access, or e-mail, internet telephony, intercept related information or call content
• CPC Class Codes: H04L 67/02 based on web technology, e.g. hypertext transfer protocol [HTTP]

Abstract

A system and method are disclosed for improving a statistical message classifier. A message may be tested with a machine classifier, wherein the machine classifier is capable of making a classification on the message. In the event the message is classifiable by the machine classifier, the statistical message classifier is updated according to the reliable classification made by the machine classifier. The message may also be tested with a first classifier. In the event that the message is not classifiable by the first classifier, it is tested with a second classifier, wherein the second classifier is capable of making a second classification. In the event that the message is classifiable by the second classifier, the statistical message classifier is updated according to the second classification.

Claims

1. A method for improving a statistical message classifier, the method comprising:

receiving a message over a network communication interface;

identifying by a processor executing instructions out of a memory a feature in the message that is associated with a junk count;

testing the message with a machine classifier by the processor performing a calculation according to a logarithmic function including the junk count, wherein the machine classifier outputs a value for the statistical message classifier for storage in the memory;

updating the statistical message classifier stored in the memory according to the logarithmic calculation made by the machine classifier, wherein the stored statistical message classifier corresponds to a probability that the feature in the message is spam;

identifying that the message is spam based on the probability that the feature in the message is spam; and

quarantining the message.

2. The method for improving a message classifier as recited in claim 1, wherein the machine classifier is further capable of making no classification on the message.

3. The method for improving a message classifier as recited in claim 1, wherein the machine classifier includes a white list classifier.

4. The method for improving a message classifier as recited in claim 1, wherein the machine classifier includes a collaborative fingerprinting classifier.

5. The method for improving a message classifier as recited in claim 1, wherein the machine classifier includes an image analyzer.

6. The method for improving a message classifier as recited in claim 1, wherein the machine classifier includes a probe account.

7. The method for improving a message classifier as recited in claim 1, wherein the machine classifier includes a challenge-response classifier.

8. The method for improving a message classifier as recited in claim 1, wherein updating the statistical message classifier comprises updating a knowledge base used to train the statistical message classifier.

9. The method for improving a message classifier as recited in claim 1, wherein updating the statistical message classifier comprises updating a statistical model used by the statistical message classifier.

10. The method for improving a message classifier as recited in claim 1, wherein updating the statistical message classifier comprises parsing the message to obtain the feature.

11. The method for improving a message classifier as recited in claim 1, wherein updating the statistical message classifier comprises parsing the message to obtain the feature and updating a counter corresponding to the feature.

12. The method for improving a message classifier as recited in claim 1, wherein updating the statistical message classifier comprises parsing the message to obtain the feature and updating a training set.

13. The method for improving a message classifier as recited in claim 1, wherein updating the statistical message classifier comprises parsing the message to obtain a feature and computing a spam probability associated with the feature.

14. The method for improving a message classifier as recited in claim 1, wherein updating the statistical message classifier comprises parsing the message to obtain the feature and computing a score associated with the feature.

15. The method for improving a message classifier as recited in claim 1, wherein the message is a previously stored message.

16. The method for improving a message classifier as recited in claim 1, wherein the message is an incoming message.

17. The method for improving a message classifier as recited in claim 1, in the event that the message is not classifiable by the classifier, further comprising testing the message with another machine classifier.

18. A method for improving a statistical message classifier comprising:

receiving a message over the network communication interface;

identifying by a processor executing instructions out of a memory a feature in the message that is associated with a junk count;

testing message with a first classifier by the processor performing a calculation according to a logarithmic function including the junk count, wherein the first classifier outputs a value for the statistical message classifier for storage in the memory and a first classification;

in the event that the message is classifiable by the first classifier, updating the statistical message classifier according to the logarithmic calculation;

in the event that the first classifier does not make the first classification, testing the message with a second classifier, wherein the second classifier is capable of making a second classification by the processor; and

updating the statistical message storage in the memory classifier according to the second classification by the processor, wherein the statistical message corresponds to a probability that the feature in the message is spam;

identifying that the message is not spam based on the probability that the feature in the message is spam; and

delivering the message to a recipient based on identifying that the message is not spam.

19. The method for improving a message classifier as recited in claim 18, wherein the second classifier is a reliable classifier having a probability of erroneous classification of less than one percent.

20. A method for improving a statistical message classifier comprising:

receiving a message over a network communication interface;

identifying by a processor executing instructions out of a memory a feature in the message that is associated with a good count;

testing the message with a first classifier wherein the first classifier is a reliable good classifier capable of making a first classification by the processor performing a calculation according to a logarithmic function including the good count, wherein the first classifier outputs a value for the statistical message classifier for storage in the memory;

in the event that the message is classifiable by the first classifier, updating the statistical message classifier stored in the memory according to the first classification by the processor performing the logarithmic calculation;

in the event that the first classifier does not make the first classification, testing the message with a second classifier, wherein the second classifier is capable of making a second classification; and

updating the statistical message classifier stored in the memory according to the second classification by the processor, wherein the statistical message classifier corresponds to a probability that the feature in the message is spam;

identifying that the message is not spam based on the probability that the feature in the message is spam; and

delivering the message to a recipient based on identifying that the message is not spam.

22. The method for improving a message classifier as recited in claim 20, wherein the second classifier is a reliable good classifier having a probability of erroneous classification of less than one percent.

21. A method for improving a statistical message classifier comprising:

receiving a message over a network communication interface;

identifying by a processor executing instructions out of a memory a feature in the message that is associated with a junk count;

testing the message with a first classifier by the processor, wherein the first classifier is a reliable junk classifier capable of making a first classification by the processor performing a calculation according to a logarithmic function including the junk count, wherein the first classifier outputs a value for the statistical message classifier for storage in the memory;

in the event that the message is classifiable by the first classifier, updating the statistical message classifier stored in the memory according to the first classification by the processor performing the logarithmic calculation;

in the event that the first classifier does not make the first classification, testing the message with a second classifier, wherein the second classifier is capable of making a second classification; and

updating the statistical message classifier stored in the memory according to the second classification by the processor, wherein the statistical message classifier corresponds to a probability that the feature in the message is spam;

identifying that the message is spam based on the probability that the feature in the message is spam; and

quarantining the message.

23. The method for improving a message classifier as recited in claim 21, wherein the second classifier is a reliable junk classifier having a probability of erroneous classification of less than one percent.

24. The method for improving a message classifier as recited in claim 21, wherein the first classifier is user-augmented.

25. A system for classifying a message, comprising:

a memory;

a processor; and

a network interface, wherein;

a message is received over the network communication interface;

the processor executing instructions out of the memory;

identifies a feature in the message that is associated with a junk count, wherein a statistical message classifier stored in the memory corresponds to a probability that the feature in the message is spam, and

tests the message with a message classifier by performing a calculation according to a logarithmic function including the junk count, machine classifier coupled to the statistical message classifier, the machine classifier configured to test the message and outputs a value for the statistical message classifier stored in the memory, and the statistical message classifier is updated according to the logarithmic calculation by the machine classifier and corresponds to a probability that the feature in the message is spam;

identifies that the message is spam based on the probability that the feature in the message is spam; and

quarantines the message.

26. A system for improving a statistical message classifier, comprising:

a memory;

a processor; and

a network interface, wherein;

a message is received over the network communication interface;

the processor executing instructions out of the memory;

identifies a feature in the message that is associated with a junk count tests the message using a first classifier making a first classification according to a calculation according to a logarithmic function including the junk count, the first classifier, the first classifier further outputs a value for the statistical message classifier for storage in the memory and updates the statistical message classifier according to the logarithmic calculation, wherein the statistical message corresponds to a probability that the feature in the message is spam; and

second classifier coupled to the first classifier, capable of reliably making a second classification, and configured to further test the message in the event that the message is not classifiable by the first classifier;

identifies that the message is spam based on the probability that the feature in the message is spam; and

quarantines the message.

27. A non-transitory computer readable storage medium having embodied thereon a program, the program being executable by a processor to perform a method for improving a statistical message classifier, the method comprising:

receiving a message over a network communication interface;

identifying by a processor executing instructions out of a memory a feature in the message that is associated with a junk count;

testing the message with a machine classifier by the processor performing a calculation according to a logarithmic function including the junk count, wherein the machine classifier is capable of making a classification of the message; and

updating the statistical message stored in the memory classifier according to the reliable classification made by the machine classifier, wherein the statistical message classifier stored in the memory corresponds to a probability that the feature in the message is spam;

identifying that the message is spam based on the probability that the feature in the message is spam; and

quarantining the message.

28. A non-transitory computer readable storage medium having embodied thereon a program, the program being executable to perform a method for improving a statistical message classifier, the method comprising:

receiving a message over a network communication interface;

identifying by a processor executing instructions out of a memory a feature in the message that is associated with a junk count

testing the message with a fist classifier wherein the first classifier by the processor performing a calculation according to a logarithmic function including the junk count;

in the event that the first classifier makes the classification, updating the statistical message classifier stored in the memory according to the first classification, wherein the statistical message classifier stored in the memory corresponds to a probability that the feature in the message is spam;

in the event that the first classifier does not make the classification, testing the message with a second classifier, wherein the second classifier is capable of reliably making a second classification; and

updating the statistical message classifier stored in the memory according to the second classifications;

identifying that the message is spam based on the probability that the feature in the message is spam; and

quarantining the message.