Fuzzy logic voting method and system for classifying e-mail using inputs from multiple spam classifiers

• US 7,680,890 B1
• Filed: 03/31/2006
• Issued: 03/16/2010
• Est. Priority Date: 06/22/2004
• Status: Active Grant

• Current Assignee: Musarubra US LLC (Musarubra US SellCo LLC)
• Original Assignee: McAfee, Inc. (McAfee, LLC)
• Inventors: Lin, Wei
• Primary Examiner(s): NEURAUTER, GEORGE C
• Application Number:
  US11/394,890
• Time in Patent Office: 1,446 Days
• Field of Search: 706 1- 9, 706 15- 25, 706/900, 709/206
• US Class Current: 709/206
• CPC Class Codes: G06Q 10/107 Computer-aided management of electronic mailing [e-mailing]

Abstract

Systems and methods for classifying e-mail messages as spam by combining outputs of a plurality of spam classifiers or classification tools using a fuzzy logic voting algorithm or formula are provided. According to one embodiment, a first classification value associated with an e-mail message and a second classification value associated with the e-mail message are received. The first classification value and the second classification value being indicative of whether the e-mail message is spam. Then, a single, aggregated classification value for the e-mail message is generated by combining the first classification value and the second classification value using a fuzzy logic-based voting mechanism.

Claims

1. A computer-implemented method comprising:

receiving a first classification value associated with an e-mail message that is an output of a first classification tool and a second classification value associated with said e-mail message that is an output of a second classification tool, said first classification value and said second classification value indicative of whether said e-mail message is spam; and

generating a single, aggregated classification value for said e-mail message by combining said first classification value and said second classification value using a fuzzy logic-based voting mechanism,

wherein said first classification value and said second classification value represent probabilities P₁ and P₂, respectively, and said single, aggregated classification value represents a combined probability P_combined, and wherein said fuzzy logic-based voting mechanism includes a voting formula comprising;

P_combined=(P₁×

P₂)/((P₁×

P₂)+(1−

P₁)(1−

P₂)).

2. The method of claim 1, further comprising:

standardizing said first classification value and said second classification value prior to said combining; and

identifying said e-mail message as spam by comparing said single, aggregated classification value to at least one spam threshold value.

3. The method of claim 1, wherein said first classification value has an associated first confidence factor and said second classification value has an associated second confidence factor, and wherein said combining said first classification value and said second classification value produces a confidence factor associated with said single, aggregated classification value that is greater than said first confidence factor and said second confidence factor.

4. The method of claim 1, wherein said first classification value and said second classification value are each is an output of a classification tool using at least one of a domain level blacklist, a domain level whitelist, a heuristics engine, a statistical classification engine, a checksum clearinghouse, a honeypot, an authenticated e-mail system, reputation information, message signatures, and sender behavior analysis.

5. The method of claim 1, wherein said generating of said single, aggregated classification value further comprises tuning said first classification value and said second classification value based on historical effectiveness data.

6. The method of claim 1, wherein said fuzzy logic-based voting mechanism takes into consideration confidence factors based on historical performance of a first classifier that generated said first classification value and a second classifier that generated said second classification value.

7. The method of claim 1, further comprising:

receiving at least one additional classification value associated with said e-mail message that is an output of at least one additional classification tool, said at least one additional classification value indicative of whether said e-mail message is spam; and

generating said single, aggregated classification value for said e-mail message by combining said first classification value, said second classification, and said at least one additional classification value using said fuzzy logic-based voting mechanism.

8. A computer-implemented method for classifying an e-mail message, said method comprising:

receiving a first classification result representing an output of a first e-mail classification tool'"'"'s analysis of said e-mail message, said first classification result associated with a first confidence level;

receiving a second classification result representing an output of a second e-mail classification tool'"'"'s analysis of said e-mail message, said second classification result associated with a second confidence level;

achieving an improved confidence level over that of said first confidence level and said second confidence level by generating a single, aggregated classification value based on a combination of said first classification result and said second classification result using a fuzzy logic-based voting mechanism; and

determining whether said e-mail message is spam by comparing said single, aggregated classification value to a spam threshold value,

wherein said first classification result and said second classification result represent probabilities P₁ and P₂, respectively, and said single, aggregated classification value represents a combined probability P_combined, and wherein said fuzzy logic-based voting mechanism includes a voting formula comprising;

P_combined=(P₁×

P₂)/((P₁×

P₂)+(1−

P₁)(1−

P₂)).

9. The method of claim 8 further comprising:

receiving at least one additional classification result representing an output of at least one additional classification tool'"'"'s analysis of said e-mail message, said at least one additional classification result associated with at least one additional confidence level; and

achieving an improved confidence level over that of said first confidence level, said second confidence level, and said at least one additional confidence level by generating a single, aggregated classification value based on a combination of said first classification result, said second classification result, and said at least one additional classification result using a fuzzy logic-based voting mechanism.

10. A machine readable storage medium that stores instructions for a computer system to operate an e-mail classification system, said e-mail classification system comprising:

a plurality of spam classifiers produced by a plurality of spam classification tools, each spam classifier of said plurality of spam classifiers configured to determine whether an e-mail message is spam;

a classifier conversion module operating based on said instructions for said computer system and configured to convert outputs of said plurality of spam classifiers into standardized values indicative of a likelihood that said e-mail message is spam; and

a voting mechanism operating based on said instructions for said computer system and configured to combine said standardized values into a single, aggregated classification output indicative of whether said e-mail message is spam using a fuzzy logic-based voting formula,

wherein said standardized values comprise at least a first standardized value and a second standardized value, wherein said first standardized value and said second standardized value represent probabilities P₁ and P₂, respectively, and said single, aggregated classification output represents a combined probability P_combined, and wherein said fuzzy logic-based voting mechanism includes a voting formula comprising;

P_combined=(P₁×

P₂)/((P₁×

P₂)+(1×

P₁)(1−

P₂)).

11. The e-mail classification system of claim 10, further comprising a control console operating based on said instructions for said computer system, accessible through a user interface, and configured to allow an administrator to view and modify parameters associated with said voting mechanism to train said voting mechanism.

12. The e-mail classification system of claim 10, wherein said voting mechanism includes a tuning module configured to automatically train said voting mechanism based on historical data indicative of past performance of said plurality of spam classifiers.

13. The e-mail classification system of claim 10, further comprising a filter mechanism operating based on said instructions for said computer system and configured to receive said single, aggregated classifier output from said voting mechanism and make a determination as to whether said e-mail message should be delivered or quarantined.

14. A machine readable storage medium that stores instructions for a computer system to operate a voting mechanism, said voting mechanism comprising:

an e-mail classification receiving means of said voting mechanism operating on said computer system for receiving a plurality of e-mail classification results generated by a plurality of e-mail classification tools;

a confidence means of said voting mechanism operating on said computer system for assigning a confidence level to said plurality of e-mail classification results based at least in part on historical data associated with said plurality of e-mail classification tools; and

a voting chairman means of said voting mechanism operating on said computer system for combining said plurality of e-mail classification results into a final, aggregated e-mail classification value via a fuzzy logic-based voting formula,

wherein said plurality of e-mail classification results comprise at least a first e-mail classification result and a second e-mail classification result, wherein said e-mail classification result and said e-mail classification result represent probabilities P₁ and P₂, respectively, and said single, aggregated e-mail classification value represents a combined probability P_combined, and wherein said fuzzy logic-based voting mechanism includes a voting formula comprising;

P_combined=(P₁×

P₂)/((P₁×

P₂)+(1−

P₁)(1−

P₂)).

15. The voting mechanism of claim 14, further comprising a classifier conversion means of said voting mechanism operating on said computer system for converting each of said plurality of e-mail classification results into corresponding standardized classification results.

16. The voting mechanism of claim 14, wherein said voting chairman means fuzzy logic-based voting formula is operable on said standardized classification results to obtain said final, aggregated e-mail classification value.