Document classifiers and methods for document classification

• US 20060218134A1
• Filed: 03/25/2005
• Published: 09/28/2006
• Est. Priority Date: 03/25/2005
• Status: Active Grant

• Current Assignee: Hewlett-Packard Development Company, L.P. (HP Inc.)
• Original Assignee: Hewlett-Packard Development Company, L.P. (HP Inc.)
• Inventors: Simske, Steven J., Wright, David W., Sturgill, Margaret M.
• Granted Patent: US 7,499,591 B2
• Time in Patent Office: Days
• US Class Current: 1/1
• CPC Class Codes: G06F 16/35 Clustering; Classification
• CPC Class Codes: Y10S 707/99936 Pattern matching access
• CPC Class Codes: Y10S 707/99937 Sorting

Abstract

A method of classifying a document includes providing a plurality of classifier engines and classifying the document using output from one or more of the classifier engines based on a comparison of one or more metrics for each classifier engine. In another embodiment, a method of classifying a document comprises providing a plurality of classifier engines and determining one or more metrics for each classifier engine. These metrics are used to determine how to use the classifier engines to classify the document, and the document is classified accordingly. A further embodiment includes a document classifier utilizing a plurality of classifier engines. In yet another embodiment, a computer-readable medium contains instructions for controlling a computer system to perform a method of using a plurality of classifier engines to classify a document.

Claims

1. A method of classifying a document comprising:

providing a plurality of classifier engines; and

classifying said document using output from one or more of said classifier engines based on a comparison of one or more metrics for each classifier engine.

2. The method of claim 1 wherein classifying said document includes:

generating, for each classifier engine, a list of probabilities of said document being classified by each classifier engine into each one of a group of possible classes;

summing said probabilities for each class; and

classifying said document into the class with the largest sum of probabilities.

3. The method of claim 2 further comprising finding confidence values for each classifier engine and multiplying each classifier engine'"'"'s probabilities by the corresponding classifier engine confidence value prior to summing said probabilities for each class.

4. The method of claim 3 further comprising finding a transformation exponent for each classifier engine and transforming each classifier engine'"'"'s probabilities by raising the probabilities to a power of the corresponding transformation exponent prior to multiplying each classifier engine'"'"'s probabilities by the corresponding classifier engine confidence value.

5. The method of claim 4 wherein said transformation exponent for each classifier engine is iteratively optimized using a set of training documents.

6. The method of claim 1 wherein classifying said document includes:

using a set of training documents to determine a precision value for each classifier engine;

identifying a highest precision value and a second highest precision value;

determining if said highest precision value is greater than said second highest precision value by a predetermined amount;

if so, using output from the classifier engine having said highest precision value to classify said document; and

if not, generating, for each classifier engine, a list of probabilities of said document being classified by each classifier engine into each one of a group of possible classes, summing said probabilities for each class, and classifying said document into the class with the largest sum of probabilities.

7. The method of claim 6 further comprising finding confidence values for each classifier engine and multiplying each classifier engine'"'"'s probabilities by the corresponding classifier engine confidence value prior to summing said probabilities for each class.

8. The method of claim 1 wherein classifying said document includes:

using a set of training documents to rank said classifier engines from best to worst;

generating, for each classifier engine, a list of probabilities of said document being classified by each classifier engine into each one of a group of possible classes;

determining if the best classifier engine has a highest probability that is greater than the highest probability of each other classifier engine by a predetermined amount;

if so, using output from said best classifier engine to classify said document; and

if not, summing said probabilities for each class, and classifying said document into the class with the largest sum of probabilities.

9. The method of claim 8 further comprising finding confidence values for each classifier engine and multiplying each classifier engine'"'"'s probabilities by the corresponding classifier engine confidence value prior to determining if the best classifier engine has a highest probability that is greater than the highest probability of each other classifier engine by a predetermined amount.

10. The method of claim 8 wherein said classifier engines are ranked from best to worst based on engine accuracy.

11. The method of claim 1 wherein classifying said document includes:

using a set of training documents to rank said classifier engines from best to worst;

generating, for each classifier engine, a list of probabilities of said document being classified by each classifier engine into each one of a group of possible classes;

determining if the best classifier engine has a highest probability that exceeds a predetermined threshold;

if so, using output from said best classifier engine to classify said document; and

if not, summing said probabilities for each class, and classifying said document into the class with the largest sum of probabilities.

12. The method of claim 11 further comprising finding confidence values for each classifier engine and multiplying each classifier engine'"'"'s probabilities by the corresponding classifier engine confidence value prior to determining if the best classifier engine has a highest probability that exceeds a predetermined threshold.

13. The method of claim 11 wherein said classifier engines are ranked from best to worst based on engine accuracy.

14. A method of classifying a document comprising:

providing a plurality of classifier engines;

determining one or more metrics for each classifier engine;

using said metrics to determine how to use said classifier engines to classify said document; and

classifying said document accordingly.

15. The method of claim 14 wherein determining one or more metrics includes generating, for each classifier engine, a list of probabilities of said document being classified by each classifier engine into each one of a group of possible classes, and wherein classifying said document includes summing said probabilities for each class and classifying said document into the class with the largest sum of probabilities.

16. The method of claim 15 further comprising finding confidence values for each classifier engine and multiplying each classifier engine'"'"'s probabilities by the corresponding classifier engine confidence value prior to summing said probabilities for each class.

17. The method of claim 16 further comprising finding a transformation exponent for each classifier engine and transforming each classifier engine'"'"'s probabilities by raising the probabilities to a power of the corresponding transformation exponent prior to multiplying each classifier engine'"'"'s probabilities by the corresponding classifier engine confidence value.

18. The method of claim 17 wherein said transformation exponent for each classifier engine is iteratively optimized using a set of training documents.

19. The method of claim 14 wherein determining one or more metrics includes using a set of training documents to determine a precision value for each classifier engine and identifying a highest precision value and a second highest precision value, and wherein classifying said document includes:

determining if said highest precision value is greater than said second highest precision value by a predetermined amount;

if so, using output from the classifier engine having said highest precision value to classify said document; and

if not, generating, for each classifier engine, a list of probabilities of said document being classified by each classifier engine into each one of a group of possible classes, summing said probabilities for each class, and classifying said document into the class with the largest sum of probabilities.

20. The method of claim 19 further comprising finding confidence values for each classifier engine and multiplying each classifier engine'"'"'s probabilities by the corresponding classifier engine confidence value prior to summing said probabilities for each class.

21. The method of claim 14 further comprising using said metrics to rank said classifier engines from best to worst, and wherein classifying said document includes:

generating, for each classifier engine, a list of probabilities of said document being classified by each classifier engine into each one of a group of possible classes;

determining if the best classifier engine has a highest probability that is greater than the highest probability of each other classifier engine by a predetermined amount;

if so, using output from said best classifier engine to classify said document; and

if not, summing said probabilities for each class, and classifying said document into the class with the largest sum of probabilities.

22. The method of claim 21 further comprising finding confidence values for each classifier engine and multiplying each classifier engine'"'"'s probabilities by the corresponding classifier engine confidence value prior to determining if the best classifier engine has a highest probability that is greater than the highest probability of each other classifier engine by a predetermined amount.

23. The method of claim 21 wherein said classifier engines are ranked from best to worst based on engine accuracy.

24. The method of claim 14 further comprising using said metrics to rank said classifier engines from best to worst, and wherein classifying said document includes:

generating, for each classifier engine, a list of probabilities of said document being classified by each classifier engine into each one of a-group of possible classes;

determining if the best classifier engine has a highest probability that exceeds a predetermined threshold;

if so, using output from said best classifier engine to classify said document; and

if not, summing said probabilities for each class, and classifying said document into the class with the largest sum of probabilities.

25. The method of claim 24 further comprising finding confidence values for each classifier engine and multiplying each classifier engine'"'"'s probabilities by the corresponding classifier engine confidence value prior to determining if the best classifier engine has a highest probability that exceeds a predetermined threshold.

26. The method of claim 24 wherein said classifier engines are ranked from best to worst based on engine accuracy.

27. A document classifier comprising:

a plurality of classifier engines;

means for generating a metric for each classifier engine;

means for comparing said metrics; and

means for classifying a document using output from one or more of said classifier engines in response to said means for comparing.

28. The document classifier of claim 27 wherein said means for generating a metric generates, for each classifier engine, a list of probabilities of said document being classified by each classifier engine into each one of a group of possible classes, and wherein said means for comparing sums said probabilities for each class and identifies the class with the largest sum of probabilities.

29. The document classifier of claim 28 further comprising means for weighting said probabilities for each class.

30. The document classifier of claim 28 further comprising means for transforming said probabilities for each class.

31. The document classifier of claim 28 wherein said means for classifying classifies said document into the class with the largest sum of probabilities.

32. The document classifier of claim 28 wherein said means for generating a metric further generates a precision value for each classifier engine, and wherein said means for comparing identifies a highest precision value and a second highest precision value and determines if said highest precision value is greater than said second highest precision value by a predetermined amount, and wherein said means for classifying classifies said document using output from the classifier engine having said highest precision value to classify said document if said highest precision value is greater than said second highest precision value by a predetermined amount and classifies said document into the class with the largest sum of probabilities if said highest precision value is not greater than said second highest precision value by a predetermined amount.

33. The document classifier of claim 28 wherein said means for generating a metric further generates an accuracy value for each classifier engine, and wherein said means for comparing determines if the classifier engine having the highest accuracy value has a highest probability that is greater than the highest probability of each other classifier engine by a predetermined amount, and wherein said means for classifying classifies said document using output from the classifier engine having the highest accuracy value to classify said document if the classifier engine having the highest accuracy value has a highest probability that is greater than the highest probability of each other classifier engine by a predetermined amount and classifies said document into the class with the largest sum of probabilities if the classifier engine having the highest accuracy value has a highest probability that is not greater than the highest probability of each other classifier engine by a predetermined amount.

34. The document classifier of claim 28 wherein said means for generating a metric further generates an accuracy value for each classifier engine, and wherein said means for comparing determines if the classifier engine having the highest accuracy value has a highest probability that exceeds a predetermined threshold, and wherein said means for classifying classifies said document using output from the classifier engine having the highest accuracy value to classify said document if the classifier engine having the highest accuracy value has a highest probability that exceeds a predetermined threshold and classifies said document into the class with the largest sum of probabilities if the classifier engine having the highest accuracy value has a highest probability that does not exceed a predetermined threshold.

35. A computer-readable medium containing instructions for controlling a computer system to perform a method of using a plurality of classifier engines to classify a document, wherein said method comprises classifying said document using output from one or more of said classifier engines based on a comparison of one or more metrics for each classifier engine.

36. A computer-readable medium containing instructions for controlling a computer system to perform a method of using a plurality of classifier engines to classify a document, wherein said method comprises:

determining one or more metrics for each classifier engine;

using said metrics to determine how to use said classifier engines to classify said document; and

classifying said document accordingly.