Document classifiers and methods for document classification

• US 7,499,591 B2
• Filed: 03/25/2005
• Issued: 03/03/2009
• Est. Priority Date: 03/25/2005
• Status: Expired due to Fees

• Current Assignee: Hewlett-Packard Development Company, L.P. (HP Inc.)
• Original Assignee: Hewlett-Packard Development Company, L.P. (HP Inc.)
• Inventors: Wright, David W., Simske, Steven J., Sturgill, Margaret M.
• Primary Examiner(s): Desire; Gregory M
• Application Number:
  US11/090,788
• Publication Number:
  US 20060218134A1
• Time in Patent Office: 1,439 Days
• Field of Search: 382/224, 382/225, 382/159, 382/227, 707/1, 707/2, 707/3, 707/4, 707/5, 707/6, 707/102, 707/204, 707/7, 715/234
• US Class Current: 382/224
• CPC Class Codes: G06F 16/35 Clustering; Classification
• CPC Class Codes: Y10S 707/99936 Pattern matching access
• CPC Class Codes: Y10S 707/99937 Sorting

Abstract

A method of classifying a document includes providing a plurality of classifier engines and classifying the document using output from one or more of the classifier engines based on a comparison of one or more metrics for each classifier engine. In another embodiment, a method of classifying a document comprises providing a plurality of classifier engines and determining one or more metrics for each classifier engine. These metrics are used to determine how to use the classifier engines to classify the document, and the document is classified accordingly. A further embodiment includes a document classifier utilizing a plurality of classifier engines. In yet another embodiment, a computer-readable medium contains instructions for controlling a computer system to perform a method of using a plurality of classifier engines to classify a document.

Claims

1. A method of classifying a document comprising:

using a processor (or computer) to perform the steps of;

classifying said document using output from one or more of said classifier engines based on a comparison of one or more metrics for each classifier engine;

using a set of training documents to determine a precision value for each classifier engine;

identifying a highest precision value and a second highest precision value;

determining if said highest precision value is greater than said second highest precision value by a predetermined amount;

if so, using output from the classifier engine having said highest precision value to classify said document; and

if not, generating for each classifier engine, a list of probabilities of said document being classified by each classifier engine into each one of a group of possible classes, summing said probabilities for each class, and classifying said document into the class with the largest sum of probabilities.

2. The method of claim 1 wherein classifying said document includes:

generating, for each classifier engine, a list of probabilities of said document being classified by each classifier engine into each one of a group of possible classes;

summing said probabilities for each class; and

classifying said document into the class with the largest sum of probabilities.

3. The method of claim 2 further comprising finding confidence values for each classifier engine and multiplying each classifier engine'"'"'s probabilities by the corresponding classifier engine confidence value prior to summing said probabilities for each class.

4. The method of claim 3 further comprising finding a transformation exponent for each classifier engine and transforming each classifier engine'"'"'s probabilities by raising the probabilities to a power of the corresponding transformation exponent prior to multiplying each classifier engine'"'"'s probabilities by the corresponding classifier engine confidence value.

5. The method of claim 4 wherein said transformation exponent for each classifier engine is iteratively optimized using a set of training documents.

6. The method of claim 1 further comprising finding confidence values for each classifier engine and multiplying each classifier engine'"'"'s probabilities by the corresponding classifier engine confidence value prior to summing said probabilities for each class.

7. A method of classifying a document comprising:

using a processor (or computer) to perform the steps of;

classifying said document using output from one or more of said classifier engines based on a comparison of one or more metrics for each classifier engine;

using a set of training documents to rank said classifier engines from best to worst;

generating, for each classifier engine, a list of probabilities of said document being classified by each classifier engine into each one of a group of possible classes;

determining if the best classifier engine has a highest probability that is greater than the highest probability of each other classifier engine by a predetermined amount;

if so, using output from said best classifier engine to classify said document; and

if not, summing said probabilities for each class, and classifying said document into the class with the largest sum of probabilities.

8. The method of claim 7 further comprising finding confidence values for each classifier engine and multiplying each classifier engine'"'"'s probabilities by the corresponding classifier engine confidence value prior to determining if the best classifier engine has a highest probability that is greater than the highest probability of each other classifier engine by a predetermined amount.

9. The method of claim 7 wherein said classifier engines are ranked from best to worst based on engine accuracy.

10. A method of classifying a document comprising:

using a processor (or computer) to perform the steps of;

classifying said document using output from one or more of said classifier engines based on a comparison of one or more metrics for each classifier engine;

using a set of training documents to rank said classifier engines from best to worst;

generating, for each classifier engine, a list of probabilities of said document being classified by each classifier engine into each one of a group of possible classes;

determining if the best classifier engine has a highest probability that exceeds a predetermined threshold;

if so, using output from said best classifier engine to classify said document; and

if not, summing said probabilities for each class, and classifying said document into the class with the largest sum of probabilities.

11. The method of claim 10 further comprising finding confidence values for each classifier engine and multiplying each classifier engine'"'"'s probabilities by the corresponding classifier engine confidence value prior to determining if the best classifier engine has a highest probability that exceeds a predetermined threshold.

12. The method of claim 10 wherein said classifier engines are ranked from best to worst based on engine accuracy.

13. A method of classifying a document comprising:

using a processor (or computer) to perform the steps of;

determining one or more metrics for each classifier engine;

using said metrics to determine how to use said classifier engines to classify said document; and

classifying said document accordingly, wherein

determining one or more metrics includes using a set of training documents to determine a precision value for each classifier engine and identifying a highest precision value and a second highest precision value, and wherein classifying said document includes;

determining if said highest precision value is greater than said second highest precision value by a predetermined amount;

if so, using output from the classifier engine having said highest precision value to classify said document; and

if not, generating, for each classifier engine, a list of probabilities of said document being classified by each classifier engine into each one of a group of possible classes, summing said probabilities for each class, and classifying said document into the class with the largest sum of probabilities.

14. The method of claim 13 wherein determining one or more metrics includes generating, for each classifier engine, a list of probabilities of said document being classified by each classifier engine into each one of a group of possible classes, and wherein classifying said document includes summing said probabilities for each class and classifying said document into the class with the largest sum of probabilities.

15. The method of claim 14 further comprising finding confidence values for each classifier engine and multiplying each classifier engine'"'"'s probabilities by the corresponding classifier engine confidence value prior to summing said probabilities for each class.

16. The method of claim 15 further comprising finding a transformation exponent for each classifier engine and transforming each classifier engine'"'"'s probabilities by raising the probabilities to a power of the corresponding transformation exponent prior to multiplying each classifier engine'"'"'s probabilities by the corresponding classifier engine confidence value.

17. The method of claim 16 wherein said transformation exponent for each classifier engine is iteratively optimized using a set of training documents.

18. The method of claim 13 further comprising finding confidence values for each classifier engine and multiplying each classifier engine'"'"'s probabilities by the corresponding classifier engine confidence value prior to summing said probabilities for each class.

19. A method of classifying a document comprising:

using a processor (or computer) to perform the steps of;

determining one or more metrics for each classifier engine;

using said metrics to determine how to use said classifier engines to classify said document;

classifying said document accordingly;

using said metrics to rank said classifier engines from best to worst, and wherein classifying said document includes;

generating, for each classifier engine, a list of probabilities of said document being classified by each classifier engine into each one of a group of possible classes;

determining if the best classifier engine has a highest probability that is greater than the highest probability of each other classifier engine by a predetermined amount;

if so, using output from said best classifier engine to classify said document; and

if not, summing said probabilities for each class, and classifying said document into the class with the largest sum of probabilities.

20. The method of claim 19 further comprising finding confidence values for each classifier engine and multiplying each classifier engine'"'"'s probabilities by the corresponding classifier engine confidence value prior to determining if the best classifier engine has a highest probability that is greater than the highest probability of each other classifier engine by a predetermined amount.

21. The method of claim 19 wherein said classifier engines are ranked from best to worst based on engine accuracy.

22. A method of classifying a document comprising:

using a processor (or computer) to perform the steps of;

determining one or more metrics for each classifier engine;

using said metrics to determine how to use said classifier engines to classify said document;

classifying said document accordingly;

using said metrics to rank said classifier engines from best to worst, and wherein classifying said document includes;

generating, for each classifier engine, a list of probabilities of said document being classified by each classifier engine into each one of a group of possible classes;

determining if the best classifier engine has a highest probability that exceeds a predetermined threshold;

if so, using output from said best classifier engine to classify said document; and

if not, summing said probabilities for each class, and classifying said document into the class with the largest sum of probabilities.

23. The method of claim 22 further comprising finding confidence values for each classifier engine and multiplying each classifier engine'"'"'s probabilities by the corresponding classifier engine confidence value prior to determining if the best classifier engine has a highest probability that exceeds a predetermined threshold.

24. The method of claim 22 wherein said classifier engines are ranked from best to worst based on engine accuracy.

25. A document classifier comprising:

a plurality of classifier engines;

means for generating a metric for each classifier engine;

means for comparing said metrics; and

means for classifying a document using output from one or more of said classifier engines in response to said means for comparing;

wherein

said means for generating a metric generates, for each classifier engine, a list of probabilities of said document being classified by each classifier engine into each one of a group of possible classes, and wherein said means for comparing sums said probabilities for each class and identifies the class with the largest sum of probabilities, wherein

said means for generating a metric further generates a precision value for each classifier engine, and wherein said means for comparing identifies a highest precision value and a second highest precision value and determines if said highest precision value is greater than said second highest precision value by a predetermined amount, and wherein said means for classifying classifies said document using output from the classifier engine having said highest precision value to classify said document if said highest precision value is greater than said second highest precision value by a predetermined amount and classifies said document into the class with the largest sum of probabilities if said highest precision value is not greater than said second highest precision value by a predetermined amount.

26. The document classifier of claim 25 further comprising means for weighting said probabilities for each class.

27. The document classifier of claim 25 further comprising means for transforming said probabilities for each class.

28. The document classifier of claim 25 wherein said means for classifying classifies said document into the class with the largest sum of probabilities.