APPLYING NON-LINEAR TRANSFORMATION OF FEATURE VALUES FOR TRAINING A CLASSIFIER

• US 20110029463A1
• Filed: 07/30/2009
• Published: 02/03/2011
• Est. Priority Date: 07/30/2009
• Status: Active Grant

• Current Assignee: Micro Focus LLC (Open Text Corporation)
• Original Assignee: Hewlett-Packard Development Company, L.P. (HP Inc.)
• Inventors: Scholz, Martin B., Rajaram, Shyam Sundar, FORMAN, GEORGE H.
• Granted Patent: US 8,725,660 B2
• Time in Patent Office: Days
• US Class Current: 706/12
• CPC Class Codes: G06F 16/355 Class or cluster creation or modification

Abstract

A collection of labeled training cases is received, where each of the labeled training cases has at least one original feature and a label with respect to at least one class. Non-linear transformation of values of the original feature in the training cases is applied to produce transformed feature values that are more linearly related to the class than the original feature values. The non-linear transformation is based on computing probabilities of the training cases that are positive with respect to the at least one class. The transformed feature values are used to train a classifier.

Claims

1. A method comprising:

receiving a collection of labeled training cases, wherein each of the labeled training cases has at least one original feature and a label with respect to at least one class;

applying, by a computer using an algorithm, non-linear transformation on values of the original feature in the training cases to produce transformed feature values that are more linearly related to the class than the original feature values, wherein the non-linear transformation is based on computing probabilities of the training cases that are positive with respect to the at least one class; and

using, by the computer, the transformed feature values to train a classifier.

2. The method of claim 1, further comprising:

receiving a target case having the original feature for classification by the trained classifier;

applying the non-linear transformation on a value of the original feature in the target case to produce a transformed feature value; and

using the transformed feature values of the target case by the trained classifier to classify the target case with respect to the class.

3. The method of claim 1, wherein the labeled training cases further have a second original feature, the method further comprising:

applying non-linear transformation of values of the second original feature in the training cases to produce transformed second feature values; and

using the transformed second feature values to train the classifier.

4. The method of claim 1, wherein the classifier is a multi-class classifier, wherein the receiving, applying, and using are performed for a first binary sub-task of the multi-class classifier, the method further comprising repeating the receiving, applying, and using for another binary sub-task of the multi-class classifier.

5. The method of claim 1, wherein applying the non-linear transformation of the values of the original feature comprises:

for a given value of the original feature, determining a proportion of the training cases having the given value of the particular original feature that are labeled positive with respect to the class.

6. The method of claim 5, wherein determining the proportion of the training cases having the given value of the original feature that are labeled positive with respect to the class comprises:

calculating a first count of the training cases having the given value of the original feature that are labeled positive with respect to the class;

calculating a second count of the training cases having the given value of the original feature that are labeled negative with respect to the class; and

dividing the first count by a sum of the first count and second count.

7. The method of claim 5, wherein determining a proportion of the training cases having the given value of the original feature that are labeled positive with respect to the class is based on calculating a count of the training cases that contain k (k≧

2) nearest neighbors of the given value of the original feature that are labeled positive with respect to the class.

8. The method of claim 5, further comprising discretizing the values of the original feature into discrete bins, the first and second counts calculated with respect to the bins.

9. The method of claim 1, wherein applying the non-linear transformation of the values of the original feature comprises transforming the values of the original feature based on computing a first probability density function that represents a probability of the original feature assuming a given value while being conditioned on the fact that corresponding training cases are positive with respect to the class.

10. The method of claim 9, wherein transforming the values of the original feature is further based on computing a second probability density function that represents a probability of the original feature assuming the given value while being conditioned on the fact that corresponding training cases are negative with respect to the class.

11. The method of claim 10, wherein transforming the values of the original feature comprises calculating a difference between a log of the first probability density function and a log of the second probability density function.

12. The method of claim 1, wherein applying the non-linear transformation of the values of the original feature comprises transforming the values of the original feature by using points of a receiver operating characteristic curve.

13. The method of claim 12, wherein transforming the values of the original feature by using points of the receiver operating characteristic curve comprises calculating a difference between a true positive rate of a given value of the original feature and a false positive rate of the given value of the original feature.

14. The method of claim 1, wherein the training cases have plural features, the method further comprising:

computing scores for corresponding plural features of the training cases; and

scaling the plural features according to the computed scores such that wider ranges are provided to features with higher scores.

15. A computer comprising:

a storage media to store labeled training cases that are labeled with respect to at least one class; and

a processor to;

construct mapping functions that are learned from the labeled training cases;

use the mapping functions to apply corresponding non-linear transformations of original features in the training cases to produce transformed feature values from values of the original features; and

use the transformed feature values to train a classifier.

16. The computer of claim 15, wherein each of the mapping functions is based on a rate of positive training cases having corresponding values of a corresponding one of the original features with respect to the class.

17. The computer of claim 16, wherein each of the mapping functions is calculated by computing a first count and a second count,

wherein the first count is a count of the number of times the corresponding original feature assumes a given value while being conditioned on the fact that the corresponding training cases are positive with respect to the class, and

wherein the second count is a count of the number of times the corresponding original feature assumes a given value while being conditioned on the fact that the corresponding training cases are negative with respect to the class.

18. The computer of claim 17, wherein each of the mapping functions is calculated by computing a first count and a second count, wherein the first and second counts are computed based on using k nearest neighbors of each given value of a particular one of the original features.

19. The computer of claim 16, wherein each of the mapping functions is calculated based on computing a true positive rate and a false positive rate.

20. An article comprising at least one computer readable storage medium containing instructions that upon execution cause a computer to:

receive a collection of labeled training cases, wherein each of the labeled training cases has at least one original feature and a label with respect to at least one class;

construct a non-linear transformation based on the labeled training cases;

apply the non-linear transformation on values of the original feature in the training cases to produce transformed feature values that are more linearly related to the class than the original feature values; and

use the transformed feature values to train a classifier.