METHODS FOR FEATURE SELECTION IN A LEARNING MACHINE

• US 20080215513A1
• Filed: 10/30/2007
• Published: 09/04/2008
• Est. Priority Date: 08/07/2000
• Status: Active Grant

• Current Assignee: Curtis Anderson, Health Discovery Corporation, James Roberts, Joe Mckenzie, Jules B. Paderewski, Julian N. Stern, Memorial Health Systems Incorporated, Timothy P. O'Hayer
• Original Assignee: Curtis Anderson, Garry L. Carls, Glynn Bergeron, James Roberts, Joe Mckenzie, John E. Matthews, Jules B. Paderewski, Julian N. Stern, Memorial Health Systems Incorporated, Peter J. Farley, Simpson K. Russell, Timothy P. O'Hayer
• Inventors: Weston, Jason Aaron Edward, Perez-Cruz, Fernando, Elisseeff, Andre', Schoelkopf, Bernard
• Granted Patent: US 7,624,074 B2
• Time in Patent Office: Days
• US Class Current: 706/13
• CPC Class Codes: G06F 18/2115 by evaluating different subsets according to an optimisation criterion, e.g. class separability, forward selection or backward elimination
• CPC Class Codes: G06N 20/00 Machine learning
• CPC Class Codes: G06N 20/10 using kernel methods, e.g. support vector machines [SVM]

Abstract

In a pre-processing step prior to training a learning machine, pre-processing includes reducing the quantity of features to be processed using feature selection methods selected from the group consisting of recursive feature elimination (RFE), minimizing the number of non-zero parameters of the system (l₀-norm minimization), evaluation of cost function to identify a subset of features that are compatible with constraints imposed by the learning set, unbalanced correlation score and transductive feature selection. The features remaining after feature selection are then used to train a learning machine for purposes of pattern classification, regression, clustering and/or novelty detection.

Claims

1. A computer-implemented method for selecting a subset of features for processing in a learning machine, wherein the features correspond to a dataset to be analyzed for patterns, the method comprising:

executing a feature selection algorithm on the dataset to identify the subset of features, wherein the algorithm is selected from the group consisting of l₀-norm minimization and unbalanced correlation.

2. The method of claim 1, wherein the learning machine is a support vector machine.

3. The method of claim 1, wherein the algorithm is l₀-norm minimization and the dataset comprises a multi-label dataset, and further comprising calculating a label set size.

4. The method of claim 3, wherein the step of calculating label set size comprises minimizing a ranking loss.

5. The method of claim 3, wherein the dataset comprises gene expression data obtained from DNA micro-arrays.

6. The method of claim 1, wherein the algorithm is l₀-norm minimization and the algorithm comprises approximating l₀-norm by minimizing l₁-norm.

7. The method of claim 1, wherein the algorithm is l₀-norm minimization and the algorithm comprises approximating l₀-norm by minimizing l₂-norm.

8. The method of claim 1, further comprising mapping the dataset into feature space prior to executing the feature selection algorithm.

9. The method of claim 1, wherein the algorithm is unbalanced correlation.

10. The method of claim 9, wherein the dataset comprises labeled and unlabeled data, the method further comprises using transductive learning to classify the unlabeled data prior to executing the feature selection algorithm.

11. The method of claim 8, wherein the dataset comprises gene expression data obtained from DNA micro-arrays.

12. A computer-implemented method for selecting a subset of features for processing in a learning machine, wherein the features correspond to a dataset to be analyzed for patterns, the method comprising:

executing a feature selection algorithm on the dataset to identify the subset of features, wherein the algorithm comprises l₀-norm minimization.

13. The method of claim 12, wherein the learning machine is a support vector machine.

14. The method of claim 12, wherein the dataset comprises a multi-label dataset, and further comprising calculating a label set size.

15. The method of claim 14, wherein the step of calculating label set size comprises minimizing a ranking loss.

16. The method of claim 12, wherein the dataset comprises gene expression data obtained from DNA micro-arrays.

17. The method of claim 12, wherein the algorithm comprises approximating l₀-norm by minimizing l₁-norm.

18. The method of claim 12, wherein the algorithm comprises approximating l₀-norm by minimizing l₂-norm.

19. The method of claim 12, further comprising mapping the dataset into feature space prior to executing the feature selection algorithm.