Method for statistical regression using ensembles of classification solutions

• US 20030033436A1
• Filed: 05/14/2001
• Published: 02/13/2003
• Est. Priority Date: 05/14/2001
• Status: Abandoned Application

• Current Assignee: International Business Machines Corporation
• Original Assignee: International Business Machines Corporation
• Inventors: Weiss, Sholom M.
• Application Number:
  US09/853,620
• Publication Number:
  US 20030033436A1
• Time in Patent Office: Days
• US Class Current: 719/310
• CPC Class Codes: G06F 18/24137 Distances to cluster centroïds
• CPC Class Codes: G06F 18/24765 Rule-based classification

Abstract

A pattern recognition method induces ensembles of decision rules from data regression problems. Instead of direct prediction of a continuous output variable, the method discretizes the variable by k-means clustering and solves the resultant classification problem. Predictions on new examples are made by averaging the mean values of classes with votes that are close in number to the most likely class.

Claims

1. A method for statistical regression using ensembles of classification solutions comprising the steps of:

running k-means clustering for k clusters on the set of values {y_l,i=1 . . . n};

recording a mean value m_j of a cluster c_j for j=1 . . . k;

transforming regression data into classification data with a class label for an i-th case being a cluster number of y_i;

applying ensemble classifier and obtain a set of rules R; and

making a prediction for new case u, using a margin of M, where 0≦

M≦

1.

2. The method recited in claim 1, wherein the step of making a prediction comprises the steps of:

applying all the rules R on the new case u;

for each class i, counting a number of satisfied rules (votes) v_i;

classifying t has the most votes, v_l;

considering a set of classes P={p} such that v_p≧

M·

v_t; and

generating a predicted output for case u,

$y_u^{\mathrm{\prime <br><br>}}=\frac{\underset{j\in <br><br>p}{\sum <br><br>}<br><br>m_j<br><br>v_j}{\underset{j\in <br><br>p}{\sum <br><br>}<br><br>v_j}.$

3. A method of pattern recognition comprising the steps of:

applying clustering processes to determine a number of classes;

applying ensemble learning classification processes to predict most likely classes for a new example; and

then averaging regression values of most likely classes to predict a value of a new example.

4. A method of pattern recognition for a set of values, said method comprising the steps of:

determining a number of classes to be generated based on a trend of error of a class mean/median for the set of values;

classifying the values using ensemble learning classification and the determined number of classes;

generating a set of classification rules; and

averaging regression values of most likely classes to predict a value of a new example based on the set of rules.

5. A method of pattern recognition according to claim 4, wherein said step of determining a number of classes comprises the steps of:

determining the class mean/median for a variable number of classes;

determining a mean absolute deviation (MAD) based on the class means/medians; and

comparing the MAD to a predetermined percentage of MAD.

6. A method of pattern recognition according to claim 4, wherein the step of averaging regression values includes using margins for predicting the value of the new example.

7. A method of pattern recognition according to claim 4, wherein the step of averaging regression values comprises the steps of:

applying the set of classification rules to the new example;

for each class i, counting a number of satisfied rules (votes) v_i;

classifying t has the most votes, v_l;

considering a set of classes P={p} such that v_p≧

M·

v_l; and

generating a predicted output for case u,

$y_u^{\mathrm{\prime <br><br>}}=\frac{\underset{j\in <br><br>p}{\sum <br><br>}<br><br>m_j<br><br>v_j}{\underset{j\in <br><br>p}{\sum <br><br>}<br><br>v_j}.$