Method and structure for transform regression

• US 7,283,982 B2
• Filed: 12/05/2003
• Issued: 10/16/2007
• Est. Priority Date: 12/05/2003
• Status: Expired due to Fees

• Current Assignee: International Business Machines Corporation
• Original Assignee: International Business Machines Corporation
• Inventors: Pednault, Edwin Peter Dawson
• Primary Examiner(s): Knight; Anthony
• Assistant Examiner(s): KENNEDY, ADRIAN L
• Application Number:
  US10/727,578
• Publication Number:
  US 20050125474A1
• Time in Patent Office: 1,411 Days
• Field of Search: 706/16, 706/25, 706/62, 706/12, 706/14, 706/15, 706/21, 700/29, 700/30, 700/31, 700/44, 705/4, 705/7, 708/400
• US Class Current: 706/12
• CPC Class Codes: G06F 17/18 for evaluating statistical data , e.g. average values, frequency distributions, probability functions, regression analysis forecasting specially adapted for a specific administrative, business or logistic context G06Q10/04

Abstract

A predictive model method (and structure) includes receiving an input data into an initial model to develop an initial model output and receiving both of the input data and the initial model output as input data into a first transform/regression stage.

Claims

1. A predictive model method, comprising:

receiving first input data into an initial model to develop an initial model output;

receiving second input data and said initial model output as inputs into a first boosting stage to develop an improvement to said initial model output, said second input data comprising one of said first input data, data not included in said first input data, and a combination thereof; and

outputting a model output resulting from a final boosting stage being one of;

said first boosting stage; and

a final one of boosting stages successively receiving model output data from a preceding boosting stage.

2. The method of claim 1, wherein said first boosting stage comprises a transform/regression stage that comprises:

a feature transform stage receiving said second input data and said initial model output;

a linear regression stage receiving an output of said feature transform stage; and

an output summing node receiving as inputs said initial model output and an output of said linear regression stage, an output of said output summing node comprising a first boosting stage model output.

3. The method of claim 2, further comprising:

successively providing, for one or more transform/regression stages, an output of a preceding transform/regression stage as an input into a succeeding transform/regression stage.

4. The method of claim 3, wherein for at least one said one or more transform/regression stages, a third input into said succeeding transform/regression stage comprises an output of said linear regression stage of a preceding transform/regression stage.

5. The method of claim 3, further comprising:

avoiding an overfitting in said predictive model by determining when a successive transform/regression stage does not add to a performance of said predictive model.

6. The method of claim 5, wherein said determining of performance degradation comprises a holdout method, said holdout method comprising:

dividing an available data into a training set and a holdout data set;

using said training set to estimate a model parameter and to construct an alternative model structure; and

using said holdout data set to make a selection among said alternative model structure.

7. The method of claim 5, wherein said determining of performance degradation comprises a cross-validation method, said cross-validation method comprising:

dividing an available data into a plurality of folds of data; and

successively, using each said fold as a holdout data set, and a remaining data not in said fold is used as a training data set to estimate model parameters and to construct alternative model structures and said training data set is used to make a selection among said alternative model structures.

19. A method for deploying computing infrastructure, comprising integrating computer-readable code into a computing system, wherein the code in combination with the computing system is capable of performing the method of claim 1.

8. A predictive modeling method, comprising:

establishing an initial model module to instance an initial model;

establishing a boosting stage model module to instance a boosting stage model for each of one or more successive boosting stages, wherein at least one instanced boosting stage model receives, as input, an input data and an output from at least one of said initial model and a preceding boosting stage mode; and

providing an output resulting from a final boosting stage of said one or more successive boosting stages to be an output of the predictive model.

9. The method of claim 8, wherein at least one said boosting stage model feeds forward a second output as another input into at least one succeeding boosting stage model.

10. The method of claim 8, further comprising:

instancing said initial model;

successively instancing one or more of said boosting stage models to be successive boosting stage models, each said boosting stage model receiving, as input, an output from at least one of said initial model and a preceding boosting stage model;

providing an input data as input to said initial model; and

for each successive boosting stage model, providing an input data as input to said successive boosting stage model.

11. The method of claim 10, further comprising:

determining when an additional successive boosting stage would not add to a performance of the predictive model.

12. The method of claim 11, wherein said determining of performance degradation comprises a holdout method, said holdout method comprising:

dividing an available data into a training set and a holdout data set;

using said training set to estimate a model parameter and to construct alternative model structures; and

using said holdout data set to make a selection among said alternative model structures.

13. The method of claim 11, wherein said determining of performance degradation comprises a cross-validation method, said cross-validation method comprising:

dividing an available data into a plurality of folds of data;

successively, using each said fold as a holdout data set, and a remaining data not in said fold is used as a training data set to estimate model parameters and to construct alternative model structures and said training data set is used to make a selection among said alternative model structures.

14. The method of claim 8, wherein said boosting stage model comprises:

a first data input port;

a second data input port;

a feature transform stage receiving data from said first data input port and said second data input port;

a linear regression stage receiving an output from said feature transform stage;

a summing node receiving data from said first data input port and output data from said linear regression stage; and

an output port receiving data outputted from said summing node.

15. The method of claim 14, wherein said boosting stage model further comprises:

a second output port to provide said output data from said linear regression stage to be a second output from said boosting stage model; and

one or more input ports to receive data from said second output port of preceding boosting stages to be input data into said feature transform stage.

16. An apparatus to perform a predictive modeling method, said apparatus comprising:

an initial model module to instance an initial model;

a boosting stage model module to instance a boosting stage model for each of one or more successive boosting stages, wherein at least one said boosting stage model receives a data input and an input from a preceding boosting stage model;

a controller to cause said initial model and each of one or more said successive boosting stage models to be instanced and to interconnect said initial model and said plurality of successive boosting stage models; and

a graphic user interface to allow a user to control said controller and said predictive modeling method, to input data into said initial model, and to at least one of display, print, and save to one of a printer, a data file, and an application program the output resulting from a final one of said successive boosting stage models.

17. A machine-readable medium tangibly embodying a program of machine-readable instructions executable by a digital processing apparatus to perform a predictive modeling method, said instructions comprising:

an initial model module to instance an initial model;

a boosting stage model module to instance a boosting stage model for each of one or more successive boosting stages, wherein at least one instanced boosting stage model receives, as input, a data input and an output from at least one of said initial model and a preceding boosting stage model;

causing said initial model and each of one or more said successive boosting stage models to be instanced and to appropriately interconnect said initial model and said successive boosting stage models;

allowing a user to control said controller and said predictive modeling method, to input data into said initial model, and to one of display, print, and save to one of a printer, a data file, a memory, and an application program the output resulting from a final one of said successive boosting stage models;

receiving input data; and

allowing the output data of said predictive modeling method to be provided as output data.

18. A method of providing a service, said method comprising at least one of:

providing an execution of a predictive modeling method, wherein said predictive modeling method comprises;

establishing an initial model module to instance an initial model;

establishing a boosting stage model module to instance a boosting stage model for each of one or more successive boosting stages, wherein at least one instanced boosting stage model receives, as input, an input data and an output from at least one of said initial model or a preceding boosting stage model;

successively providing, selectively, for one or more additional boosting stages, an output of a preceding boosting stage as an input into a succeeding boosting stage; and

providing an output resulting from a final boosting stage to at least one of a display device, a printer, and a memory device as comprising a result of said predictive model.