Predictive model generation

• US 7,933,762 B2
• Filed: 04/16/2004
• Issued: 04/26/2011
• Est. Priority Date: 04/16/2004
• Status: Active Grant

• Current Assignee: Longhorn Automotive Group LLC (Alpha Alpha Intellectual Partners LLC)
• Original Assignee: Fortelligent, Inc.
• Inventors: Jacobs, Marc, Mansfield, Richard J. W., Rubin, Donald B., Pinto, Stephen K., Hirshberg, Jay C.
• Primary Examiner(s): Rodriguez; Paul L
• Assistant Examiner(s): OCHOA, JUAN CARLOS
• Application Number:
  US10/826,630
• Publication Number:
  US 20050234688A1
• Time in Patent Office: 2,566 Days
• Field of Search: 703/22, 706/45, 706/46
• US Class Current: 703/22
• CPC Class Codes: G05B 17/02 electric

Abstract

Models are generated using a variety of tools and features of a model generation platform. For example, in connection with a project in which a user generates a predictive model based on historical data about a system being modeled, the user is provided through a graphical user interface a structured sequence of model generation activities to be followed, the sequence including dimension reduction, model generation, model process validation, and model re-generation.

Historical multi-dimensional data is received representing multiple variables transformed to be maximally predictive for at least one outcome variable to be used as an input to a predictive model of a commercial system, model development process is validated for at one or more sets of such variables and enabling a user of a model generation tool to combine at least two of the variables from the sets of variables.

Claims

1. A machine-based method comprising:

in connection with a project in which a predictive model is generated based on historical data about a system being modeled

selecting variables having at least a first predetermined level of significance from a pool of potential predictor variables associated with the historical data, to form a first population of predictor variables,

extending the first population of predictor variables to include cross products of at least two variables, each being from the first population of predictor variables,

selecting variables having at least a second predetermined level of significance from the extended first population of predictor variables to form a second population of predictor variables,

extending the second population of predictor variables to include cross products of at least two variables, at least one of the variables for at least one of the cross products being from the pool of potential predictor variables that are associated with the historical data and having less than the first predetermined level of significance,

selecting variables having at least a third predetermined level of significance from the extended second population of predictor variables to form a third population of predictor variables,

automatically selecting a model generation method from a set of available model generation methods to match characteristics of the historical data,

generating a possible model of the third population of predictor variables using a subsample of the historical data by the model generation method,

determining whether the possible model generalizes to the historical data other than the subsample,

applying the possible model to all of the historical data to generate a final model, cross-validating the final model using random portions of the historical data, and interacting with the system being modeled based on the final model.

2. The method of claim 1 also including displaying information to a user of the size of the pool of potential predictor variables.

3. The method of claim 1 also including enabling a user to point and click to reduce or extend the size of the pool of potential predictor variables, retaining only predictor variables having at least the second predetermined level of significance.

4. The method of claim 1 in which a user is enabled to invoke an automatic process to select a class of models most suitable to the pool of potential predictor variables associated with the historical data.

5. The method of claim 1 in which a user is enabled to point and click to adjust a criterion of the model selection to retain only variables having at least a certain level of significance for a target goal.

6. The method of claim 1 in which a user is enabled to point and click to cause display of information about performance of the possible model or the final model.

7. The method of claim 6 in which the information includes at least one of:

a statistical report card, a link to a statistical report chart, a lift chart, a link to the lift chart, a response comparison chart for each decile for each predictor variable in the possible model or the final model, or a link to the response comparison chart.

8. The method of claim 7 in which invocation of the link to the statistical report card causes display of the statistics of the performance of the possible model or the final model.

9. The method of claim 7 in which invocation of the link to the lift chart causes display of a non-cumulative lift chart.

10. The method of claim 7 in which invocation of the link to the response comparison chart causes display of a response chart for each predictor variable in the possible model or the final model for each segment of interest.

11. The method of claim 1 in which a user is enabled to choose interactively at least one performance criterion change or transformation or interaction of variables to improve a fit of the possible model or the final model.

12. The method of claim 1 in which a user is enabled to select at least one validation dataset and invoke a model process validation method.

13. The method of claim 12 in which the user is enabled to point and click to cause display of information about the model process validation.

14. The method of claim 13 in which the information about the model process validation includes at least one of:

a statistical report card, a link to a statistical report chart, a cumulative lift chart, a link to the cumulative lift chart, a non-cumulative lift chart, a link to the non-cumulative lift chart.

15. The method of claim 1 in which a user is enabled to select at least one machine automated model development process applied to the entire set of historical data for a validated model process.

16. The method of claim 15 in which the user is enabled to point and click to cause display of information about the performance of the validated model process applied to the entire set of historical data.

17. The method of claim 16 in which the information about the performance comprises information about the performance of the validated model process applied to two independent data subsets, the independent data subsets being randomly selected from the historical data, and includes at least one of:

a statistical report card, a link to a statistical report chart, a cumulative lift chart, a link to the cumulative lift chart, a non-cumulative lift chart, a link to the non-cumulative lift chart.

18. The method of claim 17 in which the invocation of the link to the statistical report card causes display of the statistics of model process validation.

19. The method of claim 17 in which the final model and the model process validation results are stored persistently.

20. The method of claim 1 also including enabling a user to observe the number of predictor variables available for generating the predictive model.

21. The method of claim 1 also including enabling a user to observe the performance of the possible model or the final model by means of links to a plurality of statistical and graphical reports.

22. The method of claim 1 also enabling a user to select a means of validating the selected model generation method.

23. The method of claim 1 also including enabling a user to observe the performance of the possible model or the final model when applied to a training subset and a validation subset of the historical data.

24. The method of claim 1 also including enabling a user to invoke at least one validated model generation method to produce a final model and enabling the user to observe performance of the final model on at least two independent subsets, the independent subsets being randomly selected from the historical data.

25. The method of claim 1 also including enabling the persisting of the final model and intermediate results to a project database.

26. The method of claim 1 also including enabling the final model to be applied to scoring at least one non-historical dataset wherein a propensity computed by the model is indexed by the score.