Automated systems and methods for generating statistical models

• US 20060241923A1
• Filed: 05/02/2006
• Published: 10/26/2006
• Est. Priority Date: 08/02/2002
• Status: Abandoned Application

• Current Assignee: Capital One Financial Corporation
• Original Assignee: Capital One Financial Corporation
• Inventors: Xu, Cheng (Kenneth), Wachtell, Peter James
• Application Number:
  US11/415,427
• Publication Number:
  US 20060241923A1
• Time in Patent Office: Days
• US Class Current: 703/2
• CPC Class Codes: G06F 16/906 Clustering; Classification
• CPC Class Codes: G06F 18/21 Design or setup of recognition systems or techniques; Extraction of features in feature space; Blind source separation
• CPC Class Codes: G06Q 40/08 Insurance

Abstract

Systems and methods are disclosed for generating statistical models. Such systems and methods may utilize a database comprising data representing a plurality of variables. To generate a statistical model, a set of variables may be selected in accordance with a goal of the model. Using the database, the selected set of variables may then be applied to a plurality of statistical model types and the results from each statistical model type may be analyzed. Finally, at least one of statistical model may be identified based on the analysis of the results.

Claims

1. A method for generating a statistical model, comprising:

providing a database comprising data representing a plurality of variables;

selecting a set of variables in accordance with a goal for the statistical model;

applying the selected set of variables based on the data from the database to a plurality of statistical model types;

analyzing the results for each statistical model type; and

identifying at least one statistical model based on the analysis of the results.

2. A method according to claim 1, wherein the method further comprises cleaning the data in the database to impute missing or extreme values.

3. A method according to claim 1, wherein the method further comprises coding at least one dependent variable based on the goal of the model.

4. A method according to claim 1, wherein the set of variables comprise independent variables.

5. A method according to claim 4, wherein the method further comprises sorting and ordering the independent variables into groups.

6. A method according to claim 4, wherein selecting a set of variables comprises eliminating statistically redundant data by performing at least one of factor analysis, principal component and variable clustering.

7. A method according to claim 4, wherein selecting a set of variables comprises identifying relevant variables by performing at least a stepwise analysis of the variables.

8. A method according to claim 1, wherein the data representing the set of variables is provided as part of a data mart.

9. A method according to claim 8, wherein the method further comprises dividing the data in the data mart into a development sample and a validation sample.

10. A method according to claim 9, wherein applying the selected set of variables comprises applying data from the development sample to the plurality of statistical model types and applying data from the validation sample to the plurality of statistical model types.

11. A method according to claim 1, wherein applying the selected set of variables comprises applying data from the database to a plurality of statistical models types, including at least one of regression models, parametric models, non-parametric models, tree type models, and neural network models.

12. A method according to claim 1, wherein analyzing the results for each statistical model type comprises applying at least one benchmark measurement to determine the performance of each statistical model type with respect to the goal of the model.

13. A method according to claim 12, wherein applying at least one benchmark measurement comprises performing an analysis of the results using at least one of an R2 computation, Akaike'"'"'s information criteria (AIC), and Bayesian information criteria (BIC).

14. A method according to claim 1, wherein analyzing the results for each statistical model type comprises ranking model types according to the level of performance of the model with respect to the goal of the model.

15. A method according to claim 14, wherein identifying at least one statistical model comprises selecting the highest ranked model based on performance.

16. A method according to claim 1, wherein the method further comprises segmenting the database and generating a statistical model for each segment of the database.

17. A method according to claim 16, wherein the database is segmented consistent with at least one of business objectives and statistical objectives.

18. A method according to claim 1, wherein the method further comprises refreshing the statistical model after the model is generated.

19. A method according to claim 18, wherein refreshing the statistical model comprises refreshing the model in response to a refresh trigger, the refresh trigger comprising a predetermined event.

21. A method according to claim 19, wherein the predetermined event is at least one of an update to data in the database and a predetermined time period.

70. A method according to claim 19, wherein refreshing comprises analyzing the model to determine if the model satisfies a set of minimum threshold requirements.

22. A system for generating a statistical model, comprising:

a database comprising data representing a plurality of variables;

a statistical model generator to generate statistical models; and

a user interface to receive data and provide output,

wherein the statistical model generator includes means for;

applying a set of selected variables, based on the data from the database, to a plurality of statistical model types;

means for analyzing the results for each statistical model type; and

means for identifying at least one of statistical model based on the analysis of the results.

23. A system according to claim 22, wherein the statistical model generator comprises a data engine that is adapted to clean data in the database in order to impute missing or extreme values.

24. A system according to claim 23, wherein the data engine comprises means for comprises sorting and ordering the variables into groups.

25. A system according to claim 23, wherein the data is arranged as part of a data mart, and wherein the data engine comprises means for dividing the data in the data mart into a development sample and a validation sample.

26. A system according to claim 22, wherein the means for applying a set of selected variables comprises a model engine, the model engine being adapted to select a set of variables in accordance with a goal for the statistical model.

27. A system according to claim 26, wherein the model engine comprises means for eliminating statistically redundant variables by performing at least one of factor analysis, principal component and variable clustering.

28. A system according to claim 26, wherein the model engine further comprises means for identifying relevant variables by performing at least a stepwise analysis of the variables.

29. A system according to claim 22, wherein the means for analyzing the results for each statistical model type and means for identifying at least one of statistical model comprise a statistical model generator.

30. A system according to claim 22, wherein the means for applying the set of selected variables applies data from the database to a plurality of statistical models types, including at least one of regression models, parametric models, non-parametric models, tree type models, and neural network models.

31. A system according to claim 22, wherein the means for analyzing the results for each statistical model type applies at least one benchmark measurement to determine the performance of each statistical model type with respect to a goal of the model.

32. A system according to claim 31, wherein the benchmark measurement is based on at least one of an R2 computation, Akaike'"'"'s information criteria (AIC), and Bayesian information criteria (BIC).

33. A system according to claim 22, wherein the means for analyzing the results for each statistical model type comprises means for ranking model types according to the level of performance of the model with respect to a goal of the model.

34. A system according to claim 33, wherein the means for identifying at least one statistical model comprises means for selecting the highest ranked model based on performance.

35. A system according to claim 22, wherein the system further comprises means for segmenting the database in accordance with at least one of business objectives and statistical objectives, and wherein a statistical model is built for each segment.

36. A system according to claim 22, wherein the system further comprises means for refreshing the statistical model after the model is generated.

37. A system according to claim 36, wherein the means for refreshing the statistical model refreshes the model in response to a refresh trigger, and wherein the refresh trigger comprising a predetermined event.

38. A system according to claim 37, wherein the predetermined event is at least one of an update to data in the database and a predetermined time period.

39. A computer readable medium that includes program instructions or program code for performing computer-implemented operations to provide a method for generating statistical models, the method comprising:

selecting a set of variables in accordance with a goal of the model;

applying the selected set of variables based on the data from a database to a plurality of statistical model types;

analyzing the results for each statistical model type; and

identifying at least one of the statistical model based on the analysis of the results.

40. A computer readable medium according to claim 39, wherein the program code further comprises program code for cleaning the data in the database to impute missing or extreme values.

41. A computer readable medium according to claim 39, wherein the program code further comprises program code for sorting and ordering the variables into groups.

42. A computer readable medium according to claim 39, wherein selecting a set of variables comprises eliminating statistically redundant data by performing at least one of factor analysis, principal component and variable clustering.

43. A computer readable medium according to claim 39, wherein selecting a set of variables comprises identifying relevant variables by performing at least a stepwise analysis of the variables.

44. A computer readable medium according to claim 39, wherein the data is provided as part of a data mart, and wherein the program code further comprises program code for dividing the data in the data mart into a development sample and a validation sample.

45. A computer readable medium according to claim 44, wherein applying the selected set of variables comprises applying data from the development sample to the plurality of statistical model types and applying data from the validation sample to the plurality of statistical model types.

46. A computer readable medium according to claim 39, wherein applying the selected set of variables comprises applying data from the database to a plurality of statistical models types, including at least one of regression models, parametric models, non-parametric models, tree type models, and neural network models.

47. A computer readable medium according to claim 39, wherein analyzing the results for each statistical model type comprises applying at least one benchmark measurement to determine the performance of each statistical model type with respect to the goal of the model.

48. A computer readable medium according to claim 47, wherein applying at least one benchmark measurement comprises performing an analysis of the results using at least one of an R2 computation, Akaike'"'"'s information criteria (AIC), and Bayesian information criteria (BIC).

49. A computer readable medium according to claim 39, wherein analyzing the results for each statistical model type comprises ranking model types according to the level of performance of the model with respect to the goal of the model.

50. A computer readable medium according to claim 49, wherein identifying at least one statistical model comprises selecting the highest ranked model based on performance.

51. A computer readable medium according to claim 39, wherein the program code further comprises program code for segmenting the database and generating a statistical model for each segment of the database.

52. A computer readable medium according to claim 51, wherein the database is segmented consistent with at least one of business objectives and statistical objectives.

53. A computer readable medium according to claim 39, wherein the program code further comprises program code for refreshing the statistical model after the model is generated.

54. A computer readable medium according to claim 53, wherein the program code for refreshing the statistical model comprises program code for refreshing the model in response to a refresh trigger, the refresh trigger comprising a predetermined event.

55. A computer readable medium according to claim 53, wherein the predetermined event is at least one of an update to data in the database and a predetermined time period.

56. A method for generating statistical models, comprising:

providing a database comprising data, the data representing a plurality of variables;

segmenting the data in the database into a plurality of segments; and

generating a statistical model for each segment in the database, wherein the statistical model for each segment is generated by;

selecting a set of variables from a segment in accordance with a goal for the statistical model;

applying the selected set of variables based on data from the segment in the database to a plurality of statistical model types;

analyzing the results for each statistical model type; and

identifying at least one statistical model for the segment based on the analysis of the results.

57. A method according to claim 56, wherein segmenting the data in the database comprises segmenting according to at least one of business objectives and statistical objectives.

58. A method according to claim 56, wherein applying the selected set of variables comprises applying data from the segment to a plurality of statistical models types, including at least one of regression models, parametric models, non-parametric models, tree type models, and neural network models.

59. A method according to claim 56, wherein analyzing the results for each statistical model type comprises applying at least one benchmark measurement to determine the performance of each statistical model type with respect to the goal of the model.

60. A method according to claim 59, wherein applying at least one benchmark measurement comprises performing an analysis of the results using at least one of an R2 computation, Akaike'"'"'s information criteria (AIC), and Bayesian information criteria (BIC).

61. A method for generating and maintaining statistical models, comprising:

providing a data mart comprising data, the data representing a plurality of variables;

generating a plurality of statistical models based on the data in the data mart, each of the statistical models being consistent with an identified goal for the model;

monitoring, after the statistical models are generated, for the occurrence of a refresh trigger;

identifying, in response to a refresh trigger, which of the statistical models need to be refreshed; and

refreshing the statistical models identified to be refreshed.

62. A method according to claim 61, wherein the method further comprises periodically updating the data in the data mart with new data, and wherein refreshing comprises refreshing the statistical models identified to be refreshed with the updated data in the data mart.

63. A method according to claim 61, wherein the refresh trigger comprises the occurrence of a predetermined event.

64. A method according to claim 63, wherein the predetermined event is at least one of an update to data in the data mart and a passing of a predetermined time period.

65. A method according to claim 61, wherein generating the statistical model comprises:

selecting a set of variables from the data mart in accordance with the goal for the model;

applying the selected set of variables based on data from the data mart;

analyzing the results for each statistical model type; and

identifying at least one statistical model based on the analysis of the results.

66. A method of analyzing results of statistical models comprising:

applying a coarse analysis of the results comprising;

applying one or more benchmark measurements to the results of the statistical models,

comparing the results of the statistical models with a preset goal of the statistical models,

identifying the best performing statistical models; and

applying a fine analysis of the results comprising;

checking to ensure that the variables used by the best performing statistical models are accurate.

67. The method of claim 66, wherein applying a fine analysis of the results further comprises:

comparing the best performing statistical models with a predetermined objective.

68. A computer readable medium that includes program instructions or program code for performing computer-implemented operations to provide a method for analyzing results of statistical models, the method comprising:

applying a coarse analysis of the results comprising;

applying one or more benchmark measurements to the results of the statistical models;

comparing the results of the statistical models with a preset goal of the statistical models; and

identifying the best performing statistical models;

applying a fine analysis of the results comprising;

checking to ensure that the variables used by the best performing statistical models are accurate.

69. A computer readable medium according to claim 68, wherein applying a fine analysis of the results further comprises:

comparing the best performing statistical models with a preset goal or objective.