Process and system for developing a predictive model

• US 7,043,461 B2
• Filed: 05/23/2001
• Issued: 05/09/2006
• Est. Priority Date: 01/19/2001
• Status: Expired due to Term

• Current Assignee: Genalytics, Inc. (Semcasting Incorporated)
• Original Assignee: Genalytics, Inc. (Semcasting Incorporated)
• Inventors: Dillon, David, Kehder, Matthias
• Primary Examiner(s): Hirl, Joseph P.
• Application Number:
  US09/863,175
• Publication Number:
  US 20030004903A1
• Time in Patent Office: 1,812 Days
• Field of Search: 706/12, 706/13, 706/14
• US Class Current: 706/13
• CPC Class Codes: G06N 3/126 Evolutionary algorithms, e.g. genetic algorithms or genetic programming

Abstract

The present invention relates to a computer implemented process for developing a model which predicts the value of a single dependent variable based on the value of at least one independent variable. The process comprises the steps of creating a dataset containing a plurality of observations each containing a value for the dependent variable and values for the at least one independent variable, creating from the dataset a plurality of original chromosomes each comprising a possible predictive model, developing a quantitative fitness measure for each chromosome, and creating a new generation of chromosomes by selecting a number of the original chromosomes based upon the fitness measures, crossing the selected original chromosomes by at least one of a cloning and a pure (standard) crossover technique, and mutating the crossed chromosomes. A system for carrying out the process of the present invention is also described.

Claims

1. A computer implemented process for developing a model for business applications which predicts the value of a single dependent variable based on the value of at least one independent variable comprising the steps of:

(a) providing a dataset containing a plurality of observations each containing a value for said dependent variable and values for said at least one independent variable;

(b) creating from said dataset of an initial generation of chromosomes each comprising a predictive model, said chromosome creating step comprising forming a chromosome having a plurality of observed variable segments each formed by a plurality of genes, a plurality of interaction segments each formed by a plurality of genes, and an intercept gene positioned after said plurality of interaction segments;

wherein each interaction segment consists of the following genes—

an include/exclude gene, a coefficient gene, a first variable gene, an operator gene and a second variable gene;

(c) determining a quantitative fitness measure for each chromosome in said initial generation; and

(d) creating a next generation of chromosomes by selecting a number of chromosomes from said initial generation, crossing said selected chromosomes by at least one of a cloning and a crossover technique, and mutating said chromosomes.

2. A process according to claim 1, further comprising (e) determining new fitness measures for said chromosomes in said next generation, and (f) repeating said selecting, crossing and mutating steps to create a successive generation of chromosomes.

3. A process according to claim 2, further comprising continuously repeating steps (e) and (f) for said successive generation of chromosomes to create another successive generation of chromosomes.

4. A process according to claim 1, wherein said initial generation chromosome creating step is performed using a random method.

5. A process according to claim 1, wherein said initial generation chromosome creating step is performed using a distributed method.

6. A process according to claim 1, further comprising limiting the number of interaction segments in each said chromosome to a fixed parameter or applying a predetermined rule to determine the number of interaction gene segments.

7. A process according to claim 1, wherein said chromosome forming step comprises forming at least one observed continuous variable gene segment so as to contain an include/exclude gene, a coefficient gene, a minimum outlier gene, a maximum outlier gene, a transformation gene, and a ranking gene.

8. A process according to claim 7, further comprising forming a plurality of observed continuous variable gene segments and using said include/exclude gene to indicate whether a variable for a particular one of said observed continuous variable segments is included in an embedded model.

9. A process according to claim 8, further comprising using said coefficient gene to identify a coefficient value for an observed continuous variable described by the particular one of said observed variable segments.

10. A process according to claim 9, further comprising forming said coefficient gene from a plurality of binary bits and designating a number of said binary bits as a super bit where all of said bits flip under certain conditions.

11. A process according to claim 9, further comprising using said outlier genes in each said observed continuous variable segment to trim a distribution range for the observed value in said observed continuous variable segment.

12. A process according to claim 11, further comprising representing said outlier genes in a double stranded format.

13. A process according to claim 11, further comprising using said transformation gene to eliminate skewness and kurtosis from a distribution range for said observed variable in said observed continuous variable segment and to transform said distribution range as close as possible to normality.

14. A process according to claim 1, wherein said chromosome forming step comprises forming at least one observed categorical variable segment consisting of an include/exclude gene, a contrast gene, and at least one coefficient gene.

15. A process according to claim 14, further comprising forming a plurality of observed categorical variable segments and using said include/exclude gene to indicate whether a variable for a particular one of said observed categorical variable segments is included in an embedded model.

16. A process according to claim 15, further comprising using said coefficient gene to identify a coefficient value for the observed categorical variable described by the particular one of said observed variable segments.

17. A process according to claim 16, further comprising forming said coefficient gene from a plurality of binary bits and designating a number of said binary bits as a super bit where all of said bits flip under certain conditions.

18. A process according to claim 14, further comprising using said contrast gene to indicate a maximum number of categories allowed for a requested model.

19. A process according to claim 1, further comprising using said include/exclude gene to indicate if a particular interaction variable described by a particular interaction segment is included in the model.

20. A process according to claim 19, further comprising using the coefficient gene to indicate a coefficient value for the interaction variable described by a particular interaction segment.

21. A process according to claim 20, further comprising using said first variable gene to identify a first observed variable used to derive a value of the interaction variable.

22. A process according to claim 21, further comprising using said operator gene to select an operator used to derive an interaction value from two observed variables.

23. A process according to claim 22, wherein said operator is selected from the group consisting of addition, multiplication, subtraction, difference squared, division, and absolute difference.

24. A process according to claim 22, further comprising using said second variable gene to identify a second of two observed variables used to derive the value of the interaction variable.

25. A process according to claim 24, further comprising randomly selecting said first and second variable genes in each of said interaction segments.

26. A computer implemented process for developing a model for business applications which predicts the value of a single dependent variable based on the value of at least one independent variable comprising the steps of:

(a) providing a dataset containing a plurality of observations each containing a value for said dependent variable and values for said at least one independent variable;

(b) creating from said dataset of an initial generation of chromosomes each comprising a predictive model, said chromosome creating step comprising forming a chromosome having a plurality of observed variable segments each formed by a plurality of genes, a plurality of interaction segments each formed by a plurality of genes, and an interceptor gene positioned after said plurality of interaction segments;

wherein each interaction segment consists of the following genes—

an include/exclude gene, a coefficient gene, a first variable gene, an operator gene and a second variable gene;

(c) determining a quantitative fitness measure for each chromosome in said initial generation;

(d) creating a next generation of chromosomes by selecting a number of chromosomes from said initial generation, crossing said selected chromosomes by at least one of a cloning and a crossover technique, and mutating said chromosomes; and

said fitness measure determining step comprising providing a scheduler module and a plurality of fitness evaluator modules and using said scheduler module to direct one of said chromosomes in said generation to be measure to an available one of said evaluator modules.

27. A process according to claim 26, wherein said fitness measure determining step comprises selecting a fitness metric and determining a fitness rating for said one chromosome using said fitness metric.

28. A process according to claim 26, further comprising providing at least one modeler module, accumulating a fitness measure for each said chromosome in said scheduler module, and transmitting said accumulated fitness measures from said scheduler module to said at least one modeler module.

29. A process according to claim 28, further comprising performing said new generation chromosome creating step using said at least one modeler module.

30. A process according to claim 29, further comprising weighting each said chromosome in said generation and said selecting step comprising randomly selecting said weighted chromosomes.

31. A process according to claim 30, wherein said procreated step comprises selecting a crossover rate and a number of crossover points and creating two offspring chromosomes from two selected parent chromosomes using said selected crossover rate and said crossover points.

32. A system for creating a predictive model for business applications comprising:

means for creating an initial generation of chromosomes from a dataset with each chromosome in said initial generation comprising a predictive model, said chromosome creating step comprising forming a chromosome having a plurality of observed variable segments each formed by a plurality of genes, a plurality of interaction segments each formed by a plurality of genes, and an interceptor gene positioned after said plurality of interaction segments;

wherein each interaction segment consists of the following genes—

an include/exclude gene, a coefficient gene, a first variable gene, an operator gene and a second variable gene;

means for determining a quantitative fitness measure for each chromosome in said initial generation;

means for selecting a number of said initial generation chromosomes, for crossing the selected initial generation chromosomes by at least one of a cloning and a pure (standard) crossover technique, and for mutating the crossed chromosomes into a next generation of chromosomes; and

a computer and said initial generation chromosome creating means comprising at least one modeler module within said computer.

33. A system according to claim 32, further comprising a scheduler module in said computer and at least one fitness evaluator in said computer forming said means for determining a quantitative fitness measure.

34. A system according to claim 33, further comprising a plurality of fitness evaluators in said computers and said scheduler module determining which of said plurality of fitness evaluators shall determine the fitness measure for a particular one of said chromosomes.

35. A system according to claim 32, wherein said at least one modeler module also forms said means for selecting a number of initial generation chromosomes, for crossing the selected initial generation chromosomes, and for mutating the crossed chromosomes.

36. A system according to claim 32, further comprising a user interface module.

37. A system according to claim 32, further comprising a database within said computer for storing datasets and best predictive models and said at least one modeler module communicating with said database.

38. A chromosome for predicting a model for business applications comprising a plurality of observed variable segments, a plurality of interaction segments and an intercept gene and further comprising said intercept gene being positioned on said chromosome after said plurality of interaction segments;

wherein each interaction segment consists of the following genes—

an include/exclude gene, a coefficient gene, a first variable gene, an operator gene and a second variable gene.

39. A chromosome according to claim 38, wherein said plurality of interaction segments is limited in number to a preset value or by a predetermined rule.

40. A chromosome according to claim 38, wherein said observed variable segments contain at least one observed continuous variable segment comprising an include/exclude gene, a coefficient gene, a minimum outlier gene, a maximum outlier gene, a transforming gene, and a ranking gene.

41. A chromosome according to claim 40, wherein each of said outlier genes is represented in a double stranded format.

42. A chromosome according to claim 38, wherein said observed variable segments contain at least one observed categorical variable segment comprising an include/exclude gene, a contrast gene, and at least one coefficient gene.

43. A chromosome according to claim 38, wherein each of said first and second variables is randomly selected from a number of variables in a dataset.