Methods and Systems for Determining the Importance of Individual Variables in Statistical Models

• US 20140200930A1
• Filed: 03/17/2014
• Published: 07/17/2014
• Est. Priority Date: 11/28/2001
• Status: Abandoned Application

• Current Assignee: Deloitte Development LLC
• Original Assignee: Deloitte Development LLC
• Inventors: Zizzamia, Frank M., Wu, Cheng-Sheng Peter, Greene, Michael F., Guszcza, James C., Lucker, John R., Yan, Jun, Bosch, Jonathan Vanden
• Application Number:
  US14/217,231
• Publication Number:
  US 20140200930A1
• Time in Patent Office: Days
• US Class Current: 705/4
• CPC Class Codes: G06F 17/18 for evaluating statistical data , e.g. average values, frequency distributions, probability functions, regression analysis forecasting specially adapted for a specific administrative, business or logistic context G06Q10/04
• CPC Class Codes: G06Q 10/0635 Risk analysis of enterprise or organisation activities
• CPC Class Codes: G06Q 40/08 Insurance
• CPC Class Codes: G16H 50/50 for simulation or modelling of medical disorders

Abstract

Methods and systems for determining the importance of each of the variables, or combinations of variables, that contribute to the overall score generated by a predictive statistical model are presented. In a specialized case, for each variable in the model, an importance is calculated based on the calculated slope and deviance of the predictive variable. In a more general case, for each variable in the model, an importance is calculated based on setting that variable to have the average value for the data set, and then calculating the change in score. The totality of variables (or combinations thereof) is then ranked by the Δscore, or a magnitude of it, such as |Δscore|.

Claims

1. A method of calculating the contribution of an individual term to a multivariate expression which includes that term, comprising:

obtaining an original result of the multivariate expression;

modifying an individual term of the multivariate expression to an average value for a defined population, and keeping all other terms of the multivariate expression unchanged;

using a data processor, calculating a modified result of the multivariate expression using the modified individual term;

using a data processor, calculating the difference between the original and the modified result, Δ

result; and

using a data processor, outputting Δ

result to a user as the contribution of the individual term to the result of the multivariate expression.

2. The method of claim 1, further comprising:

repeating the method for at least one additional individual term of the multivariate expression;

ranking the contribution of each of the individual terms by one of;

(i) Δ

result for each individual term;

(ii) the absolute value of Δ

result, |Δ

result|, for each individual term;

(iii) Δ

result taken to a power, or (Δ

result)ⁿ; and

(iv) the absolute value of (Δ

result)ⁿ, or |(Δ

result)ⁿ|; and

outputting the ranked contributions of the individual term and the additional individual terms to a user, indicating both the contribution, and relative rank, of each individual term.

3. The method of claim 2, wherein the at least one additional term includes all additional terms of the multivariate expression.

4. The method of claim 1, wherein the multivariate expression is nonlinear.

5. The method of claim 1, wherein the multivariate expression includes interaction terms.

6. The method of claim 2, wherein the multivariate expression is nonlinear.

7. The method of claim 2, wherein the multivariate expression includes interaction terms.

8. The method of claim 5, wherein the interaction terms include variables taken to powers, variables as arguments of functions, combinations of multiple variables, or combinations wherein one or more variables are taken to powers or arguments of functions.

9. The method of claim 7, wherein the interaction terms include variables taken to powers, variables as arguments of functions, combinations of multiple variables, or combinations wherein one or more variables are taken to powers or arguments of functions.

10. The method of claim 1, wherein if the value of an individual term, variable or element of the multivariate expression is not available, then the value of the mean for that term, variable or element is interpolated when calculating a result or modified result.

11. The method of claim 10, wherein an index of at least one of (i) how many and (ii) what proportion of terms, variables or elements of the multivariate expression are based on such interpolation is also presented to the user.

12. The method of claim 11, wherein for a multivariate expression:

DEN=SUM(abs(b1*μ

1)+abs(b2*μ

2)+ . . . +abs(bN*μ

uN),

Said index is NUM/DEN, where NUM equals the sum of these terms for which the variable is not missing.

13. A system for contribution of an individual term to a multivariate expression which includes that term, comprising:

a database for storing values for various input variables;

a display;

and at least one data processor configured to;

receive a multivariate scoring formula, said scoring formula comprising a sum of a plurality of predictive input variables each having a weighting co-efficient, values for at least some of said variables being stored in the database;

calculate a score using said scoring formula and a set of input variable values;

calculate a partial derivative of the scoring formula with respect to each of the input variables in said set;

calculate a deviance value for each of the input variables in said set, said deviance for a variable xi=(xi−

μ

i), where pi is the mean for predictive input variable xi;

calculate a contribution of one or more of the input variables in said set to the score by multiplying the partial derivative and deviance values for that variable;

create a rank for each of said one or more input variables and display the value of the variable, the score and the rank of the variable to a user.

14. The method of claim 13, further comprising repeating the method for all variables whose values are stored in the database.

15. The method of claim 13, wherein if the value of a variable of the multivariate expression is not available, then the value of the mean for that term is interpolated when calculating a score.

16. The method of claim 15, wherein an index of at least one of (i) how many and (ii) what proportion of terms of the multivariate expression are based on such interpolation is also presented to the user.

17. The method of claim 16, wherein for a multivariate expression:

DEN=SUM(abs(b1*μ

1)+abs(b2*μ

2)+ . . . +abs(bN*μ

uN),

said index is NUM/DEN, where NUM equals the sum of these terms for which the variable is not missing.

18. A non-transitory computer readable medium containing instructions that, when executed by at least one processor of a computing device, cause the computing device to:

obtain an original result of the multivariate expression;

modify an individual term of the multivariate expression to an average value for a defined population, and keeping all other terms of the multivariate expression unchanged;

calculate a modified result of the multivariate expression using the modified individual term;

calculate the difference between the original and the modified result, Δ

result; and

output Δ

result to a user as the contribution of the individual term to the result of the multivariate expression.

19. The non-transitory computer readable medium of claim 18, wherein the instructions, when executed, further cause the computing device to:

repeat the process of claim 18 for at least one additional individual term of the multivariate expression;

rank the contribution of each of the individual terms by one of;

(i) Δ

result for each individual term;

(ii) the absolute value of Δ

result, |Δ

result|, for each individual term;

(iii) Δ

result taken to a power, or (Δ

result)ⁿ; and

(iv) the absolute value of (Δ

result)ⁿ, or |(Δ

result)ⁿ|; and

output the ranked contributions of the individual term and the additional individual terms to a user, indicating both the contribution, and relative rank, of each individual term.

20. The non-transitory computer readable medium of claim 19, wherein the at least one additional term includes all additional terms of the multivariate expression.

21. The non-transitory computer readable medium of claim 18, wherein the multivariate expression includes interaction terms.

22. A method for dealing with potential collinearity of variables in a multivariate expression of N variables, comprising:

partitioning the set of variables into a set of mutually exclusive and completely exhaustive M variable clusters;

mathematically creating composite indices to summarize all of the variables within a cluster into a single composite measure;

performing a regression analysis to approximate an output of the multivariate expression as a combination of the composite indices;

rank ordering by absolute value of the composite indices and their co-efficients; and

outputting the combination of composite indices and the ranked order to a user.

23. The method of claim 23, wherein the composite indices are substantially independent.

24. The method of claim 23, wherein the regression analysis is a principal components analysis.

25. The method of claim 24, wherein the output of the multivariate expression is approximated as a linear combination of M variables, each of which is the first PC of a PCA performed on the variables within a cluster.

26. The method of claim 25, where the modified multivariate expression is expressed as:

yhat=b1PC1+b2PC2+ . . . +bkPCk

where;

yhat denotes the output of the modified multivariate expression, {PC1, . . . PCk} denote the composite PCs created for each of the M variable clusters, and {b1, . . . ,bk} denote the weights determined from the regression analysis.

27. The method of claim 26, wherein a rank ordering by the absolute value of the corresponding quantities {b1PC1, . . . , b1PCk} is performed and output to the user.