Methods and systems for variable group selection and temporal causal modeling

• US 8,255,346 B2
• Filed: 11/11/2009
• Issued: 08/28/2012
• Est. Priority Date: 11/11/2009
• Status: Active Grant

• Current Assignee: Globalfoundries US Incorporated (GlobalFoundries, Inc.)
• Original Assignee: International Business Machines Corporation
• Inventors: Abe, Naoki, Liu, Yan, Lozano, Aurelie C., Rosset, Saharon, Swirszcz, Grzegorz
• Primary Examiner(s): Chen, Alan
• Application Number:
  US12/616,534
• Publication Number:
  US 20110112998A1
• Time in Patent Office: 1,021 Days
• Field of Search: None
• US Class Current: 706/15
• CPC Class Codes: G06N 5/01 Dynamic search techniques; Heuristics; Dynamic trees; Branch-and-bound
• CPC Class Codes: G06N 7/01 Probabilistic graphical models, e.g. probabilistic networks

Abstract

A “variable group selection” system and method in which constructs are based upon a training data set, a regression modeling module that takes into account information on groups of related predictor variables given as input and outputs a regression model with selected variable groups. Optionally, the method can be employed as a component in methods of temporal causal modeling, which are applied on a time series training data set, and output a model of causal relationships between the multiple times series in the data.

Claims

1. A computer-implemented method of variable group selection comprising:

providing at a computer device, a training data set for which a predictive model is to be generated;

iteratively selecting from among a plurality of variables of said data set, one or more variable groups minimizing a residual error in each iteration; and

in each iteration, selecting a variable group with respect to the minimization of residual error of an updated predictive model when the selected variable group is included in the said predictive model in addition to those variable groups that have been selected in earlier iterations; and

,

performing a regression by an arbitrary component regression method using the selected variable groups selected into the current predictive model up to the current iteration in each iteration to obtain said updated predictive model,

wherein a program using a processor unit executes one or more said iteratively selecting and performing step.

2. The method as claimed in claim 1, further comprising, at each iteration, estimating coefficients to be used in the current estimated predictive model, by computing the coefficients of the variables in the variable group selected in the current iteration only, so as to minimize the residual error computed using the coefficients estimated in earlier iterations for the variable groups selected in those earlier iterations.

3. The method as claimed in claim 2, further comprising, upon selection of the variable groups in each iteration, re-estimating the coefficients of all of the variables in the variable groups selected into the current predictive model up to the current iteration are re-estimated so as to minimize the residual error.

4. The method as claimed in claim 2, wherein said arbitrary component regression method includes implementing a least squares method.

5. The method as claimed in claim 3, wherein said arbitrary component regression method includes implementing a least squares method.

6. A method implemented by a computer for temporal causal modeling comprising:

a) providing, at a computer system, a data set comprising a number of time series data; and

b) creating, from said data set, data for variables corresponding to a number of relative past time steps for each time series;

c) grouping said variables according to a time series they belong to; and

d) applying a variable group selection method by iteratively choosing a target time series and selecting the time series that are considered to be causing the said target time series, and, at each iteration;

selecting from among the variables of said data set one of a plurality of variable groups minimizing a residual error in each iteration; and

in each iteration, selecting a variable group with respect to the minimization of residual error of an updated predictive model when the selected variable group is included in said predictive model in addition to those variable groups that have been selected in earlier iterations; and

,

performing regression by an arbitrary component regression method using the selected variable groups selected into the current predictive model up to the current iteration in each iteration to obtain said updated predictive model,

wherein a program using a processor unit executes one or more said creating, grouping, iteratively choosing and selecting and performing step.

7. The method as claimed in claim 6, wherein said creating data variable corresponding to a number of relative past time steps includes executing a method of lagging.

8. The method as claimed in claim 7, further comprising generating a predictive model wherein said the target time series is modeled in terms of the past values of all the time series.

9. The method as claimed in claim 8, further comprising generating a predictive model wherein said the target time series is modeled in terms of the past values of all the time series including the target time series itself.

10. The method as claimed in claim 6, further comprising, at each iteration, estimating coefficients to be used in the current estimated predictive model, by computing the coefficients of the variables in the variable group selected in the current iteration only, so as to minimize the residual error computed using the coefficients estimated in earlier iterations for the variable groups selected in those earlier iterations.

11. The method as claimed in claim 10, further comprising, upon selection of the variable groups in each iteration, re-estimating the coefficients of all of the variables in the variable groups selected into the current predictive model;

and, up to the current iteration, are re-estimated so as to minimize the residual error.

12. The method as claimed in claim 6, wherein said arbitrary component regression method includes implementing a least squares method.

13. A method implemented by a computer for spatial temporal causal modeling comprising:

a) providing, at a computer system, a data set comprising a number of spatial temporal data;

b) creating, from said data set, data for variables corresponding to a number of relative past time steps and a number of relative spatial neighbors for each spatial temporal series; and

c) grouping said variables according to the spatial temporal series they belong to; and

d) applying a variable group selection method by iteratively choosing a target spatial temporal measurement and selecting the spatial temporal measurements that are causing the said target spatial temporal measurement, and, at each iteration;

selecting from among the variables of said data set one of a plurality of variable groups minimizing a residual error in each iteration; and

in each iteration, selecting a variable group with respect to the minimization of residual error of an updated predictive model when the selected variable group is included in the said predictive model in addition to those variable groups that have been selected in earlier iterations; and

,

performing regression by an arbitrary component regression method using the selected variable groups selected into the current predictive model up to the current iteration in each iteration to obtain said updated predictive model,

wherein a program using a processor unit executes one or more said creating, grouping, iteratively choosing, selecting and performing steps.

14. The method as claimed in claim 13, wherein said creating data variable corresponding to a number of relative past time steps includes executing a method of spatial and temporal lagging.

15. The method as claimed in claim 13, further comprising generating a predictive model wherein said the target time series is modeled in terms of the past values of all the time series.

16. The method as claimed in claim 13, further comprising generating a predictive model wherein said the target time series is modeled in terms of the past values of all the time series including the target time series itself.

17. The method as claimed in claim 13, further comprising, at each iteration, estimating coefficients to be used in the current estimated predictive model, by computing the coefficients of the variables in the variable group selected in the current iteration only, so as to minimize the residual error computed using the coefficients estimated in earlier iterations for the variable groups selected in those earlier iterations.

18. The method as claimed in claim 17, further comprising, upon selection of the variable groups in each iteration, re-estimating the coefficients of all of the variables in the variable groups selected into the current predictive model;

and, up to the current iteration, are re-estimated so as to minimize the residual error.

19. The method as claimed in 13, further comprising:

at each iteration, applying a data transformation comprising;

a) expanding data for the target variable by concatenating a vector of zeros whose size equals the number of lagged explanatory variables; and

b) multiplying the data for the explanatory variables by a first data matrix for controlling a spatial decay behavior of regression coefficients and expanded by a constant times an identity matrix; and

c) applying a variable group selection method to a resulting transformed data; and

d) scaling estimated regression coefficients by multiplying by a second data matrix that bears a certain relationship to the first data matrix.

20. A computer system for variable group selection, the system comprising:

a memory;

a processor in communications with the computer memory, wherein the computer system is capable of performing a method comprising;

providing a training data set for which a predictive model is to be generated;

iteratively selecting from among a plurality of variables of said data set, one or more variable groups minimizing a residual error in each iteration; and

in each iteration, selecting a variable group with respect to the minimization of residual error of an updated predictive model when the selected variable group is included in the said predictive model in addition to those variable groups that have been selected in earlier iterations; and

,

performing a regression by an arbitrary component regression method using the selected variable groups selected into the current predictive model up to the current iteration in each iteration to obtain said updated predictive model.

21. The system as claimed in claim 20, further comprising, at each iteration, estimating coefficients to be used in the current estimated predictive model, by computing the coefficients of the variables in the variable group selected in the current iteration only, so as to minimize the residual error computed using the coefficients estimated in earlier iterations for the variable groups selected in those earlier iterations.

22. The system as claimed in claim 21, further comprising, upon selection of the variable groups in each iteration, re-estimating the coefficients of all of the variables in the variable groups selected into the current predictive model up to the current iteration are re-estimated so as to minimize the residual error.

23. The system as claimed in claim 21, wherein said arbitrary component regression method includes implementing a least squares method.