INTERACTION DETECTION FOR GENERALIZED LINEAR MODELS

• US 20150006605A1
• Filed: 09/15/2014
• Published: 01/01/2015
• Est. Priority Date: 03/11/2013
• Status: Abandoned Application

• Current Assignee: International Business Machines Corporation
• Original Assignee: International Business Machines Corporation
• Inventors: Chu, Yea J., Han, Sier, Shyr, Jing-Yun
• Application Number:
  US14/486,659
• Publication Number:
  US 20150006605A1
• Time in Patent Office: Days
• US Class Current: 708/446
• CPC Class Codes: G06F 17/18 for evaluating statistical data , e.g. average values, frequency distributions, probability functions, regression analysis forecasting specially adapted for a specific administrative, business or logistic context G06Q10/04
• CPC Class Codes: G06Q 10/06 Resources, workflows, human or project management; Enterprise or organisation planning; Enterprise or organisation modelling
• CPC Class Codes: G06Q 10/067 Enterprise or organisation modelling

Abstract

Provided are techniques for interaction detection for generalized linear models. Basic statistics are calculated for a pair of categorical predictor variables and a target variable from a dataset during a single pass over the dataset. It is determined whether there is a significant interaction effect for the pair of categorical predictor variables on the target variable by: calculating a log-likelihood value for a full generalized linear model without estimating model parameters; calculating the model parameters for a reduced generalized linear model with a recursive marginal mean accumulation technique using the basic statistics; calculating a log-likelihood value for the reduced generalized linear model; calculating a likelihood ratio test statistic using the log-likelihood value for the full generalized linear model and the log-likelihood value for the reduced generalized linear model; calculating a p-value of the likelihood ratio test statistic; and comparing the p-value to a significance level.

Claims

1. A method, comprising:

calculating, using a computer, basic statistics for a pair of categorical predictor variables and a target variable from a dataset during a single pass over the dataset; and

determining, using the computer, whether there is a significant interaction effect for the pair of categorical predictor variables on the target variable by;

calculating, using the computer, a log-likelihood value for a full generalized linear model without estimating model parameters;

calculating, using the computer, the model parameters for a reduced generalized linear model with a recursive marginal mean accumulation technique using the basic statistics;

calculating, using the computer, a log-likelihood value for the reduced generalized linear model;

calculating, using the computer, a likelihood ratio test statistic using the log-likelihood value for the full generalized linear model and the log-likelihood value for the reduced generalized linear model;

calculating, using the computer, a p-value of the likelihood ratio test statistic; and

comparing, using the computer, the p-value to a significance level.

2. The method of claim 1, wherein the full generalized linear model is of the form g(μ

)=Xiβ

i+Xjβ

j+(Xi×

Xj)β

ij, wherein g(μ

) is a link function of μ and

μ

is an expected value vector of the target variable Y, wherein Xi and Xj are the categorical predictor variables, and wherein β

i, β

j, and β

ij are the model parameters.

3. The method of claim 1, wherein the reduced generalized linear model is of the form g(μ

)=Xiβ

i+Xjβ

j, wherein g(μ

) is a link function of μ and

μ

is an expected value vector of the target variable Y, wherein Xi and Xj are the categorical predictor variables, and wherein β

i and β

j are the model parameters.

4. The method of claim 1, wherein the recursive marginal mean accumulation technique calculates search directions for the model parameters calculation by an iterative process based on accumulating weighted marginal means.

5. The method of claim 1, further comprising:

performing, using the computer, m-way interaction detection among m categorical predicator variables, where m>

2.

6. The method of claim 1, further comprising:

performing, using the computer, m-way interaction detection among multiple possible combinations of m categorical predictor variables, where m>

1, wherein the dataset contains predictor variables, and the basic statistics for each of the possible combinations are calculated during a single pass over the dataset.

7. The method of claim 1, wherein a Software as a Service (SaaS) is provided to perform the method.

8-21. -21. (canceled)