Efficient gradient computation for conditional Gaussian graphical models

• US 20080010043A1
• Filed: 12/06/2004
• Published: 01/10/2008
• Est. Priority Date: 12/06/2004
• Status: Active Grant

• Current Assignee: Microsoft Technology Licensing LLC (Microsoft Corporation)
• Original Assignee: Microsoft Corporation
• Inventors: Meek, Christopher, Thiesson, Bo
• Granted Patent: US 7,596,475 B2
• Time in Patent Office: Days
• US Class Current: 703/002
• CPC Class Codes: G06F 18/29 Graphical models, e.g. Bayesian networks

Abstract

The subject invention leverages standard probabilistic inference techniques to determine a log-likelihood for a conditional Gaussian graphical model of a data set with at least one continuous variable and with data not observed for at least one of the variables. This provides an efficient means to compute gradients for CG models with continuous variables and incomplete data observations. The subject invention allows gradient-based optimization processes to employ gradients to iteratively adapt parameters of models in order to improve incomplete data log-likelihoods and identify maximum likelihood estimates (MLE) and/or local maxima of the incomplete data log-likelihoods. Conditional Gaussian local gradients along with conditional multinomial local gradients determined by the subject invention can be utilized to facilitate in providing parameter gradients for full conditional Gaussian models.

Claims

1. A system that facilitates statistical modeling, comprising

a gradient determination component that utilizes a data set for a set of variables and probabilistic inference to determine parameter gradients for a log-likelihood of a conditional Gaussian (CG) graphical model over those variables with at least one continuous variable and with incomplete observation data for at least one of the variables.

2. The system of claim 1 employed by a gradient-based optimization process that iteratively adapts parameters of a model to improve incomplete data log-likelihood.

3. The system of claim 2, the gradient-based optimization process comprising a conjugate gradient process.

4. The system of claim 1, the gradient determination component utilizes a Recursive Exponential Mixed Model (REMM) to facilitate in determining the parameter gradients.

5. The system of claim 1, the gradient determination component determines parameter gradients of a graphical model with at least two equal parameters.

6. The system of claim 1, the gradient determination component determines parameter gradients of a graphical model with at least one parameter fixed to a specific value.

7. The system of claim 1, the gradient determination component determines parameter gradients of a graphical model with at least one parameter fixed to a specific value and with at least two equal parameters.

8. The system of claim 7, the graphical model comprising a stochastic autoregressive moving average model.

9. The system of claim 1, the gradient determination component employs an exponential model representation that automatically enforces that p^s≧

0 and Σ

_sp^s=1 to facilitate in optimizing at least one gradient parameter.

10. The system of claim 1, the incomplete observation data comprising data that is incomplete in a non-informative manner.

11. The system of claim 1, the parameter gradients comprising conditional multinomial local gradients and/or at least one conditional Gaussian local gradients.

12. The system of claim 1, the probabilistic inference comprising a propagation scheme for Bayesian networks with conditional Gaussian distributions.

30. A computer readable medium having stored thereon computer executable components of the system of claim 1.

32. A device employing the system of claim 1 comprising at least one selected from the group consisting of a computer, a server, and a handheld electronic device.

13. A method for facilitating statistical modeling, comprising:

receiving at least one data set with incomplete observation data for a set of variables with at least one continuous variable; and

utilizing the data set and probabilistic inference to determine parameter gradients for a log-likelihood of a conditional Gaussian (CG) graphical model over those variables.

14. The method of claim 13 further comprising:

employing the parameter gradients in a gradient-based optimization process that iteratively adapts parameters of a model to improve incomplete data likelihood.

15. The method of claim 14, the gradient-based optimization process comprising a conjugate gradient process.

16. The method of claim 13 further comprising:

utilizing a Recursive Exponential Mixed Model (REMM) to facilitate in determining the parameter gradients.

17. The method of claim 13 further comprising:

determining parameter gradients of a graphical model with at least two equal parameters.

18. The method of claim 13 further comprising:

determining parameter gradients of a graphical model with at least one parameter fixed to a specific value.

19. The method of claim 13 further comprising:

determining parameter gradients of a graphical model with at least one parameter fixed to a specific value and with at least two equal parameters.

20. The method of claim 19, the graphical model comprising a stochastic autoregressive moving average model.

21. The method of claim 13 further comprising:

optimizing at least one gradient parameter via utilization of an exponential model representation that automatically enforces that p^s≧

0 and

$\underset{s}{\sum <br><br>}<br><br>\text{\hspace{1em}<br><br>}<br><br>p^s=1$

to facilitate in optimizing at least one gradient parameter.

22. The method of claim 13, the incomplete observation data comprising data that is incomplete in a non-informative manner.

23. The method of claim 13, the parameter gradients comprising conditional multinomial local gradients and/or at least one conditional Gaussian local gradients.

24. The method of claim 13, the probabilistic inference comprising a propagation scheme for Bayesian networks with conditional Gaussian distributions.

25. The method of claim 13 further comprising:

representing the conditional Gaussian graphical model as a recursive exponential mixed model (REMM);

determining a conditional Gaussian regression in the REMM representation model;

utilizing a chain rule to transform a resulting parameter gradient expression into a parameter representation of c, f, and a, representing an intercept, coefficients, and variance, respectively for a regression on continuous parents; and

employing probabilistic inference to determine quantities in the parameter gradient expression.

26. The method of claim 13 further comprising:

assigning, for all families (X_v,X_pa(v)) where X_v is a continuous variable, each configuration of discrete parent variables x^d_pa(v) a sum vector and a sum of squares matrix for expected posterior statistics associated with the conditional distributions for the continuous variable and its continuous parents (X_v,X^c_pa(v)) given configurations of discrete parents;

For each data case in a data set;

utilizing a propagation scheme for Bayesian networks with conditional Gaussian distributions to determine a mean vector and a covariance matrix associated with a posterior marginal distribution for all families (X_v,X_pa(v)), where X_v is a continuous variable;

employing posterior conditional means and variances to determine expected values (x_v*,x^c_pa(v)*) and ((x_vx_v)*, (x_v x^c_pa(v))*, (x_v x^c_pa(v)′

)*, (x^c_pa(v)′

x^c_pa(v))*) for continuous variables in each of these families given each configuration of discrete variables in the respective families;

adding the expected values to the sum and the sum of squares expected posterior statistics; and

determining the parameter gradients utilizing a formula for a sample of incomplete observations.

27. The method of claim 13 further comprising:

assigning, for all families (X_v,X_pa(v)) where X_v is a continuous variable, each configuration of discrete parent variables x^d_pa(v) a local sample gradient vector and setting it to zero;

For each data case in a data set;

utilizing a propagation scheme for Bayesian networks with conditional Gaussian distributions to determine a mean vector and a covariance matrix associated with a posterior marginal distribution for all families (X_v,X_pa(v)), where X_v is a continuous variable;

employing posterior conditional means and variances to determine expected values (x_v*,x^c_pa(v)*) and ((x_vx_v)*, (x_v x^c_pa(v))*, (x_v x^c_pa(v)′

)*, (x^c_pa(v)′

x^c_pa(v))*) for continuous variables in each of these families given each configuration of discrete variables in the respective families;

determining sample parameter gradients utilizing a formula for a sample of incomplete observations; and

adding single observation gradients to the sample parameter gradients.

31. A device employing the method of claim 13 comprising at least one selected from the group consisting of a computer, a server, and a handheld electronic device.

28. A system that facilitates statistical modeling, comprising:

means for receiving at least one data set containing a set of variables; and

means for utilizing the data set for the set of variables and probabilistic inference to determine parameter gradients for a log-likelihood of a conditional Gaussian (CG) graphical model over those variables with at least one continuous variable and with incomplete observation data for at least one of the variables.

29. A data packet, transmitted between two or more computer components, that facilitates statistical modeling, the data packet comprising, at least in part, information relating to a gradient determination system that determines gradients for a log-likelihood of a conditional Gaussian graphical model from a data set for a set of variables and probabilistic inference over those variables with at least one continuous variable and with incomplete observation data for at least one of the variables.