Approximate multivariate posterior probability distributions from simulated samples

• US 10,146,741 B2
• Filed: 03/18/2014
• Issued: 12/04/2018
• Est. Priority Date: 03/15/2013
• Status: Active Grant

• Current Assignee: SAS Institute Incorporated
• Original Assignee: SAS Institute Incorporated
• Inventors: Macaro, Christian, Chvosta, Jan, Little, Mark Roland
• Primary Examiner(s): Chad, Aniss
• Assistant Examiner(s): Mikowski, Justin C
• Application Number:
  US14/217,858
• Publication Number:
  US 20140278239A1
• Time in Patent Office: 1,722 Days
• Field of Search: 703 2
• CPC Class Codes: G06F 17/18 for evaluating statistical data , e.g. average values, frequency distributions, probability functions, regression analysis forecasting specially adapted for a specific administrative, business or logistic context G06Q10/04
• CPC Class Codes: G06N 7/01 Probabilistic graphical models, e.g. probabilistic networks

Abstract

Various embodiments are directed to techniques for deriving a sample representation from a random sample. A computer-program product includes instructions to cause a first computing device to fit an empirical distribution function to a marginal probability distribution of a variable within a first sample portion of a random sample to derive a partial marginal probability distribution approximation, wherein the random sample is divided into multiple sample portions distributed among multiple computing devices; fit a first portion of a copula function to a multivariate probability distribution of the first sample portion, wherein the copula function is divided into multiple portions; and transmit an indication of a first likelihood contribution of the first sample portion to a coordinating device to cause a second computing device to fit a second portion of the copula function to a multivariate probability distribution of a second sample portion. Other embodiments are described and claimed.

Claims

1. A computer-program product tangibly embodied in a non-transitory machine-readable storage medium, the computer-program product including instructions operable to cause a first computing device of a plurality of computing devices to perform operations including:

generate a first sample portion of a single random sample from a single posterior probability distribution at least partially in parallel with at least a second computing device of the plurality of computing devices, wherein;

the second computing device generates a second sample portion of the single random sample from the single posterior distribution; and

the single random sample comprises a plurality of sample portions that comprises the first sample portion and the second sample portion;

fit an empirical distribution function to a marginal probability distribution of a variable within the first sample portion of the single random sample to derive a first partial marginal probability distribution approximation of a plurality of partial marginal probability distribution approximations at least partially in parallel with the second computing device, wherein;

the second computing device fits the empirical distribution function to a marginal probability distribution of the variable within the second sample portion of the single random sample to derive a second partial marginal probability distribution approximation of the plurality of partial marginal probability distribution approximations;

transmit the first partial marginal probability distribution approximation to a coordinating device at least partially in parallel with the second computing device, wherein;

the second computing device transmits the second partial marginal probability distribution approximation to the coordinating device; and

the coordinating device generates a marginal probability distribution approximation of the single random sample from the plurality of partial marginal probability distribution approximations transmitted to the coordinating device at least partially in parallel by the plurality of computing devices;

fit a first portion of a copula function of a plurality of portions of the copula function to a multivariate probability distribution of the first sample portion of the single random sample to generate a first likelihood contribution of a plurality of likelihood contributions at least partially in parallel with the second computing device, wherein;

the second computing device fits a second portion of the plurality of portions of the copula function to a multivariate probability distribution of the second sample portion of the single random sample to generate a second likelihood contribution of the plurality of likelihood contributions;

transmit an indication of the first likelihood contribution of the first sample portion to the coordinating device at least partially in parallel with the second computing device, wherein;

the second computing device transmits an indication of the second likelihood contribution of the second sample portion to the coordinating device; and

the coordinating device determines whether the fit of the copula function, performed as the fit of the plurality of portions of the copula function by the plurality of computing devices, meets a predetermined criterion for generation of a reduced size representation of the single random sample that achieves a predetermined degree of fit of the copula function based on the marginal probability distribution approximation and based on the plurality of likelihood contributions to determine whether the fit of the copula function is to be repeated; and

in response to receipt of copula coefficients generated and transmitted by the coordinating device in response to a determination by the coordinating device that the fit of the copula function is to be repeated, perform an iteration of operations at least partially in parallel with the second computing device, each iteration comprising;

repeat the fitting of the first portion of the copula function to the multivariate probability distribution of the first sample portion based on the copula coefficients to repeat generation of the first likelihood contribution of the first sample; and

repeat the transmission of the indication of the first likelihood contribution of the first sample portion to the coordinating device to enable the coordinating device to again determine whether the fit of the copula function meets the predetermined criterion to again determine whether the fit of the copula function is to be again repeated by the plurality of computing devices in another iteration.

2. The computer-program product of claim 1, the first computing device caused to perform operations to reduce a quantity of data exchanged between the first computing device and the coordinating device, the operations including:

compare a quantity of draws of parameters within at least the first sample portion to a threshold quantity of draws;

select the empirical distribution function as the first partial marginal probability distribution approximation associated with the variable based on the quantity of draws within at least the first sample portion exceeding the threshold; and

select quantiles taken of the empirical distribution function as the first partial marginal probability distribution approximation based on the quantity of draws within at least the first sample portion not exceeding the threshold.

3. The computer-program product of claim 1, wherein

the first computing device caused to perform operations including:

combine the first partial marginal probability distribution approximation with at least the second partial marginal probability distribution approximation to generate the marginal probability distribution approximation; and

combine the first portion of the copula function with at least the second portion of the copula function to generate the copula function.

4. The computer-program product of claim 3, the first computing device caused to perform operations including:

derive an inverse cumulative distribution function (ICDF) of the marginal probability distribution approximation; and

employ at least the inverse marginal CDF, the copula function and a random number generator to generate a simulated random sample as a representation of the random sample, wherein the simulated random sample is smaller in size than the random sample, and wherein the simulated random sample and the random sample have at least one similar statistical characteristic.

5. The computer-program product of claim 3, the first computing device caused to perform operations including:

derive a prior probability distribution from the marginal probability distribution approximation and the copula function; and

perform another Bayesian analysis using the prior probability distribution.

6. The computer-program product of claim 3, the first computing device caused to perform operations including:

derive an inverse marginal CDF of the marginal probability distribution approximation; and

employ at least the inverse marginal CDF and the copula function as a representation of the random sample in an analysis of a statistical characteristic of the random sample.

7. The computer-program product of claim 3, the first computing device caused to perform operations including transmit at least the marginal probability distribution approximation and the copula function to another computing device as a representation of the random sample.

8. The computer-program product of claim 1, wherein the first computing device caused to exchange communications with at least the second computing device to coordinate generation of the single posterior probability distribution among the plurality of computing devices.

9. A computer-implemented method comprising:

generating, on a first computing device of a plurality of computing devices, a first sample portion of a single random sample from a single posterior probability distribution at least partially in parallel with at least a second computing device of the plurality of computing devices, wherein;

the second computing device generates a second sample portion of the single random sample from the single posterior distribution; and

the single random sample comprises a plurality of sample portions that comprises the first sample portion and the second sample portion;

fitting, on the first computing device, an empirical distribution function to a marginal probability distribution of a variable within the first sample portion of the single random sample to derive a first partial marginal probability distribution approximation of a plurality of partial marginal probability distribution approximations at least partially in parallel with the second computing device, wherein;

the second computing device fits the empirical distribution function to a marginal probability distribution of the variable within the second sample portion of the single random sample to derive a second partial marginal probability distribution approximation of the plurality of partial marginal probability distribution approximations;

transmitting, from the first computing device, the first partial marginal probability distribution approximation to a coordinating device at least partially in parallel with the second computing device, wherein;

the second computing device transmits the second partial marginal probability distribution approximation to the coordinating device; and

the coordinating device generates a marginal probability distribution approximation of the single random sample from the plurality of partial marginal probability distribution approximations transmitted to the coordinating device at least partially in parallel by the plurality of computing devices;

fitting, on the first computing device, a first portion of a copula function of a plurality of portions of the copula function to a multivariate probability distribution of the first sample portion of the single random sample to generate a first likelihood contribution of a plurality of likelihood contributions at least partially in parallel with the second computing device, wherein;

the second computing device fits a second portion of the plurality of portions of the copula function to a multivariate probability distribution of the second sample portion of the single random sample to generate a second likelihood contribution of the plurality of likelihood contributions;

transmitting, from the first computing device, an indication of the first likelihood contribution of the first sample portion to the coordinating device at least partially in parallel with the second computing device, wherein;

the second computing device transmits an indication of the second likelihood contribution of the second sample portion to the coordinating device; and

the coordinating device determines whether the fit of the copula function, performed as the fit of the plurality of portions of the copula function by the plurality of computing devices, meets a predetermined criterion for generation of a reduced size representation of the single random sample that achieves a predetermined degree of fit of the copula function based on the marginal probability distribution approximation and based on the plurality of likelihood contributions to determine whether the fit of the copula function is to be repeated; and

in response to receipt of copula coefficients generated and transmitted by the coordinating device in response to a determination by the coordinating device that the fit of the copula function is to be repeated, performing an iteration of operations at least partially in parallel with the second computing device, each iteration comprising;

repeating, on the first computing device, the fitting of the first portion of the copula function to the multivariate probability distribution of the first sample portion based on the copula coefficients to repeat generation of the first likelihood contribution of the first sample; and

repeating, on the first computing device, the transmission of the indication of the first likelihood contribution of the first sample portion to the coordinating device to enable the coordinating device to again determine whether the fit of the copula function meets the predetermined criterion to again determine whether the fit of the copula function is to be again repeated by the plurality of computing devices in another iteration.

10. The computer-implemented method of claim 9, comprising operations to reduce a quantity of data exchanged between the first computing device and the coordinating device, the operations comprising:

comparing, on the first computing device, a quantity of draws of parameters within at least the first sample portion to a threshold quantity of draws;

selecting, on the first computing device, the empirical distribution function as the first partial marginal probability distribution approximation associated with the variable based on the quantity of draws within at least the first sample portion exceeding the threshold; and

selecting, on the first computing device, quantiles taken of the empirical distribution function as the first partial marginal probability distribution approximation based on the quantity of draws within at least the first sample portion not exceeding the threshold.

11. The computer-implemented method of claim 9,

comprising:

combining, on the first computing device, the first partial marginal probability distribution approximation with at least a second partial marginal probability distribution approximation to generate the marginal probability distribution approximation; and

combining, on the first computing device, the first portion of the copula function with at least the second portion of the copula function to generate the copula function.

12. The computer-implemented method of claim 11, comprising:

deriving, on the first computing device, an inverse cumulative distribution function (ICDF) of the marginal probability distribution approximation; and

employing, on the first computing device, at least the inverse marginal CDF, the copula function and a random number generator to generate a simulated random sample as a representation of the random sample, wherein the simulated random sample is smaller in size than the random sample, and wherein the simulated random sample and the random sample have at least one similar statistical characteristic.

13. The computer-implemented method of claim 11, comprising:

deriving, on the first computing device, a prior probability distribution from the marginal probability distribution approximation and the copula function; and

performing another Bayesian analysis using the prior probability distribution.

14. The computer-implemented method of claim 11, comprising:

deriving, on the first computing device, an inverse marginal CDF of the marginal probability distribution approximation; and

employing at least the inverse marginal CDF and the copula function as a representation of the random sample in an analysis of a statistical characteristic of the random sample.

15. The computer-implemented method of claim 11, comprising transmitting, from the first computing device, at least the marginal probability distribution approximation and the copula function to another computing device as a representation of the random sample.

16. The computer-implemented method of claim 9, exchanging, on the first computing device, communications with at least the second computing device to coordinate generation of the single posterior probability distribution among the plurality of computing devices.

17. An apparatus comprising a processor and a storage to store instructions that, when executed by the processor, cause the processor to perform operations comprising:

generate, on a first computing device of a plurality of computing devices, a first sample portion of a single random sample from a single posterior probability distribution at least partially in parallel with at least a second computing device of the plurality of computing devices, wherein;

the second computing device generates a second sample portion of the single random sample from the single posterior distribution; and

the single random sample comprises a plurality of sample portions that comprises the first sample portion and the second sample portion;

fit, on the first computing device, an empirical distribution function to a marginal probability distribution of a variable within the first sample portion of the single random sample to derive a first partial marginal probability distribution approximation of a plurality of partial marginal probability distribution approximations at least partially in parallel with the second computing device, wherein;

the second computing device fits the empirical distribution function to a marginal probability distribution of the variable within the second sample portion of the single random sample to derive a second partial marginal probability distribution approximation of the plurality of partial marginal probability distribution approximations;

transmit, from the first computing device, the first partial marginal probability distribution approximation to a coordinating device at least partially in parallel with the second computing device, wherein;

the second computing device transmits the second partial marginal probability distribution approximation to the coordinating device; and

the coordinating device generates a marginal probability distribution approximation of the single random sample from the plurality of partial marginal probability distribution approximations transmitted to the coordinating device at least partially in parallel by the plurality of computing devices;

fit, on the first computing device, a first portion of a copula function of a plurality of portions of the copula function to a multivariate probability distribution of the first sample portion of the single random sample to generate a first likelihood contribution of a plurality of likelihood contributions at least partially in parallel with the second computing device, wherein;

the second computing device fits a second portion of the plurality of portions of the copula function to a multivariate probability distribution of the second sample portion of the single random sample to generate a second likelihood contribution of the plurality of likelihood contributions;

transmit, from the first computing device, an indication of the first likelihood contribution of the first sample portion to the coordinating device at least partially in parallel with the second computing device, wherein;

the second computing device transmits an indication of the second likelihood contribution of the second sample portion to the coordinating device; and

the coordinating device determines whether the fit of the copula function, performed as the fit of the plurality of portions of the copula function by the plurality of computing devices, meets a predetermined criterion for generation of a reduced size representation of the single random sample that achieves a predetermined degree of fit of the copula function based on the marginal probability distribution approximation and based on the plurality of likelihood contributions to determine whether the fit of the copula function is to be repeated; and

in response to receipt of copula coefficients generated and transmitted by the coordinating device in response to a determination by the coordinating device that the fit of the copula function is to be repeated, performing an iteration of operations at least partially in parallel with the second computing device, each iteration comprising;

repeat, on the first computing device, the fitting of the first portion of the copula function to the multivariate probability distribution of the first sample portion based on the copula coefficients to repeat generation of the first likelihood contribution of the first sample; and

repeat, on the first computing device, the transmission of the indication of the first likelihood contribution of the first sample portion to the coordinating device to enable the coordinating device to again determine whether the fit of the copula function meets the predetermined criterion to again determine whether the fit of the copula function is to be again repeated by the plurality of computing devices in another iteration.

18. The apparatus of claim 17, wherein the processor is caused to perform operations to reduce a quantity of data exchanged between the first computing device and the coordinating device, the operations including:

compare, on the first computing device, a quantity of draws of parameters within at least the first sample portion to a threshold quantity of draws;

select, on the first computing device, the empirical distribution function as the first partial marginal probability distribution approximation associated with the variable based on the quantity of draws within at least the first sample portion exceeding the threshold; and

select, on the first computing device, quantiles taken of the empirical distribution function as the first partial marginal probability distribution approximation based on the quantity of draws within at least the first sample portion not exceeding the threshold.

19. The apparatus of claim 17, wherein

the processor is caused to perform operations including:

combine, on the first computing device, the first partial marginal probability distribution approximation with at least the second partial marginal probability distribution approximation to generate the marginal probability distribution approximation; and

combine, on the first computing device, the first portion of the copula function with at least the second portion of the copula function to generate the copula function.

20. The apparatus of claim 19, wherein the processor is caused to perform operations including:

derive, on the first computing device, an inverse cumulative distribution function (ICDF) of the marginal probability distribution approximation; and

employ, on the first computing device, at least the inverse marginal CDF, the copula function and a random number generator to generate a simulated random sample as a representation of the random sample, wherein the simulated random sample is smaller in size than the random sample, and wherein the simulated random sample and the random sample have at least one similar statistical characteristic.

21. The apparatus of claim 19, wherein the processor is caused to perform operations including:

derive, on the first computing device, a prior probability distribution from the marginal probability distribution approximation and the copula function; and

perform, on the first computing device, another Bayesian analysis using the prior probability distribution.

22. The apparatus of claim 19, wherein the processor is caused to perform operations including:

derive, on the first computing device, an inverse marginal CDF of the marginal probability distribution approximation; and

employ, on the first computing device, at least the inverse marginal CDF and the copula function as a representation of the random sample in an analysis of a statistical characteristic of the random sample.

23. The apparatus of claim 19, wherein the processor is caused to perform operations including transmit, from the first computing device, at least the marginal probability distribution approximation and the copula function to another computing device as a representation of the random sample.

24. The apparatus of claim 17, wherein the processor is caused to, at the first computing device, exchange communications with at least the second computing device to coordinate generation of the single posterior probability distribution among the plurality of computing devices.