Device, method, and medium for predicting a probability of an occurrence of a data

• US 20030065478A1
• Filed: 08/26/2002
• Published: 04/03/2003
• Est. Priority Date: 06/18/1998
• Status: Active Grant

• Current Assignee: NEC CORPORATION, YALE UNIVERSITY
• Original Assignee: NEC CORPORATION, YALE UNIVERSITY
• Inventors: Barron, Andrew R., Takeuchi, Jun-Ichi
• Granted Patent: US 6,892,164 B2
• Time in Patent Office: Days
• US Class Current: 702/181
• CPC Class Codes: G06F 17/18 for evaluating statistical data , e.g. average values, frequency distributions, probability functions, regression analysis forecasting specially adapted for a specific administrative, business or logistic context G06Q10/04

Abstract

In a Bayes mixture probability density calculator for calculating Bayes mixture probability density which reduces a logarithmic loss A modified Bayes mixture probability density is calculated by mixing traditional Bayes mixture probability density calculated on given model S with a small part of Bayes mixture probability density for exponential fiber bundle on the S. Likewise, a prediction probability density calculator is configured by including the Bayes mixture probability density calculator, and by using Jeffreys prior distribution in traditional Bayes procedure on the S.

Claims

1. A Bayes mixture density calculator operable in response to a sequence of vectors xⁿ=(x₁, x₂, . . . , x_n) selected from a vector value set χ

to produce a Bayes mixture density on occurrence of the xⁿ, comprising;

a probability density calculator, supplied with a sequence of data x^t and a vector value parameter u, for calculating a probability density for the x^t, p(x^t|u);

a Bayes mixture calculator for calculating a first approximation value of a Bayes mixture density p_w(xⁿ) on the basis of a prior distribution w(u) predetermined by the probability density calculator to produce the first approximation value;

an enlarged mixture calculator for calculating a second approximation value of a Bayes mixture m(xⁿ) on exponential fiber bundle in cooperation with the probability density calculator to produce the second approximation value; and

a whole mixture calculator for calculating (1−

ε

)p_w(xⁿ+ε

˜

m(xⁿ) to produce a calculation result by mixing the first approximation value of the Bayes mixture density p_w(xⁿ) with a part of the second approximation value of the Bayes mixture m(xⁿ) at a rate of 1−

ε

;

ε

to produce the calculation result where ε

is a value smaller than unity.

5. A prediction probability density calculator operable in response to a sequence of vector xⁿ=(x₁, x₂, . . . , x_n) selected from a vector value set χ

and x_n+1 to produce a prediction probability density on occurrence of the x_n+1, comprising;

a joint probability calculator structured by the Bayes mixture density calculator claimed in claim 1 for calculating a modified Bayes mixture density q<sup>(ε

)</sup>(xⁿ) and q<sup>(ε

)</sup>(xⁿ⁺¹) based on predetermined prior distribution to produce first calculation results; and

a divider responsive to the calculation results for calculating probability density q<sup>(ε

)</sup>(xⁿ⁺¹)/q<sup>(ε

)</sup>(xⁿ) to produce a second calculation result with the first calculation results kept intact.

2. A Jeffreys mixture density calculator operable in response to a sequence of vector xⁿ=(x₁, x₂, . . . , x_n) selected from a vector value set χ

to produce a Bayes mixture density on occurrence of the xⁿ, comprising;

a probability density calculator responsive to a sequence of data x^t and a vector value parameter u for calculating a probability density p(x^t|u) for the x^t;

a Jeffreys mixture calculator for calculating a first approximation value of a Bayes mixture density p_J(xⁿ) based on a Jeffreys prior distribution w_J(u) in cooperation with the probability density calculator to produce the first approximation value;

an enlarged mixture calculator for calculating a second approximation value of a Bayes mixture m(xⁿ) on exponential fiber bundle in cooperation with the probability density calculator to produce the second approximation value; and

a whole mixture calculator for calculating (1−

ε

)p_J(xⁿ)+ε

·

m(xⁿ) to produce a calculation result by mixing the first approximation value of the Bayes mixture density p_J(xⁿ) with a part of the second approximation value of the Bayes mixture m(xⁿ) at a rate of 1−

ε

;

ε

to produce the calculation result where ε

is a value smaller than unity.

6. A prediction probability density calculator operable in response to a sequence of vector xⁿ=(x₁, x₂, . . . , x_n) selected from a vector value set χ

and x_n+1 to produce a prediction probability density on occurrence of the x_n+1, comprising;

a joint probability calculator structured by the Jeffreys mixture density calculator claimed in claim 2 for calculating a modified Jeffreys mixture density q<sup>(ε

)</sup>(xⁿ) and q<sup>(ε

)</sup>(xⁿ⁺¹) to produce first calculation results; and

a divider response to the calculation results for calculating a probability density q<sup>(ε

)</sup>(xⁿ⁺¹)/q<sup>(ε

)</sup>(xⁿ) to produce a second calculation result with the first calculation results kept intact.

3. A Bayes mixture density calculator operable in response to a sequence of vector xⁿ=(x₁, x₂, . . . , x_n) selected from a vector value set χ

to produce a Bayes mixture density on occurrence of the xⁿ, comprising;

a probability density calculator responsive to a sequence of data x^t and a vector value parameter u for outputting probability density p(x^t|u) for the x^t on curved exponential family;

a Bayes mixture calculator for calculating a first approximation value of a Bayes mixture density p_w(xⁿ) on the basis of a prior distribution w(u) predetermined by the probability density calculator to produce the first approximation value;

an enlarged mixture calculator for calculating a second approximation value of a Bayes mixture m(xⁿ) on exponential family including curved exponential family in cooperation with the probability density calculator to produce the second approximation value; and

a whole mixture calculator for calculating (1−

ε

)p_w(xⁿ)+ε

·

m(xⁿ) to produce a calculation result by mixing the first approximation value of the Bayes mixture density p_w(xⁿ) with a part of the second approximation value of the Bayes mixture m(xⁿ) at a rate of 1−

ε

;

ε

to produce the calculation result where ε

is a value smaller than unity.

7. A prediction probability density calculator operable in response to a sequence of vector xⁿ=(x₁, x₂, . . . , x_n) selected from a vector value set χ

and x_n+1 to produce a prediction probability density on occurrence of the x_n+1, comprising;

a joint probability calculator structured by the Bayes mixture density calculator claimed in claim 3 for calculating a modified Bayes mixture density q<sup>(ε

)</sup>(xⁿ) and q<sup>(ε

)</sup>(xⁿ⁺¹) based on a predetermined prior distribution to produce first calculation results; and

a divider responsive to the calculation results for calculating a probability density q<sup>(ε

)</sup>(xⁿ⁺¹)/q<sup>(ε

)</sup>(xⁿ) to produce a second calculation result with the first calculation results kept intact.

4. A Jeffreys mixture density calculator operable in response to a sequence of vector xⁿ=(x₁, x₂, . . . , x_n) selected from a vector value set χ

to produce a Bayes mixture density on occurrence of the xⁿ, comprising;

a probability density calculator responsive to a sequence of data x^t and a vector value parameter u for calculating probability density p(x^t|u) for the x^t on curved exponential family;

a Jeffreys mixture calculator for calculating a first approximation value of a Bayes mixture density p_J(xⁿ) based on a Jeffreys prior distribution w_J(u) in cooperation with the probability density calculator to produce the first approximation value;

an enlarged mixture calculator for calculating a second approximation value of a Bayes mixture m(xⁿ) on exponential family including curved exponential family in cooperation with the probability density calculator to produce the second approximation value; and

a whole mixture calculator for calculating (1−

ε

)p_J(xⁿ)+ε

·

m(xⁿ) to produce a calculation result by mixing the first approximation value of the Bayes mixture density p_J(xⁿ) with a part of the second approximation value of the Bayes mixture m(xⁿ) at a ratio of 1−

ε

;

ε

to produce the calculation result where ε

is a value smaller than unity.

8. A prediction probability density calculator operable in response to a sequence of vector xⁿ=(x₁, x₂, . . . , x_n) selected from a vector value set χ

and x_n+1 to produce a prediction probability density on occurrence of the x_n+1, comprising;

a joint probability calculator structured by the Jeffreys mixture density probability calculator claimed in claim 4 for calculating a modified Jeffreys mixture density q<sup>(ε

)</sup>(xⁿ) and q<sup>(ε

)</sup>(xⁿ⁺¹) to produce first calculation results; and

a divider responsive to the calculation results for calculating a probability density q<sup>(ε

)</sup>(xⁿ⁺¹)/q<sup>(ε

)</sup>(xⁿ) to produce a second calculation result with the first calculation results kept intact.

9. A mixture density calculator operable in response to a sequence of data xⁿ=(x₁, x₂, . . . , x_n) to produce a mixture density on occurrence of the xⁿ, comprising:

receiving means for receiving the sequence of data xⁿ;

first calculation means for calculating a first Bayes mixture density on a hypothesis class;

second calculation means for calculating a second Bayes mixture density on an enlarged hypothesis class; and

means for obtaining a modified Bayes mixture density for the xⁿ by mixing the first Bayes mixture density with the second Bayes mixture density in a predetermined proportion to produce the modified Bayes mixture density as said mixture density.

10. A mixture density calculator as claimed in claim 9, wherein the first Bayes mixture density and the second Bayes mixture density are calculated by the use of a predetermined prior distribution.

11. A mixture density calculator as claimed in claim 9, in calculating the first Bayes mixture density and second Bayes mixture density, uses Jeffreys prior distribution.

12. A mixture density calculator as claimed in claim 9, wherein the first Bayes mixture density and the second Bayes mixture density are mixed together at a rate of 1−

ε

;

ε

, where ε

take a value smaller than unity.

13. A mixture density calculator as claimed in claim 9, wherein the hypothesis class belongs to the curved exponential family.

14. A prediction probability density calculator operable in response to a sequence of data xⁿ=(x₁, x₂, . . . , x_n) and a data x_n+1 to produce a prediction probability density on occurrence of the x_n+1, comprising:

receiving means for receiving the data xⁿ and x_n+1;

first calculating means for calculating first Bayes mixture densities, on a hypothesis class, for the sequence of data xⁿ and an sequence of data xⁿ⁺¹ which representing (x₁, x₂, . . . , x_n, x_n+1);

second calculating means for calculating second Bayes mixture densities, on an enlarged hypothesis class, for the xⁿ and the xⁿ⁺¹;

means for obtaining modified Bayes mixture densities for the x_n and the xⁿ⁺¹ by mixing the first Bayes mixture densities for the xⁿ and the xⁿ⁺¹ with the second Bayes mixture densities for the xⁿ and the xⁿ⁺¹, in a predetermined proportion to produce the modified Bayes mixture density as said mixture density, respectively; and

means for obtaining the prediction probability density by dividing the modified Bayes mixture density for the xⁿ⁺¹ by the modified Bayes mixture density for the xⁿ.

15. A prediction probability density calculator as claimed in claim 14, wherein the first Bayes mixture densities and the second Bayes mixture densities are calculated by the use of a predetermined prior distribution.

16. A prediction probability density calculator as claimed in claim 14, wherein the first Bayes mixture densities and the second Bayes mixture densities are calculated by the use of Jeffreys prior distribution.

17. A prediction probability density calculator as claimed in claim 14, wherein the first Bayes mixture densities and the second Bayes mixture densities are mixed together at a rate of 1−

ε

;

ε

, where ε

take a value smaller than unity.

18. A prediction probability density calculator as claimed in claim 14, wherein the hypothesis class belongs to curved exponential family.

19. A method operable in response to a sequence of data xⁿ=(x₁, x₂, . . . , x_n) to produce a mixture density on occurrence of the xⁿ, comprising the steps of:

receiving the sequence of data xⁿ;

calculating a first Bayes mixture density on a hypothesis class;

calculating a second Bayes mixture density on an enlarged hypothesis class; and

obtaining a modified Bayes mixture density for the xⁿ by mixing the first Bayes mixture density with the second Bayes mixture density in a predetermined proportion.

20. A method operable in response to a sequence of data xⁿ=(x₁, x₂, . . . , x_n) and a data x_n+1 to produce a prediction probability density on occurrence of the x_n+1, comprising the steps of:

receiving the data xⁿ and x_n+1; and

repeating, for each sequence of data xⁿ and xⁿ⁺¹ representing (x₁, x₂, . . . , x_n, x_n+1), the following first through third substeps of;

(1) calculating a first Bayes mixture density, on a hypothesis class, for the sequence of data;

(2) calculating a second Bayes mixture density, on an enlarged hypothesis class, for the data; and

(3) obtaining a modified Bayes mixture density for the data by mixing the first Bayes mixture density with the second Bayes mixture density in a predetermined proportion; and

obtaining the prediction probability density by dividing the modified Bayes mixture density for the xⁿ⁺¹ by the modified Bayes mixture density for the xⁿ.

21. A computer readable medium which stores a program operable in response to an input a sequence of data xⁿ=(x₁, x₂, . . . , x_n) to produce a mixture density on occurrence of the xⁿ, the program comprising the steps of:

receiving the sequence of data xⁿ;

calculating a first Bayes mixture density on a hypothesis class;

calculating a second Bayes mixture density on an enlarged hypothesis class; and

obtaining a modified Bayes mixture density for the xⁿ by mixing the first Bayes mixture density with the second Bayes mixture density in a predetermined proportion.

22. A computer readable medium which stores a program which is operable in response to an input a sequence of data xⁿ=(x₁, x₂, . . . , x_n) and a data x_n+1 to produce a prediction probability density on occurrence of the x_n+1, the program comprising the steps of:

receiving the data xⁿ and xⁿ⁺¹;

repeating, for each sequence of data xⁿ and xⁿ⁺¹ which representing (x₁, x₂, . . . , x_n, x_n+1), the following substeps;

(1) calculating a first Bayes mixture density, on a hypothesis class, for the sequence of data;

(2) calculating a second Bayes mixture density, on an enlarged hypothesis class, for the data;

(3) obtaining a modified Bayes mixture density for the data by mixing the first Bayes mixture density with the second Bayes mixture density, in a predetermined proportion; and

obtaining the prediction probability density by dividing the modified Bayes mixture density for the xⁿ⁺¹ by the modified Bayes mixture density for the xⁿ.