KNOWLEDGE DISCOVERY FROM CITATION NETWORKS

• US 20140188780A1
• Filed: 01/14/2014
• Published: 07/03/2014
• Est. Priority Date: 12/06/2010
• Status: Active Grant

• Current Assignee: The Research Foundation for The State University of New York (State University of New York)
• Original Assignee: The Research Foundation for The State University of New York (State University of New York)
• Inventors: Guo, Zhen, Zhang, Mark
• Granted Patent: US 8,930,304 B2
• Time in Patent Office: Days
• US Class Current: 706/52
• CPC Class Codes: G06F 16/24573 using data annotations, e.g. user-defined metadata
• CPC Class Codes: G06F 16/24578 using ranking
• CPC Class Codes: G06F 16/93 Document management systems
• CPC Class Codes: G06F 16/9558 Details of hyperlinks; Management of linked annotations
• CPC Class Codes: G06N 20/00 Machine learning
• CPC Class Codes: G06N 7/01 Probabilistic graphical models, e.g. probabilistic networks
• CPC Class Codes: G06Q 10/10 Office automation; Time management

Abstract

In a corpus of scientific articles such as a digital library, documents are connected by citations and one document plays two different roles in the corpus: document itself and a citation of other documents. A Bernoulli Process Topic (BPT) model is provided which models the corpus at two levels: document level and citation level. In the BPT model, each document has two different representations in the latent topic space associated with its roles. Moreover, the multi-level hierarchical structure of the citation network is captured by a generative process involving a Bernoulli process. The distribution parameters of the BPT model are estimated by a variational approximation approach.

Claims

1. A method for characterizing a set of documents, comprising:

identifying a network of multilevel hierarchically related documents having direct and indirect references associated with content relationships;

for each respective document, determining a set of latent topic characteristics based on an intrinsic content of the respective document and a set of latent topic characteristics based on a respective content of other documents which are directly referenced and indirectly referenced through at least one other document to the respective document;

representing a set of latent topics for the respective document based on a joint probability distribution of at least the latent topic characteristics based on the intrinsic content and the respective content of other documents which are directly referenced and indirectly referenced through at least one other document to the respective document, dependent on the identified network and a random process; and

storing, in a memory, the represented set of latent topics for the respective document.

2. The method according to claim 1, wherein the network comprises a Bayesian network structure.

3. The method according to claim 1, wherein relationships among the documents are modeled by a Bernoulli process such that a topic distribution of each respective document is a mixture of distributions associated with the related document.

4. The method according to claim 1, wherein the set of documents is modeled by a generative probabilistic model of a topic content of the set of documents along with the references among the documents.

5. The method according to claim 1, wherein the represented set of latent topics is modeled at both a document level and a reference level, by differentiating the two different levels and the multilevel hierarchical network which is captured by a Bernoulli random process.

6. The method according to claim 1, wherein the iterative process at a reference level comprises iterating, for each document d_j for the i-th location in document d_j, choosing a topic z_ji from the topic distribution of document d_j, p(z|d_j,θ

_dj), where the distribution parameter θ

_dj is drawn from a Dirichlet distribution Dir(α

), choosing a word w_ji which follows the multinomial distribution p(w|z_ji, Λ

) conditioned on the topic z_ji, and incrementing the locations and documents,

wherein the iterative process at a document level comprises iterating, for each document d_s, for the i-th location in document d_s, choosing a referenced document c_si from p(c|d_s,Ξ

), a multinomial distribution conditioned on the document d_s, choosing a topic t_si from the topic distribution of the document c_si at the reference level, and choosing a word w_si which follows the multinomial distribution p(w|t_si, Λ

) conditioned on the topic t_si, where Ξ

is a mixing coefficient matrix which represents how much of the content of the respective document is from direct or indirect references, and a composition of Ξ and

θ

models the topic distribution at the document level, and incrementing the locations and documents,

wherein a number of latent topics is K and the mixing coefficients are parameterized by an N×

N matrix Ξ

where Ξ

_js=p(c_si=d_j|d_s), which are treated as a fixed quantity computed from the reference information of the set of documents,

wherein topic distributions at the reference level are parameterized by a K×

N matrix θ

where θ

_lj=p(z_ji=l|d_j), which is to be estimated, and an M×

K word probability matrix Λ

, where Λ

_hl=p(w_si^h=1|t_si=l), which is to be estimated, and

wherein the references comprise citations, each document d_s having a set of citations Q_ds, further comprising constructing a matrix S to denote direct relationships among the documents wherein

7. The method according to claim 1, wherein a generative process for the set of documents leads to a joint probability distribution

8. The method according to claim 1, wherein a generative process for the set of documents leads to a joint distribution of c,z,θ

represented as;

a→

θ

→

z→

w_|c

d→

c→

t→

w_|d|

θ

→

t

Ξ

→

C

w<sub>|c|←

Λ

→

w|z|</sub>

and update rules for the iterative process comprise;

9. The method according to claim 8, wherein the iterative update rules

10. The method according to claim 8, wherein the iterative update rules

11. A method for characterizing a set of documents, comprising:

identifying a multilevel hierarchy of documents having direct references and indirect references, having citation linkages with semantic significance;

for each respective document, determining latent topic characteristics based on an intrinsic semantic content of the respective document, semantic content associated with directly cited documents, and semantic content associated with documents referenced by directly cited documents, wherein a semantic content significance of a citation has a transitive property;

representing latent topics for documents within the set of documents based on a joint probability distribution of the latent topic characteristics, wherein distinctions in the multilevel hierarchical network are captured by a random process; and

storing, in a memory, the represented set of latent topics.

12. The method according to claim 11, wherein relationships among the set of documents are modeled by a Bernoulli process such that a topic distribution of each respective document is a mixture of distributions associated with the linked documents.

13. The method according to claim 11, wherein the set of documents is modeled by a generative probabilistic model of a topic content of each document of the set of documents along with the linkages among members of the set of documents.

14. The method according to claim 11, wherein the latent topics are modeled at both a document level and a citation level, and distinctions in the multilevel hierarchical network are captured by a Bernoulli random process.

15. The method according to claim 11, wherein the joint probability distribution is estimated by an iterative process at a citation level, comprising, for each document d_j, and for the i-th location in document d_j, choosing a topic z_ji from the topic distribution of document d_j, p(z|d_j,θ

_dj), where the distribution parameter θ

_dj is drawn from a Dirichlet distribution Dir(α

), choosing a word w_ji which follows the multinomial distribution p(w|z_ji, Λ

) conditioned on the topic z_ji, and respectively incrementing the locations and documents,

wherein the joint probability distribution is estimated by an iterative process at a document level comprising, for each document d_s, and for the i-th location in document c_si choosing a cited document c_si from p(c|d_s,Ξ

), a multinomial distribution conditioned on the document d_s, choosing a topic t_si from the topic distribution of the document c at the citation level, and choosing a word which follows the multinomial distribution p(w|t_si, Λ

) conditioned on the topic t_si where Ξ

is a mixing coefficient matrix which represents how much of the content of the respective document is from direct or indirect references, and a composition of Ξ and

θ

models the topic distribution at the document level, and respectively incrementing the locations and documents,

wherein a number of latent topics is K and the mixing coefficients are parameterized by an N×

N matrix Ξ

where Ξ

_js=p(c_si=d_j|d_s), which are treated as a fixed quantity computed from the citation information of the set of documents,

wherein topic distributions at the citation level are parameterized by a K×

N matrix θ

where θ

=p(z_ji=l|d_j), which is to be estimated, and an M×

K word probability matrix Λ

, where Λ

_hl=p(w_si^h=1|t_si=l), which is to be estimated, and

wherein each document d_s has a set of citations Q_ds, further comprising constructing a matrix S to denote direct relationships among the documents wherein S_ls=1/|Q_ds for d_lε

Q_ds and 0 otherwise, where |Q_ds| denotes the size of the set Q_ds, and employing a generative process for generating a related document c from the respective document d_s, comprising;

setting l=s;

choosing t˜

Bernoulli(β

);

if t=1, choosing h˜

Multinomial(S_.,l), where S_.,l denotes the l-th column;

setting

l=h, and returning to said choosing step; and

if t=0, letting c=d_l,

to thereby combine a Bernoulli process and a random walk on a directed graph together, where a transitive property of the citations is captured, wherein the parameter β

of the Bernoulli process determines a probability that the random walk stops at a current node, and the parameter β

also specifies how much of the content of the respective document is influenced from the direct or indirect citations.

16. The method according to claim 11, wherein the joint probability distribution is estimated by an iterative process at a citation level, comprising, for each document d_j, and for the i-th location in document d_j, choosing a topic z_ji from the topic distribution of document d_j, p(z|d_j,θ

_dj), where the distribution parameter θ

_dj is drawn from a Dirichlet distribution Dir(α

), choosing a word w_ji which follows the multinomial distribution p(w|z_ji, Λ

) conditioned on the topic z_ji, and respectively incrementing the locations and documents,

wherein generative processes for the set of documents leads to a joint probability distribution

17. The method according to claim 16, wherein a joint distribution of c,z,θ

of a generative process for the set of documents is represented as;

α

→

θ

→

z→

w_|c;

d→

c→

t→

w_|d|

θ

→

t

Ξ

→

c

and iterative update rules for the iterative process comprise;

18. A non-transitory computer readable medium storing instructions for controlling a programmable processor to characterize a set of documents, to perform a method comprising:

for each respective document in a multilevel hierarchy of documents having direct references and indirect references, having citation linkages with semantic significance, determining latent topic characteristics based on an intrinsic semantic content of the respective document, semantic content associated with directly cited documents, and semantic content associated with documents referenced by directly cited documents, wherein a semantic content significance of a citation has a transitive property;

representing latent topics for documents within the set of documents based on a joint probability distribution of the latent topic characteristics, wherein distinctions in the multilevel hierarchical network are captured by a random process; and

storing the represented set of latent topics.

19. The non-transitory computer readable medium according to claim 18, wherein relationships among the set of documents are modeled by a Bernoulli random process such that a topic distribution of each respective document is a mixture of distributions associated with the linked documents.

20. The non-transitory computer readable medium according to claim 18, wherein the set of documents is modeled by a generative probabilistic model of a topic content of each document of the set of documents along with the linkages among members of the set of documents.