Knowledge discovery from citation networks

• US 8,630,975 B1
• Filed: 12/02/2011
• Issued: 01/14/2014
• Est. Priority Date: 12/06/2010
• Status: Active Grant

• Current Assignee: The Research Foundation for The State University of New York (State University of New York)
• Original Assignee: The Research Foundation for The State University of New York (State University of New York)
• Inventors: Guo, Zhen, Zhang, Mark
• Primary Examiner(s): HO, BINH VAN
• Application Number:
  US13/310,098
• Time in Patent Office: 774 Days
• Field of Search: 707/608
• US Class Current: 707/608
• CPC Class Codes: G06F 16/24573 using data annotations, e.g. user-defined metadata
• CPC Class Codes: G06F 16/24578 using ranking
• CPC Class Codes: G06F 16/93 Document management systems
• CPC Class Codes: G06F 16/9558 Details of hyperlinks; Management of linked annotations
• CPC Class Codes: G06N 20/00 Machine learning
• CPC Class Codes: G06N 7/01 Probabilistic graphical models, e.g. probabilistic networks
• CPC Class Codes: G06Q 10/10 Office automation; Time management

Abstract

In a corpus of scientific articles such as a digital library, documents are connected by citations and one document plays two different roles in the corpus: document itself and a citation of other documents. A Bernoulli Process Topic (BPT) model is provided which models the corpus at two levels: document level and citation level. In the BPT model, each document has two different representations in the latent topic space associated with its roles. Moreover, the multi-level hierarchical structure of the citation network is captured by a generative process involving a Bernoulli process. The distribution parameters of the BPT model are estimated by a variational approximation approach.

Claims

1. A method for characterizing a corpus of documents each having one or more references, comprising:

identifying a network of multilevel hierarchically related documents having direct references, and indirect references, wherein the references are associated with content relationships;

for each respective document, determining a first set of latent topic characteristics based on an intrinsic content of the respective document;

for each document, determining a second set of latent topic characteristics based on a respective content of other documents which are referenced directly and indirectly through at least one other document to the respective document, the indirectly referenced documents contributing transitively to the latent topic characteristics of the respective document;

representing a set of latent topics for the respective document based on a joint probability distribution of at least the first and second sets of latent topic characteristics, dependent on the identified network, wherein the contributions of at least the second set of latent topic characteristics are determined by an iterative process, wherein the represented set of latent topics is modeled at both a document level and a reference level, by differentiating the two different levels and the multilevel hierarchical network which is captured by a Bernoulli random process; and

storing, in a memory, the represented set of latent topics for the respective document.

2. The method according to claim 1, wherein the network comprises a Bayesian network structure.

3. The method according to claim 1, wherein relationships among the documents are modeled by a Bernoulli process such that a topic distribution of each respective document is a mixture of distributions associated with the related documents.

4. The method according to claim 1, wherein the corpus of documents is modeled by a generative probabilistic model of a topic content of a corpus along with the references among the documents.

5. The method according to claim 1, wherein the iterative process at a reference level comprises iterating, for each document d_j, for the i-th location in document d_j, choosing a topic z_ji from the topic distribution of document d_j, p(z|d_j,θ

_dj), where the distribution parameter θ

_dj is drawn from a Dirichlet distribution Dir(α

), choosing a word w_ji which follows the multinomial distribution p(w|z_ji,Λ

) conditioned on the topic z_ji, and incrementing the locations and documents.

6. The method according to claim 5, wherein the iterative process at a document level comprises iterating, for each document d_s, for the i-th location in document d_s, choosing a referenced document c_si from p(c|d_s,Ξ

), a multinomial distribution conditioned on the document d_s, choosing a topic t_si from the topic distribution of the document c_si at the reference level, and choosing a word w_si which follows the multinomial distribution p(w|t_si,Λ

) conditioned on the topic t_si, where Ξ

is a mixing coefficient matrix which represents how much of the content of the respective document is from direct or indirect references, and a composition of Ξ and

θ

models the topic distribution at the document level, and incrementing the locations and documents.

7. The method according to claim 6, wherein a number of latent topics is K and the mixing coefficients are parameterized by an N×

N matrix Ξ

where Ξ

_js=p(c_si=d_j|d_s), which are treated as a fixed quantity computed from the reference information of the corpus.

8. The method according to claim 7, wherein topic distributions at the reference level are parameterized by a K×

N matrix Θ

where Θ

=p(z_ji=l|d_j), which is to be estimated, and an M×

K word probability matrix Λ

, where Λ

_hl=p(w_si^h=1|t_si=l), which is to be estimated.

9. The method according to claim 8, wherein the references comprise citations, each document d_s having a set of citations Q_ds, further comprising constructing a matrix S to denote direct relationships among the documents wherein

10. The method according to claim 1, wherein a generative process for the corpus leads to a joint probability distribution

11. The method according to claim 1, wherein a generative process for the corpus leads to a joint distribution of c, z, θ

represented as;

α

→

θ

→

z→

w_|c

d→

c→

t→

w_|d|

θ

→

t

Ξ

→

c

w_|c|←

Λ

→

w_|z|

and update rules for the iterative process comprise;

12. The method according to claim 11, wherein the iterative update rules

13. The method according to claim 11, wherein the iterative update rules

14. A method for characterizing a corpus of documents each having one or more citation linkages, comprising:

identifying a multilevel hierarchy of linked documents having direct references, and indirect references, wherein the citation linkages have semantic significance;

for each respective document, determining latent topic characteristics based on an intrinsic semantic content of the respective document, semantic content associated with directly cited documents, and semantic content associated with documents referenced by directly cited documents, wherein a semantic content significance of a citation has a transitive property;

representing latent topics for documents within the corpus based on a joint probability distribution of the latent topic characteristics, wherein the latent topics are modeled at both a document level and a citation level, and distinctions in the multilevel hierarchical network are captured by a Bernoulli random process; and

storing, in a memory, the represented set of latent topics.

15. The method according to claim 14, wherein relationships among the corpus of documents are modeled by a Bernoulli process such that a topic distribution of each respective document is a mixture of distributions associated with the linked documents.

16. The method according to claim 14, wherein the corpus of documents is modeled by a generative probabilistic model of a topic content of each document of the corpus of documents along with the linkages among members of the corpus of documents.

17. The method according to claim 14, wherein the joint probability distribution is estimated by an iterative process at a citation level, comprising, for each document d_j, and for the i-th location in document d_j, choosing a topic z_ji from the topic distribution of document d_j, p(z|d_j,θ

_dj), where the distribution parameter θ

_dj is drawn from a Dirichlet distribution Dir(α

), choosing a word w_ji which follows the multinomial distribution p(w|z_ji,Λ

) conditioned on the topic z_ji, and respectively incrementing the locations and documents.

18. The method according to claim 17, wherein the joint probability distribution is estimated by an iterative process at a document level comprising, for each document d_s, and for the i-th location in document d_s, choosing a cited document c_si from p(c|d_s,Ξ

), a multinomial distribution conditioned on the document d_s, choosing a topic t_si from the topic distribution of the document c_si at the citation level, and choosing a word w_si which follows the multinomial distribution p(w|t_si,Λ

) conditioned on the topic t_si, where Ξ

is a mixing coefficient matrix which represents how much of the content of the respective document is from direct or indirect references, and a composition of Ξ and

θ

models the topic distribution at the document level, and respectively incrementing the locations and documents.

19. The method according to claim 18, wherein a number of latent topics is K and the mixing coefficients are parameterized by an N×

N matrix Ξ

where Ξ

_js=p(c_si=d_j|d_s), which are treated as a fixed quantity computed from the citation information of the corpus of documents,

wherein topic distributions at the citation level are parameterized by a K×

N matrix Θ

where Θ

_lj=p(z_ji=l|d_j), which is to be estimated, and an M×

K word probability matrix Λ

, where Λ

_hl=p(w_si^h=1|t_si=l), which is to be estimated,

wherein each document d_s has a set of citations Q_ds, further comprising constructing a matrix S to denote direct relationships among the documents wherein

20. The method according to claim 17, wherein generative processes for the corpus lead to a joint probability distribution

21. The method according to claim 20, a joint distribution of c, z, θ

of a generative process for the corpus is represented as;

β

→

θ

→

z→

w_|c;

d→

c→

t→

w_|d|

θ

→

t

Ξ

→

c

w_|c|←

Λ

→

w_|z|

and iterative update rules for the iterative process comprise;

22. The method according to claim 21, wherein at least the iterative update rules