GENERATING A CONCEPTUAL ASSOCIATION GRAPH FROM LARGE-SCALE LOOSELY-GROUPED CONTENT

• US 20110113032A1
• Filed: 10/15/2010
• Published: 05/12/2011
• Est. Priority Date: 05/10/2005
• Status: Active Grant

• Current Assignee: NetSeer, Inc. (Inuvo, Inc.)
• Original Assignee: NetSeer, Inc. (Inuvo, Inc.)
• Inventors: REZAEI, BEHNAM ATTARAN, CHERN, ALBERT, ROYCHOWDHURY, VWANI P., KHAJEHNOURI, NIMA, BOSCOLO, RICCARDO
• Granted Patent: US 9,110,985 B2
• Time in Patent Office: Days
• US Class Current: 707/737
• CPC Class Codes: G06F 16/367 Ontology
• CPC Class Codes: G06F 16/958 Organisation or management of web site content, e.g. publishing, maintaining pages or automatic linking

Abstract

A method for generating a conceptual association graph from structured content includes grouping content nodes into one or more topically biased clusters, the content nodes comprising structured digital content and unstructured digital content, the grouping based at least in part on the connectedness of each content node member to other content node members in the same cluster. The method also includes, responsive to the grouping, tagging the content nodes with one or more descriptive concepts. The method also includes, responsive to the tagging, establishing one or more associations between the one or more concepts, the one or more associations indicating a relevance of the one or more associations, the indicating based at least in part on patterns of co-occurrence of concepts in the tagged content nodes.

Claims

1. A computer implemented method comprising:

grouping content nodes into one or more topically biased clusters, the content nodes comprising structured digital content and unstructured digital content, the grouping based at least in part on the connectedness of each content node member to other content node members in the same cluster;

responsive to the grouping, tagging the content nodes with one or more descriptive concepts; and

responsive to the tagging, establishing one or more associations between the one or more concepts, the one or more associations indicating a relevance of the one or more associations, the indicating based at least in part on patterns of co-occurrence of concepts in the tagged content nodes.

2. The method of claim 1 wherein the connectedness of each content node to other content node members is defined by links between each content node and the other content node members.

3. The method of claim 2, wherein the links are generated from metadata entities.

4. The method of claim 3, wherein a content node is connected to other content nodes if the content node and the other content nodes are tagged with the same metadata entity.

5. The method of claim 3, wherein the metadata entity comprises at least one of category tagging and descriptive keywords.

6. The method of claim 2, wherein the links are pre-processed and tagged.

7. The method of claim 6, wherein pre-processing the links comprises classifying the links based on type and assigning the links weights.

8. The method of claim 1, wherein the nodes are grouped using a community finding algorithm.

9. The method of claim 1, wherein tagging the content nodes comprises:

analyzing content of the content nodes in each group and extracting a plurality of collective token and potential concept statistics;

scoring and filtering the statistics based on a measure of relevance to provide a view of the content nodes within a cluster; and

selecting one or more descriptive concepts and keywords for a specific view of a content node.

10. The method of claim 9, wherein concepts are n-grams, i.e., a string of n consecutive words, phonemes or other atomic tokens in the content node.

11. The method of claim 9, wherein the measure of relevance of a concept in a group is determined by a number of statistics, including weighted token frequency, community document frequency and global document frequency.

12. The method of claim 9, wherein the most important concepts in a group are selected to tag the group.

13. The method of claim 12, wherein the most important concepts are the concepts with scores above a certain threshold.

14. The method of claim 1, wherein establishing the one or more associations between the one or more concepts comprises:

pre-processing the tagged content nodes by merging tags for the same content node; and

scoring the one or more associations based on a pattern of appearance of concept pairs within the content nodes.

15. The method of claim 14, further comprising weighing the score by a probability that a content node should be tagged with a concept.

16. The method of claim 14, further comprising weighing the score through a global measure of relevance of the content nodes that are tagged with a concept.

17. The method of claim 14, wherein the score is determined by the positional information of an extracted concept within a concept node.

18. A computer system comprising:

a content grouping module configured to group content nodes into one or more topically biased clusters, the content nodes comprising structured digital content and unstructured digital content, the grouping based at least in part on the connectedness of each content node member to other content node members in the same cluster;

a concept extraction module configured to tag the content nodes with one or more descriptive concepts; and

a conceptual map builder configured to establish one or more associations between the one or more concepts, the one or more associations indicating a relevance of the one or more associations, the indicating based at least in part on patterns of co-occurrence of concepts in the tagged content nodes.

19. The computer system of claim 18 wherein the concept extraction module comprises a text tokenizer to analyze the content of the nodes in each group and extract several collective token statistics.

20. The computer system of claim 18 wherein the conceptual map builder comprises a proximity engine to score concept associations based on a pattern of appearance of concept pairs within content nodes.

22. The computer readable media of claim 18 wherein the nodes are grouped using a community finding algorithm.

23. The computer readable media of claim 18 wherein establishing the one or more associations between the one or more concepts comprises:

pre-processing the tagged content nodes by merging tags for the same content node; and

scoring the one or more associations based on a pattern of appearance of concept pairs within the content nodes.

21. A computer-readable media having computer executable instructions stored thereon which cause a computer system to carry out a method when executed, the method comprising

grouping content nodes into one or more topically biased clusters, the content nodes comprising structured digital content and unstructured digital content, the grouping based at least in part on the connectedness of each content node member to other content node members in the same cluster;

responsive to the grouping, tagging the content nodes with one or more descriptive concepts; and

responsive to the tagging, establishing one or more associations between the one or more concepts, the one or more associations indicating a relevance of the one or more associations, the indicating based at least in part on patterns of co-occurrence of concepts in the tagged content nodes.