System And Method For Providing Multi-Core And Multi-Level Topical Organization In Social Indexes

• US 20110270830A1
• Filed: 04/30/2010
• Published: 11/03/2011
• Est. Priority Date: 04/30/2010
• Status: Active Grant

• Current Assignee: Palo Alto Research Center, Inc. (Xerox Holdings Corp.)
• Original Assignee: Palo Alto Research Center, Inc. (Xerox Holdings Corp.)
• Inventors: Good, Lance E., Mittal, Sanjay, Stefik, Mark Jeffrey
• Granted Patent: US 9,031,944 B2
• Time in Patent Office: Days
• US Class Current: 707/731
• CPC Class Codes: G06F 16/353 into predefined classes

Abstract

A computer-implemented method for providing multi-core and multi-level topical organization in social indexes is provided. A corpus of articles is accessed. Each article includes online textual materials. A finite state pattern is provided for a topic that filters the articles as candidate articles, which are potentially on-topic. Similarity-based representations for on-topic and off-topic core meanings of the topic are provided. An aggregate score for each of the candidate articles is determined using the similarity-based representations to indicate whether the candidate article is sufficiently on-topic. The candidate articles are presented ordered by their aggregate scores. In a further embodiment, a hierarchy of topics is provided and used to guide the presentation of articles from subtopics, with considerations of fairness of subtopic coverage, elimination of similarity-duplicates in articles, and article freshness.

Claims

1. A computer-implemented system for providing multi-core topic indexing in electronically-stored social indexes, comprising:

a storage device comprising a corpus of articles each comprised of online textual materials; and

a social indexing system, comprising;

a finite state pattern for a topic filtering the articles as candidate articles, which are potentially on-topic;

similarity-based representations for on-topic and off-topic core meanings of the topic; and

a finite state modeler determining an aggregate score for each of the candidate articles using the similarity-based representations to indicate whether the candidate article is sufficiently on-topic; and

a display presenting the candidate articles ordered by their aggregate scores.

3. A computer-implemented method according to claim 1, further comprising at least one of:

automatically generating the finite state pattern based on positive and negative training examples;

generating the finite state pattern through evaluation of default patterns;

suggesting the finite state pattern to the user based on an evaluation of the articles in the topic using one of the foregoing approaches; and

designating the finite state pattern as a discrete Boolean query specified by a user.

5. A computer-implemented method according to claim 1, wherein the evaluation of default patterns comprises:

specifying a hierarchically-structured tree of topics, which each comprise a label comprising one or more words;

identifying constraints inherent in the literal structure of the topic tree;

for each topic in the topic tree, creating a candidate topic model that comprises at least one term derived from the words in at least one of the labels;

evaluating the candidate topic models for the topic tree against the constraints; and

identifying the candidate topic model, which best satisfy the constraints, as the finite state pattern for the topic.

6. A computer-implemented method according to claim 1, further comprising:

selecting on-topic training examples and off-topic training examples from the articles in the corpus;

determining a similarity score between each of the new articles and each of the articles in the on-topic training examples and the off-topic training examples; and

evaluating the aggregate score for each of the new articles as the maximum similarity score for the new article as compared with all of the on-topic training examples and the topic center, minus the maximum similarity score for the new article as compared with all of the off-topic training examples.

7. A computer-implemented method according to claim 5, further comprising:

weighting one or more of the on-topic training examples and the off-topic training examples; and

proportioning the similarity scores based on the respective weightings of the on-topic training examples and the off-topic training examples.

8. A computer-implemented method according to claim 5, further comprising:

computing the similarity score by applying a cosine function to weights assigned to characteristic words identified in each of the new articles, the on-topic training examples, and the off-topic training examples.

9. A computer-implemented method according to claim 1, further comprising:

forming an average article for the topic from results of at least one of;

choosing at least one of the on-topic similarity-based representations;

identifying randomly-chosen articles in the corpus matching the finite state pattern;

combining the randomly-chosen articles and articles selected from the corpus that are sufficiently on-topic; and

identifying randomly-chosen articles in the corpus matching a revised finite state pattern that expressly excludes articles selected from the corpus that are sufficiently off-topic.

10. A computer-implemented method according to claim 1, further comprising at least one of:

explicitly defining the core meanings of the topic based on articles selected from the corpus that comprise the on-topic similarity-based representations; and

implicitly defining a core meaning of the topic by;

taking a sample of the articles in the corpus that match the finite state pattern; and

determining the average article to implicitly define a central core meaning for the topic,

wherein the sample is one of random, time-biased, and favoring more recent articles over older articles in the corpus.

11. A computer-implemented method according to claim 1, further comprising at least one of:

presenting the candidate articles without filtering through the finite state topic model; and

presenting only the candidate articles comprised of an aggregate score meeting a minimum threshold.

14. A computer-implemented method according to claim 11, further comprising:

providing a finite state pattern for each topic that filters the articles, which are potentially on-topic; and

presenting only the articles that match the finite state pattern for each topic.

16. A computer-implemented method according to claim 11, further comprising:

applying a cosine function to weights assigned to characteristic words identified in each of the articles in each topic level; and

determining the ordering of the articles in the topic level based on the results of the cosine function.

17. A computer-implemented method according to claim 11, further comprising:

repeatedly forming the articles into clusters for each topic level, wherein a threshold for cluster membership is decreased upon each repetition; and

finalizing the clusters based on one of a pre-defined limit on repetitions of cluster formation and a pre-defined lower bound on the set threshold.

18. A computer-implemented method according to claim 11, further comprising:

selecting articles in the clusters for each topic level and highlighting the selected articles most representative of the cluster.

19. A computer-implemented method according to claim 11, further comprising:

identifying the articles in the clusters comprised of substantially duplicative content;

identifying the articles in the clusters posted online substantially simultaneously for a common event; and

selecting one of the articles as representative of the remaining articles.

20. A computer-implemented method according to claim 18, further comprising:

providing a link to the group of remaining articles instead of presenting the remaining articles.

21. A computer-implemented method according to claim 11, further comprising:

choosing the articles to be presented based on at least one of timeliness, topic “

hotness,” and

fairness of coverage across subtopics.

2. A computer-implemented method for providing multi-core topic indexing in electronically-stored social indexes, comprising:

accessing a corpus of articles each comprised of online textual materials;

providing a finite state pattern for a topic that filters the articles as candidate articles, which are potentially on-topic;

providing similarity-based representations for on-topic and off-topic core meanings of the topic;

determining an aggregate score for each of the candidate articles using the similarity-based representations to indicate whether the candidate article is sufficiently on-topic; and

presenting the candidate articles ordered by their aggregate scores.

4. A computer-implemented method according to claim 2, wherein the automatic pattern generation comprises:

selecting positive training examples from the articles in the corpus;

selecting negative training examples from the articles in the corpus;

defining the finite state pattern to match as many of the positive training examples as possible and to not match as many of the negative training examples as possible.

12. A computer-implemented system for providing multi-level topic indexing in electronically-stored social indexes, comprising:

a storage device comprising a corpus of articles each comprised of online textual materials; and

a social indexing system, comprising;

a plurality of levels of topics organized in a hierarchy, wherein each subsequent topic level in the hierarchy comprises one or more subtopics; and

a finite state modeler initially identifying and ordering the articles under the current topic level, and recursively identifying and ordering the remaining articles under the subtopics under each subsequent topic level using criteria selected from the group comprising freshness of the article, balance of topical coverage of the articles across each of the subtopics, and elimination of similarity-duplicates of the articles under each of the subtopics; and

a display presenting the articles.

15. A computer-implemented method according to claim 12, further comprising:

providing similarity-based representations for on-topic and off-topic core meanings of each topic;

determining an aggregate score for each of the filtered articles using the similarity-based representations to indicate whether the filtered article is sufficiently on-topic; and

presenting only the filtered articles ordered by their aggregate scores.

13. A computer-implemented method for providing multi-level topic indexing in electronically-stored social indexes, comprising:

accessing a corpus of articles each comprised of online textual materials;

defining a plurality of levels of topics organized in a hierarchy, wherein each subsequent topic level in the hierarchy comprises one or more subtopics;

initially identifying and ordering the articles under the current topic level;

recursively identifying and ordering the remaining articles under the subtopics under each subsequent topic level using criteria selected from the group comprising freshness of the article, balance of topical coverage of the articles across each of the subtopics, and elimination of similarity-duplicates of the articles under each of the subtopics; and

presenting the articles.