System And Method For Performing Discovery Of Digital Information In A Subject Area

• US 20090099996A1
• Filed: 08/12/2008
• Published: 04/16/2009
• Est. Priority Date: 10/12/2007
• Status: Active Grant

• Current Assignee: Xerox Corporation (Xerox Holdings Corp.)
• Original Assignee: Palo Alto Research Center, Inc. (Xerox Holdings Corp.)
• Inventors: Stefik, Mark Jeffrey
• Granted Patent: US 8,165,985 B2
• Time in Patent Office: Days
• US Class Current: 706/54
• CPC Class Codes: G06F 16/3326 using relevance feedback from the user, e.g. relevance feedback on documents, documents sets, document terms or passages
• CPC Class Codes: G06F 16/355 Class or cluster creation or modification
• CPC Class Codes: G06N 5/022 Knowledge engineering; Knowledge acquisition
• CPC Class Codes: Y10S 707/917 Text

Abstract

A system and method for performing discovery of digital information in a subject area is provided. Each of topics in a subject area, training material for the topics, and a corpus comprising digital information are designated. Topic models for each of the topics are built. The topic models are evaluated against the training material. The digital information from the corpus is organized by the topics using the topic models into an evergreen index.

Claims

1. A system for performing discovery of digital information in a subject area, comprising:

an information collection configured to designate each of topics in a subject area, training material for the topics, and a corpus comprising digital information;

an index trainer configured to build topic models for each of the topics, and to evaluate the topic models against the training material; and

an index extrapolator configured to organize the digital information from the corpus by the topics using the topic models into an evergreen index.

2. A system according to claim 1, wherein the topic models are evaluated on at least one of performance, simplicity, and biasing.

3. A system according to claim 1, further comprising:

a topic modeler configured to form each topic model as a predicate in a finite state language, and to apply each predicate to the corpus as a query that returns those places in the corpus matched by the predicate.

4. A system according to claim 1, further comprising:

a classifier configured to classify articles in the corpus against the topic models in the evergreen index; and

a display configured to present the classified articles in the evergreen index.

5. A system according to claim 1, wherein one or more subtopics under at least one of the topics are hierarchically structured.

6. A system according to claim 1, further comprising:

a set of positive training examples comprising articles in the corpus corresponding to correct citations for each topic model.

7. A system according to claim 6, further comprises:

a set of negative training examples comprising articles in the corpus corresponding to incorrect citations for each topic model

8. A system according to claim 6, further comprising:

a seed word selector configured to identify basis words that are characteristic of each topic and based on the articles in the positive training examples set, and to combine one or more of the basis words as seed words into each of the topic models for the topic.

9. A system according to claim 8, wherein additional words are included with the seed words comprising at least one of words in the topic and words proximate to labels in the corpus referenced by the citations of that topic.

10. A system according to claim 1, wherein the digital information n comprises one or more of printed documents, Web pages, and material written in a digital media.

11. A method for performing discovery of digital information in a subject area, comprising:

designating each of topics in a subject area, training material for the topics, and a corpus comprising digital information;

building topic models for each of the topics;

evaluating the topic models against the training material; and

organizing the digital information from the corpus by the topics using the topic models into an evergreen index.

12. A method according to claim 11, further comprising:

evaluating the topic models on at least one of performance, simplicity, and biasing.

13. A method according to claim 11, further comprising:

forming each topic model as a predicate in a finite state language; and

applying each predicate to the corpus as a query that returns those places in the corpus matched by the predicate.

14. A method according to claim 11, further comprising:

classifying articles in the corpus against the topic models in the evergreen index; and

presenting the classified articles in the evergreen index.

15. A method according to claim 11 further comprising:

hierarchically structuring one or more subtopics under at least one of tile topics.

16. A method according to claim 11, further comprising:

defining a set of positive training examples comprising articles in the corpus corresponding to correct citations for each topic model.

17. A method according to claim 16, further comprises:

defining a set of negative training examples comprising articles in the corpus corresponding to incorrect citations for each topic model.

18. A method according to claim 16, further comprising:

identifying basis words that are characteristic of each topic and based on the articles in the positive training examples set; and

combining one or more of the basis words as seed words into each of the topic models for the topic.

19. A method according to claim 18, further comprising:

including additional words with the seed words comprising at least one of words in the topic and words proximate to labels in the corpus referenced by the citations of that topic.

20. A method according to claim 11, wherein the digital information comprises one or more of printed documents, Web pages, and material written in a digital media.

21. An apparatus for performing discovery of digital information in a subject area, comprising:

means for designating each of topics in a subject area, training material for the topics, and a corpus comprising digital information;

means for building topic models for each of the topics;

means for evaluating the topic models against the training material; and

means for organizing the digital information from the corpus by the topics using the topic models into an evergreen index.