System and method for facilitating evergreen discovery of digital information

• US 8,706,678 B2
• Filed: 04/23/2012
• Issued: 04/22/2014
• Est. Priority Date: 10/12/2007
• Status: Active Grant

• Current Assignee: Xerox Corporation (Xerox Holdings Corp.)
• Original Assignee: Palo Alto Research Center, Inc. (Xerox Holdings Corp.)
• Inventors: Stefik, Mark Jeffrey
• Primary Examiner(s): Chaki, Kakali
• Assistant Examiner(s): SITIRICHE, LUIS A
• Application Number:
  US13/453,984
• Publication Number:
  US 20120209797A1
• Time in Patent Office: 729 Days
• Field of Search: None
• US Class Current: 706/62
• CPC Class Codes: G06F 16/3326 using relevance feedback from the user, e.g. relevance feedback on documents, documents sets, document terms or passages
• CPC Class Codes: G06F 16/355 Class or cluster creation or modification
• CPC Class Codes: G06N 5/022 Knowledge engineering; Knowledge acquisition
• CPC Class Codes: Y10S 707/917 Text

Abstract

A computer-implemented system and method for facilitating evergreen discovery of digital information is provided. A hierarchy of topics for topically-limited subject areas is defined. Seed words characteristic of each topic are selected. Training material from the digital information that corresponds to the respective subject area of each of the topics is designated. Candidate topic models are formed from the seed words. Each candidate topic model includes a pattern evaluable against the digital information. An ability of each of the candidate topic models to identify such digital information matching the candidate topic model'"'"'s topic is tested by matching the pattern in the candidate topic model to the training material. The candidate topic model for each topic that includes the highest abilities with respect to the topic in performance, simplicity and bias is chosen. An evergreen index is formed by pairing the chosen candidate topic model to each topic in the hierarchy.

Claims

1. A computer-implemented system for facilitating evergreen discovery of digital information, comprising:

a hierarchy of topics for topically-limited subject areas, each of the subject areas comprising pages of electronically-stored digital information maintained in a storage device;

a computer comprising a processor and memory within which code for execution by the processor is stored, comprising;

a user interface of the computer configured to select seed words that are characteristic of each of the topics and to designate training material from the digital information that corresponds to the respective subject area of each of the topics;

a topic modeler configured to form candidate topic models from the seed words, each candidate topic model comprising a pattern evaluable against the digital information;

a topic tester configured to test an ability of each of the candidate topic models to identify such digital information matching the candidate topic model'"'"'s topic by matching the pattern in the candidate topic model to the training material;

a topic rater configured to rate the respective abilities of the candidate topic models, comprising;

a performance rater configured to rank each candidate topic model'"'"'s performance in matching the training material correctly for the corresponding topic;

a simplicity rater configured to prefer those candidate topic models with simpler patterns over the patterns of other candidate topic models that correctly match the same training material; and

a bias rater configured to assign a bias to those candidate topic models that comprise terms also found in the corresponding topic;

a topic model selector configured to choose the candidate topic model for each topic that comprises the highest abilities with respect to the topic in performance, simplicity and bias; and

an index builder configured to form an evergreen index by pairing the chosen candidate topic model to each topic in the hierarchy.

2. A system according to claim 1, further comprising:

a finite state modeler configured to specify each pattern as a predicate in a finite state language that comprise one or more predicates, terms and levels, and to apply each predicate to the digital information as a query that returns those places in the digital information matched by the predicate.

3. A system according to claim 2, further comprising:

a pattern evaluator configured to evaluate the simplicity of the pattern of each candidate topic model in terms of structural complexity as a measure of a number and complexity of the candidate topic model'"'"'s predicates, terms and levels, and to choose the candidate topic model with a lowest structural complexity measure as the candidate topic model comprising the simplest pattern.

4. A system according to claim 2, further comprising:

a predicate builder configured to define the predicate through one or more of single-word candidates, n-grams, disjunction patterns, conjunction patterns, sequence patterns, and second level disjunction patterns; and

an interpreter configured to interpret the predicate against the digital information based on the finite state pattern language.

5. A system according to claim 1, further comprising:

the user interface further configured to designate part of the training material as on-topic training material for a corresponding topic and part of the training material as off-topic training material for the corresponding topic; and

the performance rater further configured to increase the rating of the performance of each candidate topic model that matches the on-topic material for the corresponding topic, to decrease the rating of the performance of each candidate topic model that does not match the on-topic material for the corresponding topic, to increase the rating of the performance of each candidate topic model that does not match the off-topic material for the corresponding topic, and to decrease the rating of the performance of each candidate topic model that matches the off-topic material for the corresponding topic.

6. A system according to claim 5, further comprising:

the performance rater still further configured to increase the rating of the performance of each candidate topic model that matches all the on-topic material for the corresponding topic and that does not match all any of the off-topic material for the corresponding topic.

7. A system according to claim 1, further comprising:

a generate-and-test module configured to successively perform the forming, testing and rating of each of the candidate topic models for a particular topic as a generate-and-test procedure, and choose the candidate topic model that comprises the highest abilities with respect to the topic in performance, simplicity and bias with respect to only those candidate topic models already generated and tested.

8. A system according to claim 7, further comprising:

the user interface further configured to specify characteristics of a best possible topic model for the particular topic;

the simplicity rater further configured to evaluate each chosen candidate topic model against the characteristics of the best possible topic model; and

the generate-and-test module further configured to forego the generation and testing of further candidate topic models for the particular topic.

9. A system according to claim 1, further comprising:

a pattern matcher configured to match the pattern in the chosen candidate topic model for each topic against the digital information, and to register each page of the digital information that correctly matches at least one pattern under the topic corresponding to the correctly matched pattern'"'"'s chosen candidate topic model.

10. A system according to claim 9, further comprising:

the pattern matcher further configured to identify one or more pages of the digital information that do not match at least one pattern as on-topic training material for a new topic, and to append the topic into the hierarchy of topics.

11. A system according to claim 9, further comprising:

the pattern matcher further configured to identify one or more pages of the digital information that incorrectly matches at least one pattern as off-topic training material for the topic corresponding to the incorrectly matched pattern'"'"'s chosen candidate topic model.

12. A computer-implemented method for facilitating evergreen discovery of digital information, comprising the steps of:

defining a hierarchy of topics for topically-limited subject areas, each of the subject areas comprising pages of electronically-stored digital information;

selecting seed words that are characteristic of each of the topics;

designating training material from the digital information that corresponds to the respective subject area of each of the topics;

forming candidate topic models from the seed words, each candidate topic model comprising a pattern evaluable against the digital information;

testing an ability of each of the candidate topic models to identify such digital information matching the candidate topic model'"'"'s topic by matching the pattern in the candidate topic model to the training material;

rating the respective abilities of the candidate topic models, comprising;

ranking each candidate topic model'"'"'s performance in matching the training material correctly for the corresponding topic;

preferring those candidate topic models with simpler patterns over the patterns of other candidate topic models that correctly match the same training material; and

assigning a bias to those candidate topic models that comprise terms also found in the corresponding topic;

choosing the candidate topic model for each topic that comprises the highest abilities with respect to the topic in performance, simplicity and bias; and

forming an evergreen index by pairing the chosen candidate topic model to each topic in the hierarchy.

13. A method according to claim 12, further comprising the steps of:

specifying each pattern as a predicate in a finite state language that comprise one or more predicates, terms and levels; and

applying each predicate to the digital information as a query that returns those places in the digital information matched by the predicate.

14. A method according to claim 13, further comprising the steps of:

evaluating the simplicity of the pattern of each candidate topic model in terms of structural complexity as a measure of a number and complexity of the candidate topic model'"'"'s predicates, terms and levels; and

choosing the candidate topic model with a lowest structural complexity measure as the candidate topic model comprising the simplest pattern.

15. A method according to claim 13, further comprising the steps of:

defining the predicate through one or more of single-word candidates, n-grams, disjunction patterns, conjunction patterns, sequence patterns, and second level disjunction patterns; and

interpreting the predicate against the digital information based on the finite state pattern language.

16. A method according to claim 12, further comprising the steps of:

designating part of the training material as on-topic training material for a corresponding topic and part of the training material as off-topic training material for the corresponding topic;

increasing the rating of the performance of each candidate topic model that matches the on-topic material for the corresponding topic;

decreasing the rating of the performance of each candidate topic model that does not match the on-topic material for the corresponding topic;

increasing the rating of the performance of each candidate topic model that does not match the off-topic material for the corresponding topic;

decreasing the rating of the performance of each candidate topic model that matches the off-topic material for the corresponding topic.

17. A method according to claim 16, further comprising the steps of:

further increasing the rating of the performance of each candidate topic model that matches all the on-topic material for the corresponding topic and that does not match all any of the off-topic material for the corresponding topic.

18. A method according to claim 12, further comprising the steps of:

successively performing the forming, testing and rating of each of the candidate topic models for a particular topic as a generate-and-test procedure; and

choosing the candidate topic model that comprises the highest abilities with respect to the topic in performance, simplicity and bias with respect to only those candidate topic models already generated and tested.

19. A method according to claim 18, further comprising the steps of:

specifying characteristics of a best possible topic model for the particular topic;

evaluating each chosen candidate topic model against the characteristics of the best possible topic model; and

foregoing the generation and testing of further candidate topic models for the particular topic.

20. A method according to claim 12, further comprising the steps of:

matching the pattern in the chosen candidate topic model for each topic against the digital information; and

registering each page of the digital information that correctly matches at least one pattern under the topic corresponding to the correctly matched pattern'"'"'s chosen candidate topic model.

21. A method according to claim 20, further comprising the steps of:

identifying one or more pages of the digital information that do not match at least one pattern as on-topic training material for a new topic; and

appending the topic into the hierarchy of topics.

22. A method according to claim 20, further comprising the steps of:

identifying one or more pages of the digital information that incorrectly matches at least one pattern as off-topic training material for the topic corresponding to the incorrectly matched pattern'"'"'s chosen candidate topic model.

23. A non-transitory computer readable storage medium storing code for executing on a computer system to perform the method according to claim 12.

24. A computer-implemented apparatus for facilitating evergreen discovery of digital information, comprising:

means for defining a hierarchy of topics for topically-limited subject areas, each of the subject areas comprising pages of electronically-stored digital information;

means for selecting seed words that are characteristic of each of the topics though a user interface of a computer;

means for designating training material from the digital information that corresponds to the respective subject area of each of the topics;

means for forming candidate topic models from the seed words, each candidate topic model comprising a pattern evaluable against the digital information;

means for testing an ability of each of the candidate topic models to identify such digital information matching the candidate topic model'"'"'s topic by means for matching the pattern in the candidate topic model to the training material;

means for rating the respective abilities of the candidate topic models, comprising;

means for ranking each candidate topic model'"'"'s performance in matching the training material correctly for the corresponding topic;

means for preferring those candidate topic models with simpler patterns over the patterns of other candidate topic models that correctly match the same training material; and

means for assigning a bias to those candidate topic models that comprise terms also found in the corresponding topic;

means for choosing the candidate topic model for each topic that comprises the highest abilities with respect to the topic in performance, simplicity and bias; and

means for forming an evergreen index by means for pairing the chosen candidate topic model to each topic in the hierarchy.