Models for classifying documents

• US 9,760,634 B1
• Filed: 04/30/2010
• Issued: 09/12/2017
• Est. Priority Date: 03/23/2010
• Status: Active Grant

• Current Assignee: Aurea Software Incorporated (ESW Capital, LLC)
• Original Assignee: FirstRain, Inc. (Ignite Technologies Incorporated)
• Inventors: Joshi, Ashutosh, Betz, Martin, Arora, Rajiv, Srivastava, Rakesh Kumar, Cooke, David
• Primary Examiner(s): Gurmu, Muluemebet
• Application Number:
  US12/772,168
• Time in Patent Office: 2,692 Days
• Field of Search: 707E17089, 707706
• CPC Class Codes: G06F 16/353 into predefined classes
• CPC Class Codes: G06F 16/951 Indexing; Web crawling techniques
• CPC Class Codes: G06F 7/08 Sorting, i.e. grouping record carriers in numerical or other ordered sequence according to the classification of at least some of the information they carry by merging two or more sets of carriers in ordered sequence G06F7/16

Abstract

Some embodiments provide a method for defining a content relevance model for determining whether a content segment is relevant to a particular category. The method receives a first set of content segments that contain content relevant to the particular category and a second set of content segments that contain content not relevant to the particular category. The method identifies a set of key word sets more likely to appear in the first set of content segments than the second set of content segments. The method defines a content relevance model that comprises a set of groups of word sets and a score for each group, each of the groups of word sets comprising a key word set from the set of key word sets and at least one word set found in a context of the key word set in at least one of the received content segments.

Claims

1. A method for defining a content relevance model for a particular category, the content relevance model for determining whether a content segment is relevant to the particular category, the method comprising:

receiving a first set of content segments that contain content previously determined to be relevant to the particular category and a second set of content segments that contain content previously determined to be not relevant to the particular category;

identifying a set of key word sets that appear more frequently in the first set of content segments than the second set of content segments; and

defining a content relevance model that comprises a set of groups of word sets and a score for each group, each of the groups of word sets comprising a key word set from the identified set of key word sets and at least one word set found in a context of the key word set in at least one of the received content segments, the content relevance model for scoring new content segments that are different from the content segments of the first and second sets of content segments, in order to determine relevance of the new content segments to the particular category.

2. The method of claim 1, wherein the content segments comprise text documents.

3. The method of claim 1, wherein the particular category comprises one of a company, product, person, industry, or concept.

4. The method of claim 1, wherein identifying the set of key word sets comprises:

calculating a score for each word set that appears in at least one content segment of the first and second sets of content segments; and

identifying a plurality of word sets with the highest scores as the key word sets.

5. The method of claim 4, wherein calculating a score for a particular word set comprises comparing a probability of finding the particular word set in the first set of content segments and a probability of finding the particular word set in the second set of content segments.

6. The method of claim 5, wherein calculating the score for the particular word set further comprises accounting for a number of occurrences of the particular word set in the first set of content segments.

7. The method of claim 1, wherein defining the content relevance model comprises:

identifying the set of groups of word sets;

calculating a score for each group of word sets; and

storing (i) the set of groups of word sets (ii), the calculated scores, and (iii) a set of model parameters in the content relevance model.

8. The method of claim 7, wherein the content relevance model is stored as a text file.

9. The method of claim 1, wherein the groups of word sets are pairs of word sets.

10. A method for defining a content relevance model for a particular category, the method comprising:

identifying a set of key word sets for the particular category based on an analysis of (i) a first set of content segments previously defined as relevant to the particular category and (ii) a second set of content segments previously defined as not relevant to the particular category;

identifying (i) a set of pairs of word sets that each comprise a key word set and a word set that appears in a defined context of the key word set and (ii) a score for each of the word set pairs, the score for a particular word set pair quantifying a likelihood that a content segment containing the particular word set pair is relevant to the particular category, wherein appearances of the particular word set pair in the first set of content segments increase the score for the particular word set pair and appearances of the particular word set pair in the second set of content segments decrease the score for the particular word set pair; and

defining a content relevance model for the particular category, the content relevance model comprising (i) a context definition that indicates when a second word set appears within a context of a key word set and (ii) the set of word set pairs and corresponding scores.

11. The method of claim 10, wherein the defined context of a key word set is a context defined for the content relevance model.

12. The method of claim 10, wherein a word set appears in the defined context of a key word set when the word set is within a particular number of words surrounding the key word set in a content segment.

13. The method of claim 10, wherein a word set appears in the defined context of a key word set when the word set is in the same sentence as the key word set.

14. The method of claim 10, wherein a word set appears in the defined context of a key word set when the word set is in the same paragraph as the key word set.

15. The method of claim 10, wherein determining a score for the particular word set pair comprises comparing a function calculated for the particular word set pair in the first set of content segments to the same function calculated for the particular word set pair in the second set of content segments.

16. The method of claim 15, wherein the particular word set pair comprises a particular key word set, wherein calculating the function for the particular word set pair in a particular set of content segments comprises:

comparing a number of occurrences of the particular word set pair in the particular set of content segments to a number of occurrences of the particular key word set in the particular set of content segments; and

comparing a number of content segments in the particular set of content segments in which the particular word set pair appears to a number of content segments in the particular set of content segments in which the particular key word set appears.

17. A non-transitory computer readable medium storing an application which when executed by at least one processor defines a content relevance model for a particular category, the application comprising:

a keyword generator for identifying a set of key word sets for the particular category based on an analysis of (i) a first set of content segments previously defined as relevant to the particular category and (ii) a second set of content segments previously defined as relevant to a set of categories related to the particular category, but not relevant to the particular category, the set of key word sets comprising word sets that appear with greater frequency in the first set of content segments than in the second set of content segments; and

a word pair generator for;

identifying a set of pairs of word sets, each word set pair comprising a key word set identified by the keyword generator and a word set that appears within a defined context of the key word set in at least one content segment from the first and second sets of content segments; and

determining scores for each of the word set pairs by comparing a first number of occurrences of the word set pair in the first set of content segments with a second number of occurrences of the word set pair in the second set of content segments, wherein the score for a particular word set pair is for use in determining the relevancy to the particular category of a new content segment that contains the particular word set pair.

18. The non-transitory computer readable medium of claim 17, wherein the application further comprises a user interface for enabling user input.

19. The non-transitory computer readable medium of claim 18, wherein the user interface is for receiving a user selection of content segments for the first and second sets of content segments.

20. The non-transitory computer readable medium of claim 18, wherein the keyword generator is further for calculating scores for a plurality of potential key word sets and the user interface receives a selection of key word sets by a user.

21. The non-transitory computer readable medium of claim 18, wherein the user input comprises a user definition of the context of the key word sets.

22. The non-transitory computer readable medium of claim 21, wherein the user input further comprises user definitions of a plurality of additional model parameters.

23. The non-transitory computer readable medium of claim 17, wherein the application further comprises a model evaluator for scoring a set of evaluation content segments using the identified set of word pair sets and scores in order to determine whether a model defined with the set of word pair sets and scores accurately determines the relevancy of the evaluation content segments to the particular category.