System and method for automatically classifying text

• US 7,028,250 B2
• Filed: 05/25/2001
• Issued: 04/11/2006
• Est. Priority Date: 05/25/2000
• Status: Expired due to Term

• Current Assignee: Avolin LLC
• Original Assignee: Kanisa Incorporated (Aptean Superior Holdings, Inc.)
• Inventors: Copperman, Max, Huffman, Scott B., Ukrainczyk, Igor
• Primary Examiner(s): Hong, Stephen
• Assistant Examiner(s): STORK, KYLE R
• Application Number:
  US09/864,156
• Publication Number:
  US 20020022956A1
• Time in Patent Office: 1,782 Days
• Field of Search: 706/42, 706/20, 704/275, 704/232, 704/219, 704/332, 715/500, 715/531, 715/513, 715/417, 715/523, 715/530, 707/5
• US Class Current: 715/202
• CPC Class Codes: G06F 16/313 Selection or weighting of terms for indexing
• CPC Class Codes: G06F 16/353 into predefined classes
• CPC Class Codes: G06F 40/117 Tagging; Marking up details of markup languages G06F40/143; Designating a block; Setting of attributes style sheets, e.g. eXtensible Stylesheet Language Transformation [XSLT], G06F40/154
• CPC Class Codes: G06F 40/20 Natural language analysis semantic analysis of natural language G06F40/30

Abstract

A method is provided for automatically classifying text into categories. In operation, a plurality of tokens or features are manually or automatically associated with each category. A weight is then coupled to each feature, wherein the weight indicates a degree of association between the feature and the category. Next, a document is parsed into a plurality of unique tokens with associated counts, wherein the counts are indicative of the number of times the feature appears in the document. A category score representative of a sum of products of each feature count in the document times the corresponding feature weight in the category for each document is then computed. Next, the category scores are sorted by perspective, and a document is classified into a particular category, provided the category score exceeds a predetermined threshold.

Claims

1. In a system comprising perspectives and categories, each perspective including at least one category representative of that perspective, a computerized method for classifying at least one item across multiple perspectives, said computerized method comprising:

associating category features with each category, wherein each of said category features represents one of a plurality of tokens;

producing a category vector for each category, wherein each category vector includes a weight corresponding to each category feature, said weight indicative of a degree of association between said category feature and said category;

associating item features with each item, wherein each of said item features represents one of a plurality of tokens found in said item;

producing a feature vector for each item, wherein each feature vector includes said item features with a count corresponding to each item feature, said count indicative of the number of times said item feature appears in said item;

multiplying said category vector by said item vector to produce a plurality of category scores for each item; and

for each perspective, across multiple perspectives, classifying an item into a category provided said category score exceeds a predetermined threshold.

2. The computerized method of claim 1, wherein

the count includes at least one subcount indicative of the number of times the item feature appears in a particular region of the item, wherein the particular region of the item is a subset of the item taken as a whole;

and

for first and second categories, in classifying the item into the second category, substituting the at least one subcount for the count of each item feature depends on whether the item was classified into the first category.

3. The computerized method of claim 1, wherein

for first and second categories, classifying the item into the second category depends at least in part on a determination of whether the item was classified into the first category.

4. The computerized method of claim 1, wherein the classifying further comprises:

comparing a category score for a first item in a first perspective with a category score for said first item in a second perspective; and

modifying the category score in the first perspective in response to the category score in the second perspective.

5. The computerized method of claim 4, wherein said first perspective is an ancestor of said second perspective.

6. The computerized method of claim 4, wherein the comparing further comprises:

creating an ordered list of category scores for all categories in all perspectives;

identifying an item associated with a highest category score in a first perspective;

identifying said item in a second perspective; and

decreasing the category score of said item in said second perspective.

7. The computerized method of claim 6, further comprising repeating the identifying and decreasing for every perspective.

8. The computerized method of claim 1, wherein the weight corresponding to said concept node feature is between −

1 and 1.

9. The computerized method of claim 1, in which the multiplying includes using at least one attribute of at least one category vector in determining whether to include a document feature of the feature vector in the multiplying.

10. The computerized method of claim 9, in which the using at least one attribute includes using at least one of:

a Stop attribute to indicate whether a feature must constitute something other than a stop word to be included in the multiplying;

a Case attribute to indicate whether a feature must match a letter case specification to be included in the multiplying;

a Stemming attribute to indicate whether a feature includes stemmed word forms to be included in the multiplying; and

a Learned attribute to indicate how a human-specified feature is to be included in the multiplying.

11. The computerized method of claim 9, in which the perspective to which the category relates determines the value of the at least one attribute.