Method and system for classifying or clustering one item into multiple categories

• US 7,469,246 B1
• Filed: 05/18/2001
• Issued: 12/23/2008
• Est. Priority Date: 05/18/2001
• Status: Expired due to Fees

• Current Assignee: Micro Focus LLC (Open Text Corporation)
• Original Assignee: Stratify, Inc. (Open Text Corporation)
• Inventors: Lamping, John
• Primary Examiner(s): Le; Debbie
• Application Number:
  US09/860,646
• Time in Patent Office: 2,776 Days
• Field of Search: 707/1, 707 3- 6, 707/10, 707/100, 707/102, 707/104.1, 382/168, 382/181, 382224-229, 382236-240, 382/170, 704/200, 704236-240
• US Class Current: 1/1
• CPC Class Codes: G06F 16/353 into predefined classes
• CPC Class Codes: Y10S 707/99936 Pattern matching access
• CPC Class Codes: Y10S 707/99937 Sorting

Abstract

A method (and system) for associating an item with only less than ten categories from a collection of more than hundred categories. Each category includes a model, which determines a process. The process determines a degree of closeness to a set of features of items. The method includes receiving an item (e.g., document); and determining a set of features (e.g., word, number, author) associated with the item. The method also uses the set of features with two or more processes in respective categories out of the collection of more than one hundred categories to determine how well the set of features matches the two or more categories. A step of forming a blended model from two or more categories to derive a blended process is included. The blended process yields a closer fit to the set of features of the item than that of any single process of a category from the collection of categories. The method associates the item to each of the two or more categories that were used to form the blended process.

Claims

1. A method for associating an item with at least one category selected from a collection of more than one hundred categories, the method comprising:

providing a collection of more than one hundred categories, each category including a model, each model determining a process, each process determining a degree of closeness to a set of features of items;

receiving an item;

determining a set of features associated with the item;

identifying two or more categories in the collection of more than one hundred categories that relate to the set of features, the step of identifying comprising using the set of features with a process of each category in the two or more categories to determine how well the set of features matches said each category;

forming a blended model from the identified two or more categories to derive a blended process such that the blended process yields a closer fit to the set of features of the item than that of any single process of a category from the collection of categories; and

associating the item to each of the two or more categories that were used to form the blended process, wherein the item becomes associated with fewer than ten of the categories.

2. The method of claim 1 wherein the item is from a plurality of documents, the set of features of the item being a set of the words or stems of words and frequency information.

3. The method of claim 1 wherein the model for each category is a Bayesian statistical model that predicts a likelihood for a feature, and wherein forming a blended model includes averaging the likelihoods each model predicts for the feature.

4. The method of claim 1 wherein identifying two or more categories further comprises:

identifying a first category from the collection of categories whose model best matches the set of features of the item;

determining a remainder set of features from the set of features of the item that emphasizes the reminder set of features, the remainder set of features having a characteristic that is not well explained by the model for the first category; and

identifying a second category from the collection of categories whose model best matches the remainder set of features.

5. The method of claim 1 wherein the method is provided in a clustering algorithm, the categories being defined as clusters.

6. The method of claim 1 wherein the process of using the set of features to determine how well the set of features matches said each category is selected from a model selected from a Bayesian statistical model or a Euclidean distance model.

7. The method of claim 1 wherein the set of features includes one or more of elements including word, number, author, date of authorship, and readers.

8. The method of claim 1 wherein the item is selected from a document, a profile, a history information, art and literature, credit report, Web pages, technical and/or scientific papers, legal documents, and patent documents.

9. The method of claim 1 wherein the method is free from combinatorial explosion.

10. A method for associating an item with at least one category selected from a collection of more than one hundred categories, the method comprising:

providing a collection of more than one hundred categories, each category including a model, each model determining a process, each process determining a degree of closeness to a set of features of items;

receiving an item;

determining a set of features associated with the item;

identifying two or more categories in the collection of more than one hundred categories that relate to the set of features, the step of identifying comprising;

identifying a first category from the collection of categories whose model best matches the set of features of the item; and

determining a remainder set of features from the set of features of the item, the remainder set of features having a characteristic that is not well explained by the model for the first category; and

identifying a second category from the collection of categories whose model best matches the remainder set of features;

forming a blended model from the identified first and second categories to derive a blended process such that the blended process yields a closer fit to the set of features of the item than that of any single process of a category from the collection of categories; and

associating the item to each of the two or more categories that were used to form the blended process, wherein the item becomes associated with fewer than ten of the categories.

11. A system for associating an item with at least one category selected from a collection of more than one hundred categories, the system having computer memory or memories that are distributed, the memory or memories comprising:

stored data representing a collection of more than one hundred categories, the stored data for each category including a model, each model determining a process, each process determining a degree of closeness to a set of features of items;

a first code directed to receive electronic information associated with an item;

a second code directed to determine a set of features associated with the information from the item;

a third code directed to identify two or more categories in the collection of more than one hundred categories that relate to the set of features, the step of identifying comprising using the set of features with a process of each category in the two or more categories to determine how well the set of features matches said each category;

a fourth code directed to form a blended model from the identified two or more categories to derive a blended process such that the blended process yields a closer fit to the set of features of the information of the item than that of any single process of a category from the collection of categories; and

a fifth code directed to associate the item to each of the two or more categories that were used to form the blended process, wherein the item becomes associated with fewer than ten of the categories.

12. The system of claim 11 wherein the memory or memories are provided on a client device.

13. The system of claim 11 wherein the memory or memories are coupled to a microprocessing device, the microprocessing device having at least a clock speed of about 600 MHz and greater.

14. The system of claim 11 wherein the system is substantially free from combinatorial explosion.

15. The system of claim 11 further comprising one or more codes directed to identifying a first category from the collection of categories whose model best matches the set of features of the item and to determining a remainder set of features from the set of features from the item that emphasizes the reminder set of features, the remainder set of features having a characteristic that is not well explained by the model for the first category;

and one or more codes directed to identifying a second category from the collection of categories whose model best matches the remainder set of features.

16. The system of claim 11 comprises a processing device coupled to the one or more memories.

17. The system of claim 16 wherein the processing device comprises a clock speed of more than 600 MHz.

18. The system of claim 17 wherein the blended process has characteristics of processes of the identified two or more categories.

19. The system of claim 18 further comprising a sixth code directed to outputting the blended process.

20. The system of claim 17 wherein the two or more categories are less than ten categories to prevent combinatorial explosion.