Method and system for classifying text

• US 20100094875A1
• Filed: 08/11/2009
• Published: 04/15/2010
• Est. Priority Date: 08/11/2008
• Status: Active Grant

• Current Assignee: Zeta Global Corp. (Zeta Global Holdings Corp.)
• Original Assignee: Collective Media Incorporated
• Inventors: Harrison, Paul, Fulton, Hal, Roehrl, Armin, Grace, Brenden, Oliphant, James
• Granted Patent: US 8,762,382 B2
• Time in Patent Office: Days
• US Class Current: 707/741
• CPC Class Codes: G06F 16/355 Class or cluster creation or modification
• CPC Class Codes: G06F 16/367 Ontology

Abstract

A content classification system, method and computer product is presented. In connection with the invention, a data structure is created by identifying a plurality of words and mapping each word to one or more categories. The data structure is indexed. An item of content is identified and classified based on the data structure. The classification includes identifying all one—or more—word combinations in the item of content; for each word of at least a pre-determined number of characters in length in each of the word combinations, identifying each of the categories to which it is mapped; and determining a weight for each of the words based on an inverse proportion to the number of categories to which it is mapped.

Claims

1. A method comprising:

creating a data structure using a data structure generation engine by identifying a plurality of words and mapping each word to one or more categories and storing the data structure in one or more databases;

indexing the data structure using an index generation engine;

identifying an item of content; and

classifying the item of content using a classification engine based on the data structure, the classifying comprising;

identifying all one—

or more—

word combinations in the item of content;

for each word of at least a pre-determined number of characters in length in each of the word combinations, identifying each of the categories to which it is mapped; and

determining a weight for each of the words based on an inverse proportion to the number of categories to which it is mapped.

2. The method of claim 1 wherein the pre-determined number of characters is three.

3. The method of claim 1 wherein determining the weight further comprises assigning increasingly greater weight to word combinations of an increasingly greater number of words in the word combination using a multiplier.

4. The method of claim 1 further comprising:

grouping one or more of the categories into one or more channels using a channel building engine.

5. The method of claim 4 wherein determining the weight further comprising assigning an additional weight based on the one or more channels associated with the one or more categories to which the word is mapped.

6. The method of claim 1 further comprising:

assigning a value to the weight based on a relative relatedness between words using a semantic distance measure.

7. The method of claim 1 wherein the index is derived from existing structures configured in accordance with cognitive human notions of relatedness.

8. The method of claim 1 wherein the index is generated based on counts of one or both of categories and keywords in a page of content.

9. The method of claim 1 wherein the item of content comprises multilingual content and the classification is performed without translating the content.

10. The method of claim 1 wherein content is discovered using one or more crawling engines configured.

11. The method of claim 1 wherein the item of content comprises a web page and the classifying is performed by analyzing only a URL of the web page.

12. The method of claim 1 further comprising:

adding a result of classifying the content to the data structure.

13. The method of claim 1 wherein the item of content comprises a web page and wherein the classification is performed on both the web page as rendered and the language used to create the web page.

14. A system comprising:

an data structure generation engine configured to create a data structure by identifying a plurality of words and mapping each word to one or more categories;

one or more databases that store the data structure;

an index generation engine configured to index the data structure;

a classification engine configured to identify an item of content and classifying the item of content based on the data structure by identifying all one—

or more—

word combinations in the item of content;

for each word of at least a pre-determined number of characters in length in each of the word combinations, identifying each of the categories to which it is mapped; and

determining a weight for each of the words based on an inverse proportion to the number of categories to which it is mapped.

15. The system of claim 14 wherein the pre-determined number of characters is three.

16. The system of claim 14 wherein determining the weight further comprises assigning increasingly greater weight to word combinations of an increasingly greater number of words in the word combination using a multiplier.

17. The system of claim 14 further comprising:

a channel building engine configured to group one or more of the categories into one or more channels.

18. The system of claim 17 wherein determining the weight further comprises assigning an additional weight based on the one or more channels associated with the one or more categories to which the word is mapped.

19. The system of claim 14 wherein the classification engine is further configured to assign a value to the weight based on a relative relatedness between words using a semantic distance measure.

20. The system of claim 14 wherein the index is derived from existing structures configured in accordance with cognitive human notions of relatedness.

21. The system of claim 14 wherein the index is generated based on counts of one or both of categories and keywords in a page of content.

22. The system of claim 14 wherein the item of content comprises multilingual content and the classification is performed without translating the content.

23. The system of claim 14 further comprising:

one or more crawling engines configured to discover content.

24. The system of claim 14 wherein the item of content comprises a web page and the classifying is performed by analyzing only a URL of the web page.

25. The system of claim 14 further wherein a result of classifying the content is added to the data structure.

26. The system of claim 14 wherein the item of content comprises a web page and wherein the classification is performed on both the web page as rendered and the language used to create the web page.

27. A computer program product including a computer readable medium having stored thereon computer executable instructions that, when executed by a computer, direct the computer to perform a method comprising the steps of:

creating a data structure by identifying a plurality of words and mapping each word to one or more categories;

indexing the data structure;

identifying an item of content; and

classifying the item of content based on the data structure, the classifying comprising;

identifying all one—

or more—

word combinations in the item of content;

for each word of at least a pre-determined number of characters in length in each of the word combinations, identifying each of the categories to which it is mapped; and

determining a weight for each of the words based on an inverse proportion to the number of categories to which it is mapped.

28. The computer program product of claim 27 wherein the pre-determined number of characters is three.

29. The computer program product of claim 27 wherein determining the weight further comprises assigning increasingly greater weight to word combinations of an increasingly greater number of words in the word combination using a multiplier.

30. The computer program product of claim 27 further comprising:

grouping one or more of the categories into one or more channels.

31. The computer program product of claim 30 wherein determining the weight further comprising assigning an additional weight based on the one or more channels associated with the one or more categories to which the word is mapped.

32. The computer program product of claim 27 further comprising:

assigning a value to the weight based on a relative relatedness between words using a semantic distance measure.

33. The computer program product of claim 27 wherein the index is derived from existing structures configured in accordance with cognitive human notions of relatedness.

34. The computer program product of claim 27 wherein the index is generated based on counts of one or both of categories and keywords in a page of content.

35. The computer program product of claim 27 wherein the item of content comprises multilingual content and the classification is performed without translating the content.

36. The computer program product of claim 27 wherein content is discovered using a crawling engine.

37. The computer program product of claim 27 wherein the item of content comprises a web page and the classifying is performed by analyzing only a URL of the web page.

38. The computer program product of claim 27 further comprising:

adding a result of classifying the content to the data structure.

39. The computer program product of claim 27 wherein the item of content comprises a web page and wherein the classification is performed on both the web page as rendered and the language used to create the web page.