SYSTEMS AND METHODS FOR ANALYZING ELECTRONIC TEXT

• US 20100145940A1
• Filed: 12/09/2008
• Published: 06/10/2010
• Est. Priority Date: 12/09/2008
• Status: Active Grant

• Current Assignee: DoorDash, Inc.
• Original Assignee: International Business Machines Corporation
• Inventors: Proctor, Larry, Chen, Ying, Spangler, William Scott
• Granted Patent: US 8,606,815 B2
• Time in Patent Office: Days
• US Class Current: 707/736
• CPC Class Codes: G06F 16/35 Clustering; Classification
• CPC Class Codes: G06F 16/951 Indexing; Web crawling techniques

Abstract

Systems and methods for systematically analyzing an electronic text are described. In one embodiment, the method includes receiving the electronic text from a plurality of sources. The method also includes determining an at least one term of interest to be identified in the electronic text. The method further includes identifying a plurality of locations within the electronic text including the at least one term of interest. The method also includes for each location within a plurality of locations, creating a snippet from a text segment around the at least one term of interest at the location within the electronic text. The method further includes creating multiple taxonomies for the at least one term of interest from the snippets, wherein the taxonomies include an at least one category. The method also includes determining co-occurrences between the multiple taxonomies to determine associations between categories of a different taxonomies of the multiple taxonomies.

Claims

1. A computer-implemented method for systematically analyzing an electronic text, comprising:

receiving the electronic text from a plurality of sources;

determining an at least one term of interest to be identified in the electronic text;

identifying a plurality of locations within the electronic text including the at least one term of interest;

for each location within a plurality of locations, creating a snippet from a text segment around the at least one term of interest at the location within the electronic text;

creating multiple taxonomies for the at least one term of interest from the snippets, wherein the taxonomies include an at least one category; and

determining co-occurrences between the multiple taxonomies to determine associations between categories of a different taxonomies of the multiple taxonomies.

2. The computer-implemented method of claim 1, further comprising:

determining co-occurrences between a category of a single taxonomy and the at least one term of interest to determine significance of the at least one term of interest; and

sorting the at least one term of interest by significance.

3. The computer-implemented method of claim 2, further comprising sending the sorted at least one term of interest for user review.

4. The computer-implemented method of claim 2, wherein each taxonomy of the multiple taxonomies is one of the group consisting of:

a text clustering based taxonomy,

a taxonomy created from the occurrence of terms of interest,

a sentiment based taxonomy, and

a time based taxonomy.

5. The computer-implemented method of claim 4, wherein for each determined co-occurrence, determining a meaning of the co-occurrence from the term of interest, electronic text and sources of the electronic text involved in the co-occurrence.

6. The computer-implemented method of claim 5, further comprising:

creating from the categories of the electronic text a plurality of category/term of interest statistics of importance; and

determining from the electronic text within each category and the category/term of interest statistics the importance of each co-occurrence.

7. The computer-implemented method of claim 6, wherein the text clustering is configured to use a method based on selecting a cohesive terms of the electronic text to seed category selection.

8. The computer-implemented method of claim 2, wherein the electronic text is web based.

9. A system for systematically analyzing an electronic text, comprising:

a module to receive the electronic text from a plurality of sources;

a module to determine an at least one term of interest to be identified in the electronic text;

a module to identify a plurality of locations within the electronic text including the at least one term of interest;

a module to create for each location within a plurality of locations a snippet from a text segment around the at least one term of interest at the location within the electronic text;

a module to create multiple taxonomies for the at least one term of interest from the snippets, wherein the taxonomies include an at least one category; and

a module to determine co-occurrences between the multiple taxonomies to determine associations between categories of a different taxonomies of the multiple taxonomies.

10. The system of claim 9, further comprising:

a module to determine co-occurrences for a single taxonomy against a term feature space to determine significance of the at least one term of interest; and

a module to sort the at least one term of interest by significance.

11. The system of claim 10, further comprising a module to send the sorted at least one term of interest for review.

12. The system of claim 10, wherein each taxonomy of the multiple taxonomies is one of the group consisting of:

a text clustering based taxonomy,

a taxonomy created from the occurrence of terms of interest,

a sentiment based taxonomy, and

a time based taxonomy.

13. The system of claim 12, wherein the module to determine co-occurrences for each co-occurrences determines a meaning of the co-occurrence from the term of interest, electronic text and sources of the electronic text involved in the co-occurrence.

14. The system of claim 13, further comprising:

determining for the at least one term of interest categories of the electronic text in the taxonomies;

creating from the categories of the electronic text a plurality of category/term of interest statistics of importance; and

determining from the electronic text within each category and the category/term of interest statistics the importance of each category.

15. The system of claim 14, wherein the text clustering is configured to use a method based on selecting a cohesive terms of the electronic text to seed category selection.

16. The system of claim 10, wherein the electronic text is web based.

17. A computer program product comprising a computer useable storage medium to store a computer readable program, wherein the computer readable program, when executed on a computer, causes the computer to perform operations comprising:

receiving the electronic text from a plurality of sources;

determining an at least one term of interest to be identified in the electronic text;

identifying a plurality of locations within the electronic text including the at least one term of interest;

for each location within a plurality of locations, creating a snippet from a text segment around the at least one term of interest at the location within the electronic text;

creating multiple taxonomies for the at least one term of interest from the snippets, wherein the taxonomies include an at least one category; and

determining co-occurrences between the multiple taxonomies to determine associations between categories of a different taxonomies of the multiple taxonomies;

determining co-occurrences between a category of a single taxonomy and the at least one term of interest to determine significance of the at least one term of interest; and

sorting the at least one term of interest by significance; and

outputting the sorted at least one term of interest.

18. The computer program product of claim 17, wherein each taxonomy of the multiple taxonomies is one of the group consisting of:

a text clustering based taxonomy,

a taxonomy created from the occurrence of terms of interest,

a sentiment based taxonomy, and

a time based taxonomy.

19. The computer program product of claim 18, wherein the text clustering is configured to use a method based on selecting a cohesive terms of the electronic text to seed category selection.

20. The computer program product of claim 17, wherein the electronic text is web based.