RECOMMENDING TOPIC CLUSTERS FOR UNSTRUCTURED TEXT DOCUMENTS

• US 20180032606A1
• Filed: 07/13/2017
• Published: 02/01/2018
• Est. Priority Date: 07/26/2016
• Status: Active Grant

• Current Assignee: Qualtrics LLC (SAP SE)
• Original Assignee: Qualtrics LLC (SAP SE)
• Inventors: Tolman, Bradley, Norton, Robert David, Mumford, Martin D.
• Granted Patent: US 11,645,317 B2
• Time in Patent Office: Days
• CPC Class Codes: G06F 16/35 Clustering; Classification
• CPC Class Codes: G06F 16/93 Document management systems
• CPC Class Codes: G06F 16/951 Indexing; Web crawling techniques

Abstract

Embodiments of the present disclosure generally relate to a content management system that automatically determines and generates topic clusters from a collection of electronic text documents. For example, the content management system analyzes a collection of electronic text documents to identify key terms and terms related to the key terms. Based on the key terms and related terms, the content management system generates a topic cluster that includes the key term and related terms. The content management system then organizes the electronic text documents based on terms within a given text document matching terms within a given topic cluster. Further, the content management system presents the topic clusters and organized electronic text documents to a user.

Claims

1. A method comprising:

accessing a plurality of electronic text documents comprising a plurality of terms;

analyzing, by at least one processor, the plurality of terms to determine a significance value for each term;

identifying, based on the significance value determined for each term, a key term from the plurality of terms;

determining, from the plurality of terms, one or more related terms associated with the key term;

generating a topic cluster comprising the key term and the one or more related terms associated with the key term; and

providing, to a client device associated with a user, at least one electronic text document from the plurality of electronic text documents that corresponds to the topic cluster.

2. The method of claim 1, further comprising ranking the plurality of terms based on the significance value for each term, and wherein identifying the key term from the plurality of terms comprises determining that the key term is a highest ranked term from the plurality of terms.

3. The method of claim 1, further comprising determining the significance value of each term of the plurality of terms by generating a statistically improbable phrase value for each term.

4. The method of claim 3, further comprising:

identifying a text corpus comprising corpus terms; and

generating the statistically improbable phrase value for each term within the plurality of electronic text documents by comparing a sample frequency occurrence of a given term in the plurality of electronic text documents with a corpus frequency occurrence of the given term in the text corpus.

5. The method of claim 1, further comprising identifying one or more terms from the plurality of terms of the plurality of electronic text documents that are located proximate the key term in an n-dimensional vector space, wherein the one or more related terms associated with the key term comprises the one or more terms that are located proximate the key term in an n-dimensional vector space.

6. The method of claim 5, wherein the one or more related terms associated with the key term are located proximate the key term when the one or more key terms are located within a threshold distance from the key term in the n-dimensional vector space.

7. The method of claim 6, further comprising:

receiving an indication of a user selection of a topic corresponding to the topic cluster; and

providing, for presentation to the user and in response to the indication of the user selection of the topic cluster, one or more electronic text documents from the plurality of electronic text documents that include at least one term from the topic cluster.

8. The method of claim 7, further comprising:

receiving an indication of a user selection to expand the topic cluster;

adjusting, in response to the indication of a user selection to expand the topic cluster, the threshold distance from the key term in the n-dimensional vector space to include one or more additional terms associated with the key term;

modifying the topic cluster to comprise the key term, the one or more related terms associated with the key term, and the one or more additional terms associated with the key term; and

providing, for presentation to the user and in response to the indication of the user selection to expand the topic cluster, an additional electronic text document from the plurality of electronic text documents that includes at least one term from the one or more additional terms associated with the key term.

9. The method of claim 7, further comprising:

receiving an indication of a user selection of a term to exclude from the one or more related terms associated with the key term;

modifying the topic cluster by removing the term to exclude from the topic cluster; and

providing, for presentation to the user and in response to the indication of the user selection of the term to exclude, one or more electronic text documents from the plurality of electronic text documents that have at least one term from the modified topic cluster.

10. The method of claim 9, wherein receiving the indication of the user selection to exclude the term from the one or more related terms associated with the key term comprises an indication to split the term as a new key term, the method further comprising:

determining, from the plurality of terms, one or more new related terms associated with the new key term;

generating a new topic cluster comprising the new key term and the one or more new related terms associated with the new key term; and

providing, to the client device associated with the user, at least one electronic text document from the plurality of electronic text documents that corresponds to the new topic cluster.

11. A system comprising:

at least one processor; and

at least one non-transitory computer-readable storage medium storing instructions that, when executed by the at least one processor, cause the system to;

access a plurality of electronic text documents comprising a plurality of terms;

analyze the plurality of terms to determine a significance value for each term;

identify a first key term from the plurality of terms based on the first key term having a highest significance value;

determine, from the plurality of terms, a first set of related terms associated with the first key term;

generate a first topic cluster comprising the first key term and the first set of related terms associated with the first key term; and

provide, for presentation to a user, a first topic corresponding to the first topic cluster.

12. The system of claim 11, further comprising instructions that, when executed by the at least one processor, cause the system to:

receive an indication of a user selection of the first topic; and

providing, for presentation to the user and in response to the indication of the user selection of the first topic, one or more electronic text documents from the plurality of electronic text documents that include at least one term from the first topic cluster.

13. The system of claim 12, further comprising instructions that, when executed by the at least one processor, cause the system to:

identify a second key term from the plurality of terms based on the second key term having a second highest significance value;

determine, from the plurality of terms, a second set of related terms associated with the second key term; and

generate a second topic cluster comprising the second key term and the second set of related terms associated with the second key term.

14. The system of claim 13, further comprising instructions that, when executed by the at least one processor, cause the system to:

receive an indication of a user selection to merge the first topic cluster with the second topic cluster;

merge, the second topic cluster with the first topic cluster to create a merged topic cluster; and

provide the merged topic cluster for presentation to the user.

15. The system of claim 14, further comprising instructions that, when executed by the at least one processor, cause the system to provide, for presentation to the user and in response to the indication of the user selection to merge the first topic cluster with the second topic cluster, one or more electronic text documents from the plurality of electronic text documents that include at least one term from the merged topic cluster.

16. The system of claim 15, further comprising instructions that, when executed by the at least one processor, cause the system to organize a presentation of topic clusters within a graphical user interface to list the first topic cluster before the second topic cluster based on the first key term associated with the first topic cluster having a higher significance value than the second key term associated with the second topic cluster.

17. The system of claim 16, further comprising instructions that, when executed by the at least one processor, cause the system to:

receive an indication of a user selection to add a topic cluster to the presentation of topic clusters within the graphical user interface;

generate an additional topic cluster comprising an additional key term identified based on the determined significance values and corresponding related terms associated with the additional key term; and

provide, for presentation to the user and in response to the indication of the user selection to add a topic cluster, one or more electronic text documents from the plurality of electronic text documents that include at least one term from the additional topic cluster.

18. A method comprising:

receiving, from a plurality of users, a plurality of unstructured text responses received in response to an electronic survey;

analyzing, by at least one processor, the plurality of unstructured text responses to identify a plurality of terms;

determining a significance value for each term of the plurality of terms;

identifying, based on the significance value determined for each term, key terms from the plurality of terms;

determining, from the plurality of terms, one or more related terms corresponding to each key term;

generating topic clusters, wherein each topic cluster comprises a key term and at least one related term corresponding to the key term; and

providing, for presentation to an administrative user, a list of topics corresponding to the topic clusters.

19. The method of claim 18, further comprising:

receiving an indication of a user selection of a topic from the list of topics;

providing, for presentation to the administrator user and in response to the indication of the user selection of the topic from the list of topics, one or more responses from the plurality of unstructured text responses that include at least one term from a topic cluster associated with the selected topic; and

emphasizing that the at least one term from the topic cluster is included within each of the one or more responses.

20. The method of claim 19, further comprising labeling metadata of the one or more responses from the plurality of unstructured text responses with the at least one term from the topic cluster.