System and method for good nearest neighbor clustering of text
First Claim
1. A computer-implemented method for clustering text, comprising:
- representing at least one text in a set of texts as a dimensional vector of words;
representing an other text in the set of texts as a dimensional vector of words;
determining a dot-product of the dimensional vector of the other text and the dimensional vector of the at least one text;
comparing the dot-product to a threshold, wherein the threshold comprises an upper bound of a value in a range from zero to one that represents a cosine similarity between the other text and the at least one text;
if the dot-product exceeds the threshold, determining the at least one text to be the good nearest neighbor of the other text;
clustering the other text in a cluster; and
clustering the at least one text determined to be the good nearest neighbor of the other text in the cluster.
9 Assignments
0 Petitions
Accused Products
Abstract
An improved system and method for clustering text or content described by text is provided. Each text in a set of texts may be represented as a dimensional vector of words. Singleton texts that may not be similar to another text may be excluded from the set of texts for clustering. Texts identified as good nearest neighbors may then be grouped in the same cluster. In addition, metadata describing content may be used for clustering items of aggregated content from content feeds. Metadata describing items of content from content feeds may be converted into a set of texts and texts identified as good nearest neighbors may then be clustered. Items of content feeds described by the clustered texts may then be similarly clustered. Any types of items of content that may be described by text may be clustered, including audio, images, video, multimedia content, and so forth.
17 Citations
10 Claims
-
1. A computer-implemented method for clustering text, comprising:
-
representing at least one text in a set of texts as a dimensional vector of words; representing an other text in the set of texts as a dimensional vector of words; determining a dot-product of the dimensional vector of the other text and the dimensional vector of the at least one text; comparing the dot-product to a threshold, wherein the threshold comprises an upper bound of a value in a range from zero to one that represents a cosine similarity between the other text and the at least one text; if the dot-product exceeds the threshold, determining the at least one text to be the good nearest neighbor of the other text; clustering the other text in a cluster; and clustering the at least one text determined to be the good nearest neighbor of the other text in the cluster. - View Dependent Claims (2, 3, 4)
-
-
5. A computer-implemented method for clustering text, comprising:
-
representing a text in a set of texts as a dimensional vector of words; representing each of one or more texts in the set of texts as a dimensional vector of words; for each of the one or more texts, determining a dot-product of the dimensional vector of the each of the one or more texts and the dimensional vector of the text; comparing the dot-product for each of the one or more texts to a similarity threshold; for each of the one or more texts, determining a measure of a number of words used both by the text and the each of the one or more texts; comparing the measure of the number of words for each of the one or more texts to an overlap threshold; if the similarity threshold exceeds the dot-product for each of the one or more texts and the overlap threshold exceeds the measure of the number of words for each of the one or more texts, determining the text not to be similar to the one or more texts in the set of texts; excluding the text determined not to be similar from clustering with the set of texts; determining at least one text of the one or more texts in the set of texts to be a good nearest neighbor of an other text of the one or more texts in the set of texts; clustering the other text in a cluster; and clustering the at least one text determined to be the good nearest neighbor of the other text in the cluster. - View Dependent Claims (6, 7, 8, 9)
-
-
10. A computer-readable storage medium having computer-executable instructions for performing the steps of:
-
representing at least one text in a set of texts as a dimensional vector of words; representing an other text in the set of texts as a dimensional vector of words; determining a dot-product of the dimensional vector of the other text and the dimensional vector of the at least one text; comparing the dot-product to a threshold, wherein the threshold comprises an upper bound of a value in a range from zero to one that represents a cosine similarity between the other text and the at least one text; if the dot-product exceeds the threshold, determining the at least one text to be the good nearest neighbor of the other text; clustering the other text in a cluster; and clustering the at least one text determined to be the good nearest neighbor of the other text in the cluster.
-
Specification