Identifying Information Related to a Particular Entity from Electronic Sources

• US 20090070325A1
• Filed: 09/11/2008
• Published: 03/12/2009
• Est. Priority Date: 09/12/2007
• Status: Abandoned Application

• Current Assignee: Reputation.com, Inc.
• Original Assignee: Reputation.com, Inc.
• Inventors: Fertik, Michael Benjamin, Gabriel, Raefer Christopher, Tripp, Owen Wheble
• Application Number:
  US12/209,169
• Publication Number:
  US 20090070325A1
• Time in Patent Office: Days
• US Class Current: 1/1
• CPC Class Codes: G06F 16/338 Presentation of query results
• CPC Class Codes: G06F 16/35 Clustering; Classification

Abstract

Presented are systems, apparatuses, articles of manufacture, and methods for identifying information about a particular entity including receiving electronic documents selected based on one or more search terms from a plurality of terms related to the particular entity, determining one or more feature vectors for each received electronic document, where each feature vector is determined based on the associated electronic document, clustering the received electronic documents into a first set of clusters of documents based on the similarity among the determined feature vectors, and determining a rank for each cluster of documents in the first set of clusters of documents based on one or more ranking terms from the plurality of terms related to the particular entity, where the one or more ranking terms contain at least one term from the plurality of terms for the particular entity that is not in the one or more search terms.

Claims

1. A method for identifying information about a particular entity comprising:

receiving electronic documents selected based on one or more search terms from a plurality of terms related to the particular entity;

determining one or more feature vectors for each received electronic document, wherein each feature vector is determined based on the associated electronic document;

clustering the received electronic documents into a first set of clusters of documents based on the similarity among the determined feature vectors; and

determining a rank for each cluster of documents in the first set of clusters of documents based on one or more ranking terms from the plurality of terms related to the particular entity, wherein the one or more ranking terms contain at least one term from the plurality of terms for the particular entity that is not in the one or more search terms.

2. The method of claim 1, wherein the one or more feature vectors comprise one or more feature vectors from the group selected from a term frequency inverse document frequency vector, a proper noun vector, a metadata vector, and a personal information vector.

3. The method of claim 1, further comprising presenting the ranked clusters to the particular entity.

4. The method of claim 1, further comprising:

reviewing the ranked clusters;

modifying the ranking of the clusters; and

presenting the modified ranking of the clusters to the particular entity.

5. The method of claim 4, wherein modifying the ranking of the clusters comprises removing or combining one or more clusters from the results.

6. The method of claim 1, further comprising:

determining a second set of one or more search terms based on one or more features in the determined feature vectors of one or more received electronic documents;

receiving a second set of electronic documents selected based on the second set of one or search terms;

determining a second set of one or more feature vectors for each electronic document in the second set of electronic documents, wherein each feature vector is determined based on the associated electronic document;

clustering the second set of received electronic documents into a second set of clusters of documents based on the similarity among the second set of one or more feature vectors; and

determining a rank for each cluster of documents in the first set of clusters of documents and the second set of clustered documents based on the one or more ranking terms from the plurality of terms related to the particular entity, wherein the one or more ranking terms contains at least one term from the plurality of terms for the particular entity that is not in the second set of one or more search terms.

7. The method of claim 6, wherein the second set of one or more search terms are determined based on the frequency of occurrence of those features in the one or more feature vectors that do not have a corresponding term in the plurality of terms related to the particular entity.

8. The method of claim 1, further comprising:

submitting a query to an electronic information module, wherein the query is determined based on the one or more search terms; and

receiving the electronic documents comprises receiving a response to the query from the electronic information module.

9. The method of claim 1, further comprising:

receiving a set of electronic documents, wherein the set of electronic documents are selected based on a first set of one or more search terms from the plurality of terms related to the particular entity;

if the set of electronic documents contains more than a threshold number of electronic documents, then determining the one or more search terms used in the receiving step as the first set of one or more search terms combined with a second set of one or more search terms from the plurality of terms related to the particular entity, wherein the search terms in the second set of one or more search terms and the search terms in the first set of one or more search terms do not overlap; and

if the set of electronic documents contains no more than the threshold number of electronic documents, then the step of receiving the electronic documents comprises receiving the set of electronic documents.

10. The method of claim 1, further comprising:

receiving a set of electronic documents, wherein the set of electronic documents are selected based on a first set of one or more search terms from the plurality of terms related to the particular entity;

determining a count of direct pages in the first set of electronic documents;

if the set of electronic documents contains more than a threshold count of direct pages, then determining the one or more search terms used in the receiving step as the first set of one or more search terms in combination with a second set of one or more search terms from the plurality of terms related to the particular entity, wherein the features in the second set of one or more search terms and the features in the first set of one or more search terms do not overlap; and

if the set of electronic documents contains no more than the threshold count of direct pages, then the step of receiving the electronic documents comprises receiving the set of electronic documents.

11. The method of claim 1, wherein clustering the received electronic documents comprises:

(a) creating initial clusters of documents;

(b) for each cluster of documents, determining the similarity of the feature vectors of the documents within each cluster with those in each other cluster;

(c) determining a highest similarity measure among all of the clusters; and

(d) if the highest similarity measure is at least a threshold value, combining the two clusters with the highest determined similarity measure.

12. The method of claim 11, wherein clustering the received electronic documents further comprises repeating steps (b), (c), and (d) until the highest similarity measure among the clusters is below the threshold value.

13. The method of claim 11, wherein the similarity of the feature vectors of a document is calculated based on a normalized dot product of the feature vectors.

14. The method of claim 1, wherein determining the rank for each cluster of documents comprises assigning a higher rank to those clusters of documents that contain documents that have a higher similarity measure with the one or more ranking terms.

15. A system for identifying information about a particular entity comprising:

a harvesting module configured to receive electronic documents selected based on one or more search terms from a plurality of terms related to the particular entity;

a feature extracting module configured to determine one or more feature vectors associated with each received electronic document, wherein each feature vector is determined based on the associated electronic document;

a clustering module configured to cluster the received electronic documents into a first set of clusters of documents based on the similarity among the determined feature vectors; and

a ranking module configured to determine a rank for each cluster of documents in the first set of clusters of documents based on one or more ranking terms from the plurality of terms related to the particular entity, wherein the one or more ranking terms contain at least one term from the plurality of terms for the particular entity that is not in the one or more search terms.

16. The system of claim 15, wherein the feature extracting module is further configured to determine the one or more feature vectors from the group selected from a term frequency inverse document frequency vector, a proper noun vector, a metadata vector, and a personal information vector.

17. The system of claim 15, further comprising a display module configured to present the ranked clusters to the particular entity.

18. The system of claim 15, wherein:

the harvesting module is further configured to receive a second set of electronic documents selected based on a second set of one or more search terms wherein the second set of search terms is determined based on one or more features in the determined feature vectors of one or more received electronic documents;

the feature extracting module is further configured to determine a second set of one or more feature vectors for each electronic document in the second set of electronic documents, wherein each feature vector is determined based on the associated electronic document;

the clustering module is further configured to cluster the second set of received electronic documents into a second set of clusters of documents based on the similarity among the second set of one or more feature vectors; and

the ranking module is configured to determine a rank for each cluster of documents in the first set of clusters of documents and the second set of clustered documents based on the one or more ranking terms from the plurality of terms related to the particular entity, wherein the one or more ranking terms contains at least one term from the plurality of terms for the particular entity that is not in the second set of one or more search terms.

19. The system of claim 20, wherein the harvesting module is further configured to determine the second set of one or more search terms based on the frequency of occurrence of those features in the one or more feature vectors that do not have a corresponding term in the plurality of terms related to the particular entity.

20. The system of claim 15, wherein the harvesting module is further configured to:

submit a query to an electronic information module, wherein the query is determined based on the one or more search terms; and

receive the electronic documents via a response to the query from the electronic information module.

21. The system of claim 15, wherein the harvesting module is configured to:

select a set of electronic documents based on a first set of one or more search terms from the plurality of terms related to the particular entity; and

determine whether the set of electronic documents contains more than a threshold number of electronic documents.

22. The system of claim 21, wherein the harvesting module is further configured to refine the selection, if the first set of electronic documents contains more than the threshold number of electronic documents, by determining the one or more search terms used to select the set of electronic documents as the first set of one or more search terms combined with a second set of one or more search terms from the plurality of terms related to the particular entity, wherein the search terms in the second set of one or more search terms and the search terms in the first set of one or more search terms do not overlap.

23. The system of claim 21, wherein the harvesting module is further configured to receive the set of electronic documents if the set of electronic documents contains no more than the threshold number of electronic documents.

24. The system of claim 15, wherein the harvesting module is configured to:

select a set of electronic documents based on a first set of one or more search terms from the plurality of terms related to the particular entity; and

determine a count of direct pages in the set of electronic documents.

25. The system of claim 24, wherein the harvesting module is further configured to refine the selection, if the count of direct pages in the set of electronic documents contains more than a threshold count of direct pages, by determining the one or more search terms used to select the set of electronic documents as the first set of one or more search terms in combination with a second set of one or more search terms from the plurality of terms related to the particular entity, wherein the features in the second set of one or more search terms and the features in the first set of one or more search terms do not overlap.

26. The system of claim 24, wherein the harvesting module is further configured to receive the set of electronic documents if the set of electronic documents contains no more than the threshold count of direct pages.

27. The system of claim 15, wherein the clustering module is further configured to:

(a) create initial clusters of documents;

(b) determine the similarity of the feature vectors of the documents within each cluster with those in each other cluster for each cluster of documents;

(c) determine a highest similarity measure among all of the clusters; and

(d) combine the two clusters with the highest determined similarity measure if the highest similarity measure is at least a threshold value.

28. The system of claim 27, wherein the clustering module is further configured to repeat steps (b), (c), and (d) until the highest similarity measure among the clusters is below the threshold value.

29. The system of claim 27, wherein the feature extracting module is further configured to calculate the similarity of the feature vectors of a document based on a normalized dot product of the feature vectors.

30. The system of claim 15, wherein the ranking module is configured to determine the rank for each cluster of documents by assigning a higher rank to those clusters of documents that contain documents that have a higher similarity measure with the one or more ranking terms.

31. A computer readable medium including instructions that, when executed, cause a computer to perform a method for identifying information about a particular entity, the method comprising:

receiving electronic documents selected based on one or more search terms from a plurality of terms related to the particular entity;

determining one or more feature vectors for each received electronic document, wherein each feature vector is determined based on the associated electronic document;

clustering the received electronic documents into a first set of clusters of documents based on the similarity among the determined feature vectors; and

determining a rank for each cluster of documents in the first set of clusters of documents based on one or more ranking terms from the plurality of terms related to the particular entity, wherein the one or more ranking terms contain at least one term from the plurality of terms for the particular entity that is not in the one or more search terms.

32. The computer readable medium of claim 31, wherein the one or more feature vectors comprise one or more feature vectors from the group selected from a term frequency inverse document frequency vector, a proper noun vector, a metadata vector, and a personal information vector.

33. The computer readable medium of claim 31, further comprising presenting the ranked clusters to the particular entity.

34. The computer readable medium of claim 31, further comprising

reviewing the ranked clusters;

modifying the ranking of the clusters; and

presenting the modified ranking of the clusters to the particular entity.

35. The computer readable medium of claim 34, wherein modifying the ranking of the clusters comprises combining or removing one or more clusters from the results.

36. The computer readable medium of claim 31, further comprising:

determining a second set of one or more search terms based on one or more features in the determined feature vectors of one or more received electronic documents;

receiving a second set of electronic documents selected based on the second set of one or search terms;

determining a second set of one or more feature vectors for each electronic document in the second set of electronic documents, wherein each feature vector is determined based on the associated electronic document;

clustering the second set of received electronic documents into a second set of clusters of documents based on the similarity among the second set of one or more feature vectors; and

determining a rank for each cluster of documents in the first set of clusters of documents and the second set of clustered documents based on the one or more ranking terms from the plurality of terms related to the particular entity, wherein the one or more ranking terms contains at least one term from the plurality of terms for the particular entity that is not in the second set of one or more search terms.

37. The computer readable medium of claim 36, wherein the second set of one or more search terms are determined based on the frequency of occurrence of those features in the one or more feature vectors that do not have a corresponding term in the plurality of terms related to the particular entity.

38. The computer readable medium of claim 31, further comprising:

submitting a query to an electronic information module, wherein the query is determined based on the one or more search terms; and

receiving the electronic documents comprises receiving a response to the query from the electronic information module.

39. The computer readable medium of claim 31, further comprising:

receiving a set of electronic documents, wherein the set of electronic documents are selected based on a first set of one or more search terms from the plurality of terms related to the particular entity;

if the set of electronic documents contains more than a threshold number of electronic documents, then determining the one or more search terms used in the receiving step as the first set of one or more search terms combined with a second set of one or more search terms from the plurality of terms related to the particular entity, wherein the search terms in the second set of one or more search terms and the search terms in the first set of one or more search terms do not overlap; and

if the set of electronic documents contains no more than the threshold number of electronic documents, then step of receiving the electronic documents comprises receiving the set of electronic documents.

40. The computer readable medium of claim 31, further comprising:

receiving a set of electronic documents, wherein the set of electronic documents are selected based on a first set of one or more search terms from the plurality of terms related to the particular entity;

determining a count of direct pages in the set of electronic documents;

if the set of electronic documents contains more than a threshold count of direct pages, then determining the one or more search terms used in the receiving step as the first set of one or more search terms in combination with a second set of one or more search terms from the plurality of terms related to the particular entity, wherein the features in the second set of one or more search terms and the features in the first set of one or more search terms do not overlap; and

if the set of electronic documents contains no more than the threshold count of direct pages, then step of receiving the electronic documents comprises receiving the set of electronic documents.

41. The computer readable medium of claim 31, wherein clustering the received electronic documents comprises:

(a) creating initial clusters of documents;

(b) for each cluster of documents, determining the similarity of the feature vectors of the documents within each cluster with those in each other cluster;

(c) determining a highest similarity measure among all of the clusters; and

(d) if the highest similarity measure is at least a threshold value, combining the two clusters with the highest determined similarity measure.

42. The computer readable medium of claim 41, wherein clustering the received electronic documents further comprises repeating steps (b), (c), and (d) until the highest similarity measure among the clusters is below the threshold value.

43. The computer readable medium of claim 41, wherein the similarity of the feature vectors of a document is calculated based on a normalized dot product of the feature vectors.

44. The computer readable medium of claim 31, wherein determining the rank for each cluster of documents comprises assigning a higher rank to those clusters of documents that contain documents that have a higher similarity measure with the one or more ranking terms.

45. An apparatus for identifying information about a particular entity comprising:

means for receiving electronic documents selected based on one or more search terms from a plurality of terms related to the particular entity;

means for determining one or more feature vectors for each received electronic document, wherein each feature vector is determined based on the associated electronic document;

means for clustering the received electronic documents into a first set of clusters of documents based on the similarity among the determined feature vectors; and

means for determining a rank for each cluster of documents in the first set of clusters of documents based on one or more ranking terms from the plurality of terms related to the particular entity, wherein the one or more ranking terms contain at least one term from the plurality of terms for the particular entity that is not in the one or more search terms.