Methods and apparatus for clustering news content

• US 7,568,148 B1
• Filed: 06/30/2003
• Issued: 07/28/2009
• Est. Priority Date: 09/20/2002
• Status: Active Grant

• Current Assignee: Google LLC (Alphabet Inc.)
• Original Assignee: Google Inc. (Alphabet Inc.)
• Inventors: Schmitt, Michael, Bharat, Krishna, Curtiss, Michael
• Primary Examiner(s): Huynh; Thu
• Application Number:
  US10/611,269
• Time in Patent Office: 2,220 Days
• Field of Search: 715/513, 715/511, 715/500, 715/200, 715/203, 715/205, 715/234, 715/229, 707/200, 707/203, 707/7, 707/205
• US Class Current: 715/200
• CPC Class Codes: G06F 16/24578 using ranking
• CPC Class Codes: G06F 16/355 Class or cluster creation or modification
• CPC Class Codes: G06F 17/00 Digital computing or data processing equipment or methods, specially adapted for specific functions information retrieval, database structures or file system structures therefor G06F16/00
• CPC Class Codes: G06F 40/143 Markup, e.g. Standard Generalized Markup Language [SGML] or Document Type Definition [DTD]
• CPC Class Codes: H04L 67/02 based on web technology, e.g. hypertext transfer protocol [HTTP]
• CPC Class Codes: Y10S 707/99937 Sorting
• CPC Class Codes: Y10S 707/99953 Recoverability

Abstract

Methods and apparatus are described for scoring documents in response, in part, to parameters related to the document, source, and/or cluster score. Methods and apparatus are also described for scoring a cluster in response, in part, to parameters related to documents within the cluster and/or sources corresponding to the documents within the cluster. In one embodiment, the invention may identify the source; detect a plurality of documents published by the source; analyze the plurality of documents with respect to at least one parameter; and determine a source score for the source in response, in part, to the parameter. In another embodiment, the invention may identify a topic; identify a plurality of clusters in response to the topic; analyze at least one parameter corresponding to each of the plurality of clusters; and calculate a cluster score for each of the plurality of clusters in response, in part, to the parameter.

Claims

1. A method of scoring a source, comprising:

identifying the source;

detecting a plurality of documents published by the source;

calculating measures of freshness of the plurality of documents based on determining a difference between times that events occur and times that the identified source published the plurality of documents that includes content relating to the events;

calculating measures of quality for the plurality of documents based on at least one of;

numbers of views of the plurality of documents during a time period, or

numbers of links pointing to the plurality of documents; and

determining a source score for the source based on the measures of freshness and the measures of quality.

2. The method according to claim 1 where the calculating measures of freshness is further based on:

determining a number of canonical documents within the plurality of documents.

3. The method according to claim 1 further comprising:

removing a duplicate document within the plurality of documents.

4. The method according to claim 3 where the removing a duplicate document includes:

comparing text of one of the plurality of documents with text of another one of the plurality of documents to determine the duplicate document.

5. The method according to claim 1 where the determining a source score is further based on an originality of the plurality of documents.

6. The method according to claim 1 where the calculating measures of freshness is further based on:

measuring a frequency in which the source generates a canonical document.

7. The method according to claim 6 where measuring the frequency occurs over a predetermined length of time.

8. The method according to claim 1 where the times that the source published the plurality of documents is determined by a time stamp of publication.

9. The method according to claim 1 further comprising:

weighting at least one of the measures of freshness or the measures of quality prior to the determining the source score.

10. The method according to claim 1 further comprising

categorizing the source based on the source score.

11. The method according to claim 1 where the plurality of documents includes a plurality of articles.

12. A computer-readable medium having computer executable instructions for performing a method comprising:

identifying a news source;

detecting a plurality of documents published by the news source;

calculating a measure of freshness of a first document of the plurality of documents based on determining a difference between a time that an event occurred and a time that the identified news source published the first document that includes content relating to the event;

calculating a measure of quality for a second document of the plurality of documents based on at least one of;

numbers of views of the second document during a time period, or

numbers of links pointing to the second document; and

determining a source score for the news source based on the measure of freshness and the measure of quality.

13. The computer-readable medium of claim 12, where the method further comprises:

removing a duplicate document within the plurality of documents.

14. The computer-readable medium of claim 13 where the removing a duplicate document includes:

-

comparing text of one of the plurality of documents with text of another one of the plurality of documents to determine the duplicate document.

15. The computer-readable medium of claim 12, where the method further comprises:

placing the news source into a category based on the source score.

16. The computer-readable medium of claim 12 where the calculating a measure of freshness is further based on:

determining a number of canonical documents within the plurality of documents.

17. The computer-readable medium of claim 12 where the determining a source score is further based on an originality of the plurality of documents.

18. The computer-readable medium of claim 12 where the calculating a measure of freshness is further based on:

measuring a frequency in which the news source generates a canonical document.

19. The computer-readable medium of claim 18 where measuring the frequency occurs over a predetermined length of time.

20. The computer-readable medium of claim 12 where the time that the news source published the first document is determined by a time stamp of publication.

21. The computer-readable medium of claim 12 where the method further comprises:

weighting at least one of the measure of freshness or the measure of quality prior to the determining the source score.

22. A system for scoring a source, comprising:

means for identifying the source;

means for detecting a plurality of documents published by the source;

means for calculating a measure of freshness of a first document of the plurality of documents based on determining a difference between a time that an event occurred and a time that the source published the first document that includes content relating to the event;

means for calculating a measure of quality for a second document of the plurality of documents based on at least one of;

numbers of views of the second document during a time period, or

numbers of links pointing to the second documents;

means for determining a source score for the source based on the measure of freshness and the measure of quality; and

a memory device to store the source score.

24. The system of claim 22, further comprising:

means for associating the source with a category based on the source score; and

means for storing the association between the source and the category.

23. The system of 22, further comprising:

means for removing duplicate documents in the plurality of documents.

25. A computer-implemented method comprising:

identifying a news source;

detecting a plurality of news articles published by the news source;

removing news articles in the plurality of news articles that are determined to be duplicates of other news articles in the plurality of news articles to form a second plurality of news articles;

calculating measures of freshness of the second plurality of news articles based on determining a difference between times that events occur and the news source published the second plurality of news articles that includes content relating to the events;

calculating measures of quality for the second plurality of news articles based on at least one of;

numbers of views of the second plurality of news articles during a time period, or

numbers of links pointing to the second plurality of news articles;

determining a score for the news source based on the measures of freshness and the measures of quality; and

storing the score for the news source.

26. The method of claim 25 where the removing news articles further includes:

identifying at least one news article of the plurality of news articles that is re-published from another news source; and

removing the at least one news article from the plurality of news articles.

27. The method of claim 25 where the determining a score for the news source is further based on a number of news articles published by the news source.

28. The method of claim 25 further comprising:

associating the news source with a category based on the score for the news source.