Ranking content using content and content authors

• US 10,185,778 B1
• Filed: 12/30/2016
• Issued: 01/22/2019
• Est. Priority Date: 12/07/2006
• Status: Active Grant

• Current Assignee: Google LLC (Alphabet Inc.)
• Original Assignee: Google LLC (Alphabet Inc.)
• Inventors: Osinga, Douwe, Christoph, Stefan
• Primary Examiner(s): Ruiz, Angelica
• Application Number:
  US15/395,316
• Time in Patent Office: 753 Days
• CPC Class Codes: G06F 16/382 using citations hypermedia G06F16/94
• CPC Class Codes: G06F 16/93 Document management systems
• CPC Class Codes: G06F 16/955 using information identifiers, e.g. uniform resource locators [URL]
• CPC Class Codes: G06V 30/413 Classification of content, e.g. text, photographs or tables
• CPC Class Codes: G06V 30/414 Extracting the geometrical structure, e.g. layout tree; Block segmentation, e.g. bounding boxes for graphics or text

Abstract

Methods, systems, and apparatus, including computer program products for identifying original content. In one aspect a method is described that includes identifying a first document in a collection of documents. The first document contains a content piece and the content piece does not occur in any earlier document in the collection. The first document is associated with a first author and the first author associated with a first rank. The first rank of the first author is determined using a score of the content piece. The score is a figure of merit of the content piece.

Claims

1. A computer-implemented method comprising:

accessing, by one or more processors, a corpus of documents;

determining, by the one or more processors, that a particular document in the corpus of documents includes multiple content pieces that each occur in at least one other earlier document in the corpus of documents;

determining, by the one or more processors, an extent to which the entire content of the particular document is made up of the multiple content pieces that each occur in the at least one other earlier document in the corpus of documents;

adjusting, by the one or more processors, a rank of a source of the particular document in relation to a source of other documents in the corpus of documents based at least in part on the extent to which the entire content of the particular document is made up of the multiple content pieces that each occur in the at least one other earlier document in the corpus of documents; and

configuring a web crawling or search result ranking process for the source of the particular document based at least in part on the adjusted rank.

2. The method of claim 1, wherein configuring the web crawling process comprises configuring a frequency with which a web crawler crawls a web server associated with the source based at least on the rank.

3. The method of claim 1, wherein adjusting the rank of the source comprises penalizing the rank of the source when the extent to which the entire content of the particular document is made up of the multiple content pieces that each occur in the at least one other earlier document exceeds a threshold, or promoting the rank of the source when the extent to which the entire content of the particular document is made up of the multiple content pieces that each occur in the at least one other earlier document does not exceed a threshold.

4. The method of claim 1, wherein multiple content pieces represents a number of non-adjacent words.

5. The method of claim 1, comprising fragmenting the particular document into a multiple content pieces.

6. The method of claim 1, wherein the corpus of documents comprises documents that are classified as news-related documents.

7. The method of claim 1, wherein the rank of the source is penalized in proportion to the extent to which the entire content of the particular document is made up of the multiple content pieces that each occur in the at least one other earlier document in the corpus of documents.

8. A non-transitory computer readable medium having stored thereon instructions, which, when executed by one or more processors, causes the one or more processors to perform the operations comprising:

accessing, by one or more processors, a corpus of documents;

determining, by the one or more processors, that a particular document in the corpus of documents includes multiple content pieces that each occur in at least one other earlier document in the corpus of documents;

determining, by the one or more processors, an extent to which the entire content of the particular document is made up of the multiple content pieces that each occur in the at least one other earlier document in the corpus of documents;

adjusting, by the one or more processors, a rank of a source of the particular document in relation to a source of other documents in the corpus of documents based at least in part on the extent to which the entire content of the particular document is made up of the multiple content pieces that each occur in the at least one other earlier document in the corpus of documents; and

configuring a web crawling or search result ranking process for the source of the particular document based at least in part on the adjusted rank.

9. The medium of claim 8, wherein configuring the web crawling process comprises configuring a frequency with which a web crawler crawls a web server associated with the source based at least on the rank.

10. The medium of claim 8, wherein adjusting the rank of the source comprises penalizing the rank of the source when the extent to which the entire content of the particular document is made up of the multiple content pieces that each occur in the at least one other earlier document exceeds a threshold, or promoting the rank of the source when the extent to which the entire content of the particular document is made up of the multiple content pieces that each occur in the at least one other earlier document does not exceed a threshold.

11. The medium of claim 8, wherein multiple content pieces represents a number of non-adjacent words.

12. The medium of claim 8, wherein the operations comprise fragmenting the particular document into a multiple content pieces.

13. The medium of claim 8, wherein the corpus of documents comprises documents that are classified as news-related documents.

14. The medium of claim 8, wherein the rank of the source is penalized in proportion to the extent to which the entire content of the particular document is made up of the multiple content pieces that each occur in the at least one other earlier document in the corpus of documents.

15. A system comprising:

one or more processors; and

a computer readable medium coupled to the data processing apparatus, having instructions stored thereon which, when executed by the data processing apparatus, cause the data processing apparatus to perform operations comprising;

accessing, by one or more processors, a corpus of documents;

determining, by the one or more processors, that a particular document in the corpus of documents includes multiple content pieces that each occur in at least one other earlier document in the corpus of documents;

determining, by the one or more processors, an extent to which the entire content of the particular document is made up of the multiple content pieces that each occur in the at least one other earlier document in the corpus of documents;

adjusting, by the one or more processors, a rank of a source of the particular document in relation to a source of other documents in the corpus of documents based at least in part on the extent to which the entire content of the particular document is made up of the multiple content pieces that each occur in the at least one other earlier document in the corpus of documents; and

configuring a web crawling or search result ranking process for the source of the particular document based at least in part on the adjusted rank.

16. The system of claim 15, wherein configuring the web crawling process comprises configuring a frequency with which a web crawler crawls a web server associated with the source based at least on the rank.

17. The system of claim 15, wherein adjusting the rank of the source comprises penalizing the rank of the source when the extent to which the entire content of the particular document is made up of the multiple content pieces that each occur in the at least one other earlier document exceeds a threshold, or promoting the rank of the source when the extent to which the entire content of the particular document is made up of the multiple content pieces that each occur in the at least one other earlier document does not exceed a threshold.

18. The system of claim 15, wherein multiple content pieces represents a number of non-adjacent words.

19. The system of claim 15, wherein the operations comprise fragmenting the particular document into a multiple content pieces.

20. The system of claim 15, wherein the corpus of documents comprises documents that are classified as news-related documents.