Determining a quality measure for a resource

• US 9,558,233 B1
• Filed: 12/31/2012
• Issued: 01/31/2017
• Est. Priority Date: 11/30/2012
• Status: Active Grant

• Current Assignee: Google LLC (Alphabet Inc.)
• Original Assignee: Google Inc. (Alphabet Inc.)
• Inventors: Ng, Kien, Kim, Hyung-Jin, Haahr, Paul, Kwok, Chung Tin, Hammad, Moustafa A., Karanjkar, Sushrut
• Primary Examiner(s): Beausoliel, Jr., Robert
• Assistant Examiner(s): Santos, Pedro J
• Application Number:
  US13/731,354
• Time in Patent Office: 1,492 Days
• Field of Search: 707/E17.108, 707/999.003, 707/723, 707/748, 707/706, 707/722, 707/749, 707/709, 707/713, 707/705, 707/726, 707/707, 707/751, 707/710, 707/E17.199, 706/14
• US Class Current: 1/1
• CPC Class Codes: G06F 16/00 Information retrieval; Database structures therefor; File system structures therefor
• CPC Class Codes: G06F 16/24 Querying
• CPC Class Codes: G06F 16/951 Indexing; Web crawling techniques

Abstract

Methods, systems, and apparatus, including computer programs encoded on a computer storage medium, for determining a measure of quality for a resource. In one aspect, a method includes determining a seed score for each seed resource in a set. The seed score for a seed resource can be based on a number of resources that include a link to the seed resource and a number of selections of the links A set of source resources is identified. A source score is determined for each source resource. The source score for a source resource is based on the seed score for each seed resource linked to by the source resource. Source-referenced resources are identified. A resource score is determined for each source-referenced resource. The resource score for a source-referenced resource can be based on the source score for each source resource that includes a link to the source-referenced resource.

Claims

1. A method performed by a data processing apparatus, the method comprising:

determining a seed score for each seed resource in a set of seed resources in accordance with a seed selection quality score,

the seed score for a seed resource being based on a total number of resources that each include a link to the seed resource and a total number of selections of the links to the seed resource, each selection counted in the total number of selections being a selection by a respective user causing a navigation to the seed resource, and

the seed selection quality score representing qualities of selections of the links to the seed resource;

identifying a set of source resources, each source resource in the set of source resources being a resource that includes a link to at least one of the seed resources;

for each source resource, determining a source score, the source score for the source resource being based on the seed score for each seed resource linked to by a link in the source resource;

identifying, for each source resource, source-referenced resources, each source-referenced resource being a resource linked to by a link in the source resource, and wherein the source-referenced resources include at least one resource that is not in the set of seed resources; and

,

for each source-referenced resource, determining a resource score, the resource score for the source-referenced resource being based on the source score for each source resource that includes a link to the source-referenced resource.

2. The method of claim 1, wherein the seed resources, source resources, and the source-referenced resources each comprise a web page.

3. The method of claim 1, wherein the seed score for each seed resource is based on a ratio between the total number of selections of the links to the seed resource and the total number of resources that each include a link to the seed resource.

4. The method of claim 1, wherein determining a seed score for each seed resource comprises:

determining a selection quality score for each selection of a link to the seed resource, the selection quality score being a measure of quality of the selection of the link to the seed resource;

determining the seed selection quality score based on the selection quality scores; and

determining the seed score for the seed resource based on the seed selection quality score, the total number of resources that each include a link to the seed resource, and the total number of selections of the links to the seed resource,

wherein the selection quality scores determined for the selections of the links include at least two distinct values.

5. The method of claim 4, wherein the quality measure for each selection causing a navigation to a respective resource is based, in part, on a click duration determined for the selection, wherein the click duration represents a length of time the user making the selection dwelled on the resource after making the selection.

6. The method of claim 5, wherein the selection quality score for each seed resource is based on at least one of a total number of selections having a click duration classified as a short click, a total number of the selections having a click duration classified as a medium click, or a total number of the selections having a click duration classified as a long click.

7. The method of claim 1, further comprising adjusting a search result ranking of at least one of the source-referenced resources for a particular search query, the adjusting being based on the resource score for the at least one source-referenced resource.

8. The method of claim 1, further comprising:

classifying each source resource having a source score that does not exceed a source score threshold as an unqualified resource;

for each source-referenced resource for which at least one unqualified includes a link;

identifying a total number of source resources classified as unqualified resources that include a link to the source-referenced resource; and

,

in a ranking process that ranks the source-referenced resource based, at least in part on the total number of source resources that include a link to the source-referenced resource, discounting the total number of source resources for the ranking process by the total number of source resources classified as unqualified resources that include a link to the source-referenced resource.

9. The method of claim 1, further comprising:

for each source resource;

identifying anchor text for each link to another resource included in the source resource;

identifying a set of n-grams for each identified anchor text by breaking the anchor text for each link into n-grams; and

,

assigning the source score for the source resource to each identified n-gram.

10. The method of claim 9, further comprising determining an n-gram score for each n-gram based on source scores assigned to the n-gram for each source resource having a link that includes the n-gram in anchor text for the link.

11. A system, comprising:

a data processing apparatus; and

,

a memory storage apparatus in data communication with the data processing apparatus, the memory storage apparatus storing instructions executable by the data processing apparatus and that upon such execution cause the data processing apparatus to perform operations comprising;

determining a seed score for each seed resource in a set of seed resources in accordance with a seed selection quality score,

the seed score for a seed resource being based on a total number of resources that each include a link to the seed resource and a total number of selections of the links to the seed resource, each selection counted in the total number of selections being a selection by a respective user causing a navigation to the seed resource, and

the seed selection quality score representing qualities of selections of the links to the seed resource;

identifying a set of source resources, each source resource in the set of source resources being a resource that includes a link to at least one of the seed resources;

for each source resource, determining a source score, the source score for the source resource being based on the seed score for each seed resource linked to by a link in the source resource;

identifying, for each source resource, source-referenced resources, each source-referenced resource being a resource linked to by a link in the source resource, and wherein the source-referenced resources include at least one resource that is not in the set of seed resources; and

,

for each source-referenced resource, determining a resource score, the resource score for the source-referenced resource being based on the source score for each source resource that includes a link to the source-referenced resource.

12. The system of claim 11, wherein the seed score for each seed resource is based on a ratio between the total number of selections of the links to the seed resource and the total number of resources that each include a link to the seed resource.

13. The system of claim 11, wherein determining a seed score for each seed resource comprises:

determining a selection quality score for each selection of a link to the seed resource, the selection quality score being a measure of quality of the selection of the link to the seed resource;

determining the seed selection quality score based on the selection quality scores; and

,

determining the seed score for the seed resource based on the seed selection quality score, the total number of resources that each include a link to the seed resource, and the total number of selections of the links to the seed resource,

wherein the selection quality scores determined for the selections of the links include at least two distinct values.

14. The system of claim 13, wherein the quality measure for each selection causing a navigation to a respective resource is based, in part, on a click duration determined for the selection, wherein the click duration represents a length of time the user making the selection dwelled on the resource after making the selection.

15. The system of claim 14, wherein the selection quality score for each seed resource is based on at least one of a total number of selections having a click duration classified as a short click, a total number of the selections having a click duration classified as a medium click, or a total number of the selections having a click duration classified as a long click.

16. The system of claim 11, wherein the instructions upon such execution cause the data processing apparatus to perform further operations comprising adjusting a search result ranking of at least one of the source-referenced resources for a particular search query, the adjusting being based on the resource score for the at least one source-referenced resource.

17. The system of claim 11, wherein the instructions upon such execution cause the data processing apparatus to perform further operations comprising:

classifying each source resource having a source score that does not exceed a source score threshold as an unqualified resource;

for each source-referenced resource for which at least one unqualified resource includes a link;

identifying a total number of source resources classified as unqualified resources that include a link to the source-referenced resource; and

,

in a ranking process that ranks the source-referenced resource based, at least in part on the total number of source resources that include a link to the source-referenced resource, discounting the total number of source resources for the ranking process by the total number of source resources classified as unqualified resources that include a link to the source-referenced resource.

18. The system of claim 11, wherein the instructions upon such execution cause the data processing apparatus to perform further operations comprising:

for each source resource;

identifying anchor text for each link to another resource included in the source resource;

identifying a set of n-grams for each identified anchor text by breaking the anchor text for each link into n-grams; and

,

assigning the source score for the source resource to each identified n-gram.

19. The system of claim 18, wherein the instructions upon such execution cause the data processing apparatus to perform further operations comprising determining an n-gram score for each n-gram based on source scores assigned to the n-gram for each source resource having a link that includes the n-gram in anchor text for the link.

20. A computer storage medium encoded with a computer program, the program comprising instructions that when executed by a data processing apparatus cause the data processing apparatus to perform operations comprising:

determining a seed score for each seed resource in a set of seed resources in accordance with a seed selection quality score,

the seed score for a seed resource being based on a total number of resources that each include a link to the seed resource and a total number of selections of the links to the seed resource, each selection counted in the total number of selections being a selection by a respective user causing a navigation to the seed resource, and

the seed selection quality score representing qualities of selections of the links to the seed resource;

identifying a set of source resources, each source resource in the set of source resources being a resource that includes a link to at least one of the seed resources;

for each source resource, determining a source score, the source score for the source resource being based on the seed score for each seed resource linked to by a link in the source resource;

identifying, for each source resource, source-referenced resources, each source-referenced resource being a resource linked to by a link in the source resource, and wherein the source-referenced resources include at least one resource that is not in the set of seed resources; and

,

for each source-referenced resource, determining a resource score, the resource score for the source-referenced resource being based on the source score for each source resource that includes a link to the source-referenced resource.

21. The computer storage medium of claim 20, wherein the seed score for each seed resource is based on a ratio between the total number of selections of the links to the seed resource and the total number of resources that each include a link to the seed resource.

22. The computer storage medium of claim 20, wherein determining a seed score for each seed resource comprises:

determining a selection quality score for each selection of a link to the seed resource, the selection quality score being a measure of quality of the selection of the link to the seed resource;

determining the seed selection quality score based on the selection quality scores; and

,

determining the seed score for the seed resource based on the seed selection quality score, the total number of resources that each include a link to the seed resource, and the total number of selections of the links to the seed resource,

wherein the selection quality scores determined for the selections of the links include at least two distinct values.

23. The computer storage medium of claim 20, wherein the instructions when executed by the data processing apparatus cause the data processing apparatus to perform further operations comprising adjusting a search result ranking of at least one of the source-referenced resources for a particular search query, the adjusting being based on the resource score for the at least one source-referenced resource.

24. The computer storage medium of claim 20, wherein the instructions when executed by the data processing apparatus cause the data processing apparatus to perform further operations comprising:

classifying each source resource having a source score that does not exceed a source score threshold as an unqualified resource;

for each source-referenced resource for which at least one unqualified resource includes a link;

identifying a total number of source resources classified as unqualified resources that include a link to the source-referenced resource; and

,

in a ranking process that ranks the source-referenced resource based, at least in part on the total number of source resources that include a link to the source-referenced resource, discounting the total number of source resources for the ranking process by the total number of source resources classified as unqualified resources that include a link to the source-referenced resource.