Evaluating quality based on neighbor features

• US 9,183,499 B1
• Filed: 04/19/2013
• Issued: 11/10/2015
• Est. Priority Date: 04/19/2013
• Status: Active Grant

• Current Assignee: Google LLC (Alphabet Inc.)
• Original Assignee: Google Inc. (Alphabet Inc.)
• Inventors: Krivokon, Igor, Ofitserov, Vladimir, Kislyuk, Oleg
• Primary Examiner(s): Gaffin, Jeffrey A
• Assistant Examiner(s): Kim, David H
• Application Number:
  US13/866,589
• Time in Patent Office: 935 Days
• Field of Search: None
• US Class Current: 1/1
• CPC Class Codes: G06F 16/951 Indexing; Web crawling techniques
• CPC Class Codes: G06N 5/02 Knowledge representation; Symbolic representation

Abstract

Methods, systems, and apparatus for computing quality scores based on neighbor features. In one aspect, a method includes obtaining a quality model that was trained using a set of training entities; identifying a set of candidate entities that are different from each of the training entities; for each candidate entity: obtaining a first quality score for the candidate entity; obtaining one or more neighbor features for neighbor entities of the candidate entity, where each neighbor entity of the candidate entity is linked to the candidate entity; obtaining one or more entity specific feature values for the candidate entity, where each entity specific feature value is determined independent of the neighbor entities of the candidate entity; and determining a second quality score for the candidate entity using the quality model, the second quality score being computed based on the first quality score, the neighbor features, and the entity specific feature values.

Claims

1. A computer-implemented method, comprising:

obtaining a first quality model that was trained using a first set of training entities;

identifying a set of candidate entities, where each candidate entity is different from each of the training entities;

for each candidate entity in the set of candidate entities;

obtaining a first quality score for the candidate entity;

obtaining one or more neighbor features for neighbor entities of the candidate entity, where each neighbor entity of the candidate entity is an entity that is linked to the candidate entity;

obtaining one or more entity specific feature values for the candidate entity, where each entity specific feature value is determined independent of the neighbor entities of the candidate entity; and

determining a second quality score for the candidate entity using the first quality model, the second quality score being computed based on the first quality score, the neighbor features, and the entity specific feature values.

2. The method of claim 1, further comprising:

training a second quality model based on training features associated with the training entities, where the training features for each training entity include;

a given quality score for the training entity;

at least one neighbor quality score for at least one neighbor entity of the training entity; and

at least one entity specific feature value of the training entity;

wherein at least one neighbor entity of at least one training entity is a candidate entity, and wherein the second quality model provides, as output, a quality score for an entity.

3. The method of claim 1, wherein the first quality model was trained based on training features associated with the training entities, where the training features for each training entity include:

a given quality score for the training entity;

at least one neighbor quality score for at least one neighbor entity of the training entity; and

at least one entity specific feature value of the training entity.

4. The method of claim 1, further comprising:

identifying a link between a candidate entity and a neighbor entity, the link specifying that the neighbor entity includes a link that references the candidate entity.

5. The method of claim 1, further comprising:

identifying a link between a candidate entity and a neighbor entity, the link specifying that the neighbor entity and the candidate entity are each hosted by a single host data processing apparatus.

6. The method of claim 1, wherein obtaining one or more neighbor features for neighbor entities of the candidate entity comprises obtaining an average neighbor quality score, the average neighbor quality score specifying an average of one or more quality scores that correspond to the one or more of the neighbor entities.

7. The method of claim 1, wherein entity specific feature values comprise one or more of:

a layout score that indicates a quality associated with a visual layout of the entity;

a selection rate that indicates a rate at which the entity is selected when presented as a search result in response to a search query; and

a selection duration that indicates an average time the entity is displayed on a user device when selected.

8. The method of claim 1, wherein an entity is a set of websites that share a common characteristic, and wherein the common characteristic shared by each entity is different from each other common characteristic shared by each other entity.

9. The method of claim 1, wherein each entity is represented by a node in an irregular graph, and wherein an entity is linked to a candidate entity if the nodes representing the entity and the candidate entity share an edge in the irregular graph.

10. A system comprising:

a data processing apparatus; and

a data store storing instructions that, when executed by the data processing apparatus, cause the data processing apparatus to perform operations comprising;

obtaining a first quality model that was trained using a first set of training entities;

identifying a set of candidate entities, where each candidate entity is different from each of the training entities;

for each candidate entity in the set of candidate entities;

obtaining a first quality score for the candidate entity;

obtaining one or more neighbor features for neighbor entities of the candidate entity, where each neighbor entity of the candidate entity is an entity that is linked to the candidate entity;

obtaining one or more entity specific feature values for the candidate entity, where each entity specific feature value is determined independent of the neighbor entities of the candidate entity; and

determining a second quality score for the candidate entity using the first quality model, the second quality score being computed based on the first quality score, the neighbor features, and the entity specific feature values.

11. The system of claim 10, wherein the operations further comprise:

training a second quality model based on training features associated with the training entities, where the training features for each training entity include;

a given quality score for the training entity;

at least one neighbor quality score for at least one neighbor entity of the training entity; and

at least one entity specific feature value of the training entity;

wherein at least one neighbor entity of at least one training entity is a candidate entity, and wherein the second quality model provides, as output, a quality score for an entity.

12. The system of claim 10, wherein the first quality model was trained based on training features associated with the training entities, where the training features for each training entity include:

a given quality score for the training entity;

at least one neighbor quality score for at least one neighbor entity of the training entity; and

at least one entity specific feature value of the training entity.

13. The system of claim 10, wherein the operations further comprise:

identifying a link between a candidate entity and a neighbor entity, the link specifying that the neighbor entity includes a link that references the candidate entity.

14. The system of claim 10, wherein the operations further comprise:

identifying a link between a candidate entity and a neighbor entity, the link specifying that the neighbor entity and the candidate entity are each hosted by a single host data processing apparatus.

15. The system of claim 10, wherein obtaining one or more neighbor features for neighbor entities of the candidate entity comprises obtaining an average neighbor quality score, the average neighbor quality score specifying an average of one or more quality scores that correspond to the one or more of the neighbor entities.

16. A system comprising:

a data processing apparatus; and

a data store storing instructions that, when executed by the data processing apparatus, cause the data processing apparatus to perform operations comprising;

obtaining a first quality model that was trained using a first set of training entities;

identifying a set of candidate entities, where each candidate entity is different from each of the training entities;

for each candidate entity in the set of candidate entities;

obtaining a first quality score for the candidate entity;

obtaining one or more neighbor features for neighbor entities of the candidate entity, where each neighbor entity of the candidate entity is an entity that is linked to the candidate entity;

obtaining one or more entity specific feature values for the candidate entity, where each entity specific feature value is determined independent of the neighbor entities of the candidate entity; and

determining a second quality score for the candidate entity using the first quality model, the second quality score being computed based on the first quality score, the neighbor features, and the entity specific feature values.

17. The system of claim 16, wherein the operations further comprise:

training a second quality model based on training features associated with the training entities, where the training features for each training entity include;

a given quality score for the training entity;

at least one neighbor quality score for at least one neighbor entity of the training entity; and

at least one entity specific feature value of the training entity;

wherein at least one neighbor entity of at least one training entity is a candidate entity, and wherein the second quality model provides, as output, a quality score for an entity.

18. The system of claim 16, wherein the first quality model was trained based on training features associated with the training entities, where the training features for each training entity include:

a given quality score for the training entity;

at least one neighbor quality score for at least one neighbor entity of the training entity; and

at least one entity specific feature value of the training entity.

19. The system of claim 16, wherein the operations further comprise:

identifying a link between a candidate entity and a neighbor entity, the link specifying that the neighbor entity includes a link that references the candidate entity.

20. The system of claim 16, wherein obtaining one or more neighbor features for neighbor entities of the candidate entity comprises obtaining an average neighbor quality score, the average neighbor quality score specifying an average of one or more quality scores that correspond to the one or more of the neighbor entities.