Scoring criteria for a content item

• US 9,501,549 B1
• Filed: 04/28/2014
• Issued: 11/22/2016
• Est. Priority Date: 04/28/2014
• Status: Active Grant

• Current Assignee: Google LLC (Alphabet Inc.)
• Original Assignee: Google Inc. (Alphabet Inc.)
• Inventors: Zhao, Gaofeng, Wang, Xuerui
• Primary Examiner(s): Burke, Jeffrey A
• Assistant Examiner(s): Vu, Thong
• Application Number:
  US14/263,554
• Time in Patent Office: 939 Days
• Field of Search: 707/748, 707/740, 707/769, 707/756, 382/158, 386/278
• US Class Current: 1/1
• CPC Class Codes: G06F 16/28 Databases characterised by their database models, e.g. relational or object models
• CPC Class Codes: G06F 16/951 Indexing; Web crawling techniques

Abstract

Methods, systems, and apparatus, including computer programs encoded on a computer storage medium, for scoring criteria for content items. In one aspect, a method includes identifying a primary ranking signal and a set of auxiliary ranking signals for ranking a set of criteria for a content item. A primary score and a set of auxiliary scores can be identified for each particular criterion in the set of criteria. Each auxiliary score can be adjusted to generate adjusted auxiliary scores. The adjusting can include applying, to at least a portion of the auxiliary scores, a transformation function that reduces an amount of skewness among the auxiliary scores. A ranking score can be determined for each particular criterion based on a function of the primary score for the particular criterion and the adjusted auxiliary scores.

Claims

1. A method performed by data processing apparatus, the method comprising:

identifying a primary ranking signal and a set of auxiliary ranking signals for ranking a set of criteria for a content item, wherein the primary ranking signal defines a first attribute for one of the criteria in the set of criteria and each auxiliary ranking signal in the set of auxiliary ranking signals a) defines a second attribute for one of the criteria in the set of criteria and b) is different than the primary ranking signal;

for each particular criterion in the set of criteria;

identifying a primary score representing a value of the primary ranking signal for the particular criterion;

identifying a set of auxiliary scores for the particular criterion, each auxiliary score representing a value of an auxiliary ranking signal of the set of auxiliary ranking signals for the particular criterion;

adjusting each auxiliary score in the set of auxiliary scores to generate adjusted auxiliary scores, the adjusting comprising applying, to at least a portion of the auxiliary scores, a transformation function that reduces an amount of skewness among the auxiliary scores that are associated with a particular auxiliary ranking signal from the set of auxiliary ranking signals; and

determining a ranking score for the particular criterion based on a function of the primary score for the particular criterion and the adjusted auxiliary scores, the function suppressing effects of adjusted auxiliary scores for the particular criterion that do not satisfy a specified value and boosting the ranking score using adjusted auxiliary scores for the particular criterion that satisfy the specified value;

selecting one or more criteria in the set of criteria for which to associate the content item based on the ranking score for each particular criterion; and

providing the content item in response to receiving data specifying one or more of the selected criteria.

2. The method of claim 1, wherein determining the ranking score for the particular criterion comprises combining the transformed auxiliary scores to generate a combined auxiliary score, wherein the function comprises a function of the primary score and the combined auxiliary score.

3. The method of claim 2, further comprising:

for at least one of the particular criterion in the set of criteria;

determining whether the combined auxiliary score satisfies a specified value, wherein determining the ranking score for the particular criterion comprises increasing the ranking score in response to determining that the combined auxiliary score satisfies the specified value.

4. The method of claim 2, wherein determining the ranking score for the particular criterion comprises:

adding the combined auxiliary score to a constant value to generate an adjusted auxiliary score; and

determining a product of the adjusted auxiliary score and the primary score.

5. The method of claim 1, further comprising:

assigning the auxiliary scores to two or more groups based on a type of auxiliary ranking signal for each auxiliary score;

determining, for each group, a combined auxiliary score for the group based on auxiliary scores assigned to the group; and

applying a respective weighting factor to the combined auxiliary score for the group, wherein the respective weighting factor for a particular group is based on relative importance of the group'"'"'s auxiliary scores, and wherein the function comprises a function of the primary score and the combined auxiliary score for each group.

6. The method of claim 1, further comprising:

determining, for each particular auxiliary ranking signal, a maximum value of auxiliary scores for the auxiliary ranking signal, the maximum value being based on values of the auxiliary scores for the auxiliary ranking signal across the set of criteria and being less than a highest auxiliary value for the auxiliary ranking signal across the set of criteria;

identifying a particular auxiliary score for the auxiliary ranking signal that represents a value for the auxiliary ranking signal and that has a value that exceeds the maximum value; and

reducing the particular auxiliary score to a value that is equal to or less than the maximum value.

7. The method of claim 1, further comprising normalizing auxiliary scores for each auxiliary ranking signal.

8. A system comprising:

a data store for storing primary scores and auxiliary scores for a set of criteria; and

one or more processors configured to interact with the data store, the one or more processors being further configured to perform operations comprising;

identifying a primary ranking signal and a set of auxiliary ranking signals for ranking the set of criteria for a content item, wherein the primary ranking signal defines a first attribute for one of the criteria in the set of criteria and each auxiliary ranking signal in the set of auxiliary ranking signals a) defines a second attribute for one of the criteria in the set of criteria and b) is different than the primary ranking signal;

for each particular criterion in the set of criteria;

identifying a primary score representing a value of the primary ranking signal for the particular criterion;

identifying a set of auxiliary scores for the particular criterion, each auxiliary score representing a value of an auxiliary ranking signal of the set of auxiliary ranking signals for the particular criterion;

adjusting each auxiliary score in the set of auxiliary scores to generate adjusted auxiliary scores, the adjusting comprising applying, to at least a portion of the auxiliary scores, a transformation function that reduces an amount of skewness among the auxiliary scores that are associated with a particular auxiliary ranking signal from the set of auxiliary ranking signals; and

determining a ranking score for the particular criterion based on a function of the primary score for the particular criterion and the adjusted auxiliary scores, the function suppressing effects of adjusted auxiliary scores for the particular criterion that do not satisfy a specified value and boosting the ranking score using adjusted auxiliary scores for the particular criterion that satisfy the specified value;

selecting one or more criteria in the set of criteria for which to associate the content item based on the ranking score for each particular criterion; and

providing the content item in response to receiving data specifying one or more of the selected criteria.

9. The system of claim 8, wherein determining the ranking score for the particular criterion comprises combining the transformed auxiliary scores to generate a combined auxiliary score, wherein the function comprises a function of the primary score and the combined auxiliary score.

10. The system of claim 9, wherein the one or more processors are further configured to perform further operations comprising:

for at least one of the particular criterion in the set of criteria;

determining whether the combined auxiliary score satisfies a specified value, wherein determining the ranking score for the particular criterion comprises increasing the ranking score in response to determining that the combined auxiliary score satisfies the specified value.

11. The system of claim 9, wherein determining the ranking score for the particular criterion comprises:

adding the combined auxiliary score to a constant value to generate an adjusted auxiliary score; and

determining a product of the adjusted auxiliary score and the primary score.

12. The system of claim 8, wherein the one or more processors are further configured to perform further operations comprising:

assigning the auxiliary scores to two or more groups based on a type of auxiliary ranking signal for each auxiliary score;

determining, for each group, a combined auxiliary score for the group based on auxiliary scores assigned to the group; and

applying a respective weighting factor to the combined auxiliary score for the group, wherein the respective weighting factor for a particular group is based on relative importance of the group'"'"'s auxiliary scores, and wherein the function comprises a function of the primary score and the combined auxiliary score for each group.

13. The system of claim 8, wherein the one or more processors are further configured to perform further operations comprising:

determining, for each particular auxiliary ranking signal, a maximum value of auxiliary scores for the auxiliary ranking signal, the maximum value being based on values of the auxiliary scores for the auxiliary ranking signal across the set of criteria and being less than a highest auxiliary value for the auxiliary ranking signal across the set of criteria;

identifying a particular auxiliary score for the auxiliary ranking signal that represents a value for the auxiliary ranking signal and that has a value that exceeds the maximum value; and

reducing the particular auxiliary score to a value that is equal to or less than the maximum value.

14. The system of claim 8, wherein the one or more processors are further configured to perform further operations comprising normalizing auxiliary scores for each auxiliary ranking signal.

15. A computer storage medium encoded with a computer program, the program comprising instructions that when executed by data processing apparatus cause the data processing apparatus to perform operations comprising:

identifying a primary ranking signal and a set of auxiliary ranking signals for ranking a set of criteria for a content item, wherein the primary ranking signal defines a first attribute for one of the criteria in the set of criteria and each auxiliary ranking signal in the set of auxiliary ranking signals a) defines a second attribute for one of the criteria in the set of criteria and b) is different than the primary ranking signal;

for each particular criterion in the set of criteria;

identifying a primary score representing a value of the primary ranking signal for the particular criterion;

identifying a set of auxiliary scores for the particular criterion, each auxiliary score representing a value of an auxiliary ranking signal of the set of auxiliary ranking signals for the particular criterion;

adjusting each auxiliary score in the set of auxiliary scores to generate adjusted auxiliary scores, the adjusting comprising applying, to at least a portion of the auxiliary scores, a transformation function that reduces an amount of skewness among the auxiliary scores that are associated with a particular auxiliary ranking signal from the set of auxiliary ranking signals; and

determining a ranking score for the particular criterion based on a function of the primary score for the particular criterion and the adjusted auxiliary scores, the function suppressing effects of adjusted auxiliary scores for the particular criterion that do not satisfy a specified value and boosting the ranking score using adjusted auxiliary scores for the particular criterion that satisfy the specified value;

selecting one or more criteria in the set of criteria for which to associate the content item based on the ranking score for each particular criterion; and

providing the content item in response to receiving data specifying one or more of the selected criteria.

16. The computer storage medium of claim 15, wherein determining the ranking score for the particular criterion comprises combining the transformed auxiliary scores to generate a combined auxiliary score, wherein the function comprises a function of the primary score and the combined auxiliary score.

17. The computer storage medium of claim 16, further comprising:

for at least one of the particular criterion in the set of criteria;

determining whether the combined auxiliary score satisfies a specified value, wherein determining the ranking score for the particular criterion comprises increasing the ranking score in response to determining that the combined auxiliary score satisfies the specified value.

18. The computer storage medium of claim 16, wherein determining the ranking score for the particular criterion comprises:

adding the combined auxiliary score to a constant value to generate an adjusted auxiliary score; and

determining a product of the adjusted auxiliary score and the primary score.

19. The computer storage medium of claim 15, wherein the instructions, when executed by the data processing apparatus, cause the data processing apparatus to perform further operations comprising:

assigning the auxiliary scores to two or more groups based on a type of auxiliary ranking signal for each auxiliary score;

determining, for each group, a combined auxiliary score for the group based on auxiliary scores assigned to the group; and

applying a respective weighting factor to the combined auxiliary score for the group, wherein the respective weighting factor for a particular group is based on relative importance of the group'"'"'s auxiliary scores, and wherein the function comprises a function of the primary score and the combined auxiliary score for each group.

20. The computer storage medium of claim 15, wherein the instructions, when executed by the data processing apparatus, cause the data processing apparatus to perform further operations comprising:

determining, for each particular auxiliary ranking signal, a maximum value of auxiliary scores for the auxiliary ranking signal, the maximum value being based on values of the auxiliary scores for the auxiliary ranking signal across the set of criteria and being less than a highest auxiliary value for the auxiliary ranking signal across the set of criteria;

identifying a particular auxiliary score for the auxiliary ranking signal that represents a value for the auxiliary ranking signal and that has a value that exceeds the maximum value; and

reducing the particular auxiliary score to a value that is equal to or less than the maximum value.