Estimating ad quality from observed user behavior

• US 8,065,184 B2
• Filed: 12/30/2005
• Issued: 11/22/2011
• Est. Priority Date: 12/30/2005
• Status: Active Grant

• Current Assignee: Google LLC (Alphabet Inc.)
• Original Assignee: Google Inc. (Alphabet Inc.)
• Inventors: Pregibon, Daryl, Wright, Daniel, Tang, Diane
• Primary Examiner(s): Weiss; John
• Assistant Examiner(s): VANDERHORST, MARIA VICTORIA
• Application Number:
  US11/321,076
• Publication Number:
  US 20070156514A1
• Time in Patent Office: 2,153 Days
• Field of Search: 705/14.41
• US Class Current: 705/14.41
• CPC Class Codes: G06Q 30/02 Marketing; Price estimation or determination; Fundraising
• CPC Class Codes: G06Q 30/0242 Determining effectiveness of advertisements
• CPC Class Codes: G06Q 30/0253 During e-commerce, i.e. online transactions
• CPC Class Codes: G06Q 30/0254 based on statistics
• CPC Class Codes: G06Q 30/0257 User requested
• CPC Class Codes: G06Q 30/0272 Period of advertisement exposure

Abstract

A system obtains ratings associated with a first set of advertisements hosted by one or more servers, where the ratings indicate a quality of the first set of advertisements. The system observes multiple different first user actions associated with user selection of advertisements of the first set of advertisements and derives a statistical model using the observed first user actions and the obtained ratings. The system further observes second user actions associated with user selection of a second advertisement hosted by the one or more servers and uses the statistical model and the second user actions to estimate a quality of the second advertisement.

Claims

1. A method implemented by one or more processors of a computer system, comprising:

obtaining, using one or more processors of the computer system, ratings associated with a set of advertisements, where the ratings, which include a measure of quality of the set of advertisements, are received by manual input from human raters;

logging, using one or more processors of the computer system, first user actions associated with user selection of advertisements of the set of advertisements, the first user actions representing a measure of user satisfaction with regard to the selected advertisements;

logging, using one or more processors of the computer system, second user actions associated with user selection of an unrated advertisement of a set of unrated advertisements;

calculating, by one or more processors of the computer system, a probability that the unrated advertisement is of a certain measure of quality based on the logged second user actions and on a probability generated by a probability model that operates based on the logged first user actions and the obtained ratings.

2. The method of claim 1, further comprising;

testing a fit of the probability model to determine which of the logged first user actions are correlated with advertisements of a certain measure of quality.

3. The method of claim 1, further comprising:

calculating a quality score associated with the unrated advertisement using the calculated probability; and

estimating a measure of quality of the unrated advertisement based on the calculated quality score.

4. The method of claim 3, where calculating the quality score associated with the unrated advertisement comprises:

applying a function to the calculated probability to calculate the quality score.

5. The method of claim 1, where the first user actions or the second user actions each comprise at least one of the following:

a duration of the selection of the selected rated and unrated advertisements;

a number of selections of other advertisements before or after the selection of the selected rated and unrated advertisements;

a number of selections of search results before or after the selection of the selected rated and unrated advertisements;

a number of selections of other types of results before or after the selection of the selected rated and unrated advertisements;

a number of document accesses before or after the selection of the selected rated and unrated advertisements;

a number of search queries before or after the selection of the selected rated and unrated advertisements;

a number of search queries associated with a user session that shows advertisements;

a number of repeat selections on a same given advertisement of the selected rated and unrated advertisements;

or

an indication of whether a given advertisement, of the selected rated and unrated advertisements, was a last advertisement selection for a given query or a last selection in a user session.

6. The method of claim 1, where the logged first user actions include a period of time from the selection of the selected advertisements until a next user action;

or

the logged second user actions include a period of time from the selection of the unrated advertisement to a next user action.

7. The method of claim 6, where the period of time from the selection of the selected advertisements until a next user action or the period of time from the selection of the unrated advertisement to a next user action is a measure of user satisfaction with regard to the selected advertisements or the selected unrated advertisement.

8. A method implemented by one or more processors of a computer system, comprising:

obtaining, using one or more processors of the computer system, first session features associated with user selections of first advertisements hosted by one or more servers, the first session features representing a measure of user satisfaction with the selected first advertisements;

correlating, using one or more processors of the computer system, known quality ratings, associated with the first advertisements, with the first session features;

obtaining, using one or more processors of the computer system, second session features associated with user selection of a second unrated advertisement; and

estimating, using one or more processors of the computer system, a quality rating of the second unrated advertisement based on the obtained second session features and a probability generated by a probability model that operates based on the obtained first session features and the correlated quality ratings.

9. The method of claim 8, where the first session features comprise at least one of the following:

a duration of the selection of the selected first advertisements;

a number of selections on other advertisements before or after the selection of the selected first advertisements;

a number of selections of search results before or after the selection of the selected first advertisements;

a number of selections of other types of results before or after the selection of the selected first advertisements;

a number of document accesses before or after the selection of the selected first advertisements;

a number of search queries before or after the selection of the selected first advertisements;

a number of search queries associated with a user session that shows advertisements;

a number of repeat selections on a same given advertisement of the selected first advertisements;

or

an indication of whether a given advertisement, of the selected first advertisements, was a last advertisement selection for a given query or a last selection in a user session.

10. The method of claim 8, further comprising:

testing a fit of the probability model to determine which of the obtained first session features are correlated with advertisements of at least a certain measure of quality.

11. The method of claim 8, further comprising:

predicting a quality rating for another unrated advertisement based on the calculated quality rating of the second unrated advertisement.

12. A system, comprising:

one or more processors to;

obtain a set of rated;

advertisements in response to a user query;

obtain first session features associated with user selection of advertisements of the set of rated advertisements, the first session features representing a measure of user satisfaction with the selected rated advertisements;

obtain second session features associated with user selection of an unrated advertisement of a set of unrated advertisements; and

determine a probability that the unrated advertisement is of a certain measure of quality based on the second session features and a probability generated by a probability model that is based on the obtained first session features and ratings associated with the selected rated advertisements;

calculate a quality score for the unrated advertisement based on the calculated probability; and

promote, rank, or filter the unrated advertisement based on the calculated score.

13. The system of claim 12, where the first session features or the second session features each comprise at least one of the following:

a duration of the selection of the selected rated and unrated advertisements;

a number of selections of other advertisements before or after the selection of the selected rated and unrated advertisements;

a number of selections of search results before or after the selection of the selected rated and unrated advertisements a given advertisement selection;

a number of selections of other types of results before or after the selection of the selected rated and unrated advertisements;

a number of document accesses before or after the selection of the selected rated and unrated advertisements;

a number of search queries before or after the selection of the selected rated and unrated advertisements;

a number of search queries associated with a user session that shows advertisements;

a number of repeat selections on a same given advertisement of the selected rated and unrated advertisements;

or

an indication of whether a given advertisement, of the selected rated and unrated advertisements, was a last advertisement selection for a given query or a last selection in a user session.

14. The system of claim 12, where the one or more processors further are to:

predict a quality score for another unrated advertisement based on the calculated quality score of the unrated advertisement.