Multi-campaign content allocation based on experiment difference data

• US 8,539,067 B2
• Filed: 05/11/2010
• Issued: 09/17/2013
• Est. Priority Date: 12/31/2009
• Status: Active Grant

• Current Assignee: Google LLC (Alphabet Inc.)
• Original Assignee: Google Inc. (Alphabet Inc.)
• Inventors: Juda, Adam I., Baugher, Alexandra, Meyer, Michael M., Desikan, Pavan K., Mirkin, Ilia, Black, Julie A.
• Primary Examiner(s): Lin, Kenny
• Assistant Examiner(s): Murray, Daniel C
• Application Number:
  US12/777,554
• Publication Number:
  US 20110161407A1
• Time in Patent Office: 1,225 Days
• Field of Search: 709/203, 709/224, 705/14.42, 705/14.43
• US Class Current: 709/224
• CPC Class Codes: G06Q 30/02 Marketing; Price estimation or determination; Fundraising
• CPC Class Codes: G06Q 30/0243 Comparative campaigns
• CPC Class Codes: G06Q 30/0244 Optimization

Abstract

Methods, systems, and apparatus, including computer programs encoded on a computer storage medium, for creating experiment content distribution campaigns (“experiment campaign”) and allocating content in accordance with the experiment campaign. In one aspect, a control campaign or an experiment campaign are selected as an available campaign for providing content in response to receiving a content request. The selection can be a probability constrained selection where the probability with which the experiment campaign is selected is specified as a portion of a total number of requests received during an experiment period for which the experiment campaign is available to provide content. Content is provided in response to the request, where the provided content being content corresponding to the selected campaign.

Claims

1. A computer-implemented method, comprising:

receiving, by a data processing apparatus, a content request, the request specifying one or more environment parameters corresponding to the environment for which the requested content is being requested;

identifying, by a data processing apparatus and based on the environment parameters, a content distribution account for which a control campaign is eligible to provide a content item in response to the request, the content distribution account also including an experiment campaign;

selecting, by a data processing apparatus and as an available campaign for the received content request, the experiment campaign from among the control campaign and the experiment campaign, the selection being based on a portion of content item requests for which the experiment campaign is required to be selected;

identifying, for the control campaign, a set of campaign parameters that control distribution of content in the control campaign;

identifying, for the experiment campaign, experiment difference data specifying differences between the set of campaign parameters and experiment parameters for the experiment campaign, the experiment parameters being parameters that control distribution of content in the experiment campaign, and the experiment difference data excluding one or more of the experiment parameters that match a campaign parameter in the set of campaign parameters;

determining the experiment parameters based on a combination of the set of campaign parameters and the experiment difference data; and

providing, by a data processing apparatus, content from the experiment campaign in response to the request, the content being provided according to the experiment parameters.

2. The method of claim 1, further comprising:

updating, by a data processing apparatus, performance measures for the available campaign corresponding to the provided content, the performance measures including a total number of presentations of the content and a total number of user selections of the content; and

providing, by a data processing apparatus, data that causes the updated performance measures to be presented on a presentation device.

3. The method of claim 2, wherein providing data that causes the updated performance measures to be presented comprises providing data that causes a total keyword performance to be provided for each keyword corresponding to the experiment campaign and the corresponding control campaign.

4. The method of claim 2, wherein providing data that causes the updated performance measures to be presented comprises providing data that causes presentation of delineated performance measures for at least one specified delineation of the experiment and control campaigns.

5. The method of claim 2, wherein providing data that causes the updated performance measures to be presented comprises providing data that causes presentation of an updated experiment performance at a presentation location that is adjacent to an updated control performance.

6. The method of claim 1, wherein only one of the experiment campaign and the control campaign is selected as the available campaign in response to a single content request.

7. The method of claim 1, wherein both the experiment campaign and the control campaign are available for selection as the available campaign in response to each content request.

8. The method of claim 1, further comprising:

in response to selecting the control campaign as the available campaign, selecting content to be provided in response to the request, the content being selected based on the set of campaign parameters.

9. The method of claim 1, further comprising:

in response to selecting the experiment campaign as the available campaign, selecting content to be provided in response the request, the content being selected based on the set of campaign parameters and the experiment difference data.

10. The method of claim 1, further comprising receiving a budget that specifies an amount that an advertiser is willing to pay for distribution of content over a specified budget period, wherein the budget is allocated to the experiment campaign in proportion to a portion of a total number of requests that are both received during an experiment period and in response to which the experiment campaign is available to provide content.

11. A system, comprising:

a data store storing a set of campaign parameters for a control campaign and experiment difference data for a corresponding experiment campaign, the experiment difference data being data specifying differences between experiment parameters for the experiment campaign and the set of campaign parameters for the control campaign, the experiment parameters being parameters that control distribution of content in the experiment campaign, and the experiment difference data excluding one of more of the experiment parameters that match a campaign parameter in the set of campaign parameters; and

an experiment subsystem coupled to the data store, the experiment subsystem comprising at least one processor configured to;

select, from among the control campaign and the experiment campaign and as an available campaign for a content request received during an experiment period, the experiment campaign;

determine, in response to selecting the experiment campaign as the available campaign, the experiment parameters based on a combination of the set of campaign parameters and the experiment difference data; and

provide content from the experiment campaign according to the experiment parameters.

12. The system of claim 11, wherein the experiment subsystem is further configured to receive a content request specifying one or more environment parameters corresponding to the environment for which the requested content is being requested and provide content in response to the request, the provided content being content corresponding to the selected available campaign.

13. The system of claim 11, wherein the experiment subsystem is further configured to update performance measures for the available campaign corresponding to the provided content, the performance measures including a total number of presentations of the content and a total number of user selections of the content, and provide data that causes the updated performance measures to be presented on a presentation device.

14. The system of claim 12, wherein the experiment subsystem is further configured to provide data that causes a total keyword performance to be provided for each keyword corresponding to the experiment campaign and the control campaign.

15. The system of claim 12, wherein the experiment subsystem is further configured to provide data that causes presentation of delineated performance measures for at least one specified delineation of the experiment and control campaigns.

16. The system of claim 12, wherein the experiment subsystem is further configured to provide data that causes presentation of an updated experiment performance at a presentation location that is adjacent to an updated control performance.

17. The system of claim 11, wherein the experiment subsystem is further configured to select only one of the experiment campaign and the control campaign, as the available campaign, in response to a single content request.

18. The system of claim 11, wherein the experiment subsystem is further configured to select content to be provided in response to the request, the content being selected based on the set of campaign parameters corresponding to the control campaign when the control campaign is selected as the available campaign for the received content request.

19. The system of claim 11, wherein the experiment subsystem is further configured to select content to be provided in response to the request, the content being selected based on the set of campaign parameters and the experiment difference data when the experiment campaign is selected as the available campaign for the received content request.

20. The system of claim 11, further comprising a user device configured to receive data that causes presentation of updated performance measures that have been generated by the experiment subsystem.

21. A non-transitory computer storage medium encoded with a computer program, the program comprising instructions that when executed by data processing apparatus cause the data processing apparatus to perform operations comprising:

receiving a content request, the request specifying one or more environment parameters corresponding to the environment for which the requested content is being requested;

identifying, based on the environment parameters, a content distribution account for which a control campaign is eligible to provide a content item in response to the request, the content distribution account also including an experiment campaign;

selecting, by a data processing apparatus and as an available campaign for the received content request, the experiment campaign from among the control campaign and the experiment campaign, the selection being based on a portion of content item requests for which the experiment campaign is required to be selected;

identifying, for the control campaign, a set of campaign parameters that control distribution of content in the control campaign;

identifying, for the experiment campaign, experiment difference data specifying differences between the set of campaign parameters and experiment parameters for the experiment campaign, the experiment parameters being parameters that control distribution of content in the experiment campaign, and the experiment difference data excluding one or more of the experiment parameters that match a campaign parameter in the set of campaign parameters;

determining the experiment parameters based on a combination of the set of campaign parameters and the experiment difference data; and

providing content from the experiment campaign in response to the request, the content being provided according to the experiment parameters.