MULTI-CAMPAIGN CONTENT ALLOCATION

• US 20130254015A1
• Filed: 05/16/2013
• Published: 09/26/2013
• Est. Priority Date: 12/31/2009
• Status: Active Grant

• Current Assignee: Google LLC (Alphabet Inc.)
• Original Assignee: Google Inc. (Alphabet Inc.)
• Inventors: Juda, Adam I., Baugher, Alexandra, Meyer, Michael M., Desikan, Pavan K., Mirkin, Ilia, Black, Julie A.
• Granted Patent: US 9,460,449 B2
• Time in Patent Office: Days
• US Class Current: 705/14.42
• CPC Class Codes: G06Q 30/02 Marketing; Price estimation or determination; Fundraising
• CPC Class Codes: G06Q 30/0243 Comparative campaigns
• CPC Class Codes: G06Q 30/0244 Optimization

Abstract

Methods, systems, and apparatus, including computer programs encoded on a computer storage medium, for creating experiment content distribution campaigns (“experiment campaign”) and allocating content in accordance with the experiment campaign. In one aspect, a control campaign or an experiment campaign are selected as an available campaign for providing content in response to receiving a content request. The selection can be a probability constrained selection where the probability with which the experiment campaign is selected is specified as a portion of a total number of requests received during an experiment period for which the experiment campaign is available to provide content. Content is provided in response to the request, where the provided content being content corresponding to the selected campaign.

Claims

1-24. -24. (canceled)

25. A method, comprising:

receiving, by one or more data processing apparatus, a request to create an experiment campaign based on an existing campaign;

receiving, by one or more data processing apparatus, data specifying an experiment parameter in the experiment campaign that differs from existing parameters of the existing campaign;

storing, by one or more data processing apparatus, a set of experiment difference data that excludes at least one of the existing parameters and species the difference between the experiment parameter and the existing parameters;

receiving, by one or more data processing apparatus, a request for content;

identifying, in response to the request, a set of experiment parameters for the experiment campaign based on a combination of the existing parameters and the experiment difference data; and

providing, in response to the request, content based on the set of experiment parameters.

26. The method of claim 25, comprising:

identifying, from among multiple different experiment campaigns that have been created for the existing campaign, one experiment campaign that will be used to provide content in response to the request, wherein;

identifying a set of experiment parameters comprises identifying a set of experiment parameters for the one experiment campaign based on a combination of the existing parameters and the experiment difference data; and

providing content comprises providing content based on the experiment parameters for the one experiment campaign.

27. The method of claim 26, wherein identifying the one experiment campaign comprises pseudo-randomly selecting the one experiment campaign from among the multiple different experiment campaigns.

28. The method of claim 26, comprising:

determining, for each of two or more of the multiple different experiment campaigns, a performance measure specifying a user interaction rate for content provided using the experiment campaign; and

providing data that cause the performance measure for each of the multiple different experiment campaigns and a performance measure for the existing campaign in a user interface.

29. The method of claim 28, comprising providing data that cause presentation of a variation identifier specifying that a difference between performance measures for the existing campaign and one of the multiple different experiment campaigns is greater than a pre-specified difference.

30. The method of claim 29, wherein the data cause the variation identifier to be presented next to the performance measure for the one of the multiple different experiment campaigns.

31. The method of claim 25, comprising:

receiving a start date for the experiment campaign specifying a date at which the experiment campaign becomes available for distributing content;

determining that the start date for the experiment campaign has been reached; and

activating the experiment campaign, wherein the identifying and providing are performed after activating the experiment campaign.

32. A system comprising:

a data store storing existing parameters for an existing campaign; and

one or more data processing apparatus that interact with the data store and execute instructions that cause the one or more data processing apparatus to perform operations comprising;

receiving a request to create an experiment campaign based on the existing campaign;

receiving data specifying an experiment parameter in the experiment campaign that differs from the existing parameters of the existing campaign;

storing, in the data store, a set of experiment difference data that excludes at least one of the existing parameters and species the difference between the experiment parameter and the existing parameters;

receiving a request for content;

identifying, in response to the request, a set of experiment parameters for the experiment campaign based on a combination of the existing parameters and the experiment difference data; and

providing, in response to the request, content based on the set of experiment parameters.

33. The system of claim 32, wherein:

the instructions cause the one or more data processing apparatus to perform operations comprising identifying, from among multiple different experiment campaigns that have been created for the existing campaign, one experiment campaign that will be used to provide content in response to the request;

identifying a set of experiment parameters comprises identifying a set of experiment parameters for the one experiment campaign based on a combination of the existing parameters and the experiment difference data; and

providing content comprises providing content based on the experiment parameters for the one experiment campaign.

34. The system of claim 33, wherein identifying the one experiment campaign comprises pseudo-randomly selecting the one experiment campaign from among the multiple different experiment campaigns.

35. The system of claim 33, wherein the instructions cause the one or more data processing apparatus to perform operations comprising:

determining, for each of two or more of the multiple different experiment campaigns, a performance measure specifying a user interaction rate for content provided using the experiment campaign; and

providing data that cause the performance measure for each of the multiple different experiment campaigns and a performance measure for the existing campaign in a user interface.

36. The system of claim 35, wherein the instructions cause the one or more data processing apparatus to perform operations comprising providing data that cause presentation of a variation identifier specifying that a difference between performance measures for the existing campaign and one of the multiple different experiment campaigns is greater than a pre-specified difference.

37. The system of claim 36, wherein the data cause the variation identifier to be presented next to the performance measure for the one of the multiple different experiment campaigns.

38. The system of claim 32, wherein the instructions cause the one or more data processing apparatus to perform operations comprising:

receiving a start date for the experiment campaign specifying a date at which the experiment campaign becomes available for distributing content;

determining that the start date for the experiment campaign has been reached; and

activating the experiment campaign, wherein the identifying and providing are performed after activating the experiment campaign.

39. A non-transitory computer readable medium storing instructions that upon execution of the instructions by one or more data processing apparatus cause the one or more data processing apparatus to perform operations comprising:

receiving a request to create an experiment campaign based on an existing campaign;

receiving data specifying an experiment parameter in the experiment campaign that differs from the existing parameters of the existing campaign;

storing, in a data store, a set of experiment difference data that excludes at least one of the existing parameters and species the difference between the experiment parameter and the existing parameters;

receiving a request for content;

identifying, in response to the request, a set of experiment parameters for the experiment campaign based on a combination of the existing parameters and the experiment difference data; and

providing, in response to the request, content based on the set of experiment parameters.

40. The computer readable medium of claim 39, wherein:

the instructions cause the one or more data processing apparatus to perform operations comprising identifying, from among multiple different experiment campaigns that have been created for the existing campaign, one experiment campaign that will be used to provide content in response to the request;

identifying a set of experiment parameters comprises identifying a set of experiment parameters for the one experiment campaign based on a combination of the existing parameters and the experiment difference data; and

providing content comprises providing content based on the experiment parameters for the one experiment campaign.

41. The computer readable medium of claim 40, wherein identifying the one experiment campaign comprises pseudo-randomly selecting the one experiment campaign from among the multiple different experiment campaigns.

42. The computer readable medium of claim 40, wherein the instructions cause the one or more data processing apparatus to perform operations comprising:

determining, for each of two or more of the multiple different experiment campaigns, a performance measure specifying a user interaction rate for content provided using the experiment campaign; and

providing data that cause the performance measure for each of the multiple different experiment campaigns and a performance measure for the existing campaign in a user interface.

43. The computer readable medium of claim 42, wherein the instructions cause the one or more data processing apparatus to perform operations comprising providing data that cause presentation of a variation identifier specifying that a difference between performance measures for the existing campaign and one of the multiple different experiment campaigns is greater than a pre-specified difference.

44. The computer readable medium of claim 39, wherein the instructions cause the one or more data processing apparatus to perform operations comprising:

receiving a start date for the experiment campaign specifying a date at which the experiment campaign becomes available for distributing content;

determining that the start date for the experiment campaign has been reached; and

activating the experiment campaign, wherein the identifying and providing are performed after activating the experiment campaign.