System and method for recommending a grammar for a message campaign used by a message optimization system
First Claim
1. A method performed by a computer system for recommending a grammar for a message campaign used by a message optimization system, the method comprising:
- providing a user interface that enables a campaign manager to specify one or more parameters for a new campaign, including audience size, effect size, and expected response rate;
calculating a set of statistical design budgets for the message campaign based on the audience size, effect size, and expected response rate specified by the campaign manager, wherein each statistical design budget specifies a number of components in a message and a number of values to test for each component;
recommending at least one grammar structure from one or more past campaigns that are within the set of statistical design budgets or from a default grammar that complies with the statistical design budget in the event that none of the past campaigns has a grammar within the set of statistical design budgets, the grammar structure specifying a plurality of message component types;
providing a user interface that enables a campaign manager to select one of the recommended grammar structures for the new campaign;
for each message component type in the selected grammar structure, generating a ranked list of previously-used values for the component type in the one or more past campaigns, wherein the previously-used values are each associated with a semantic tag and generating the ranked list comprises;
identifying the semantic tags associated with the previously-used values in the one or more past campaigns, wherein each semantic tag identifies the semantic meaning of the associated value,creating a list of the previously-used values in the one or more past campaigns grouped by semantic tag,ranking groups of semantic tags based on performance in the one or more past campaigns of the previously-used values within a tag group versus other tag groups, andordering the previously-used values first by their ranked tag group and second, within each tag group, by the number of times an individual value has been identified as the winning value in the one or more past campaigns;
for each message component type in the selected grammar structure, generating a ranked list of untested values for the component type, wherein the untested values are each associated with a semantic tag and generating the ranked list comprises;
retrieving the untested values for the component type from a database, wherein each untested value is associated with a semantic tag that identifies the semantic meaning of the associated value and wherein each semantic tag is associated with a ranked tag group of previously-used values in the one or more past campaigns,creating a list of the untested values grouped by semantic tag, andordering the untested values first by the ranked tag group and second, randomly within each tag group;
for each message component type, selecting a plurality of values to recommend testing based at least in part on the ranked list of previously-used values and the ranked list of untested values;
enabling the campaign manager to reject one or more of the recommended values;
in response to the campaign manager rejecting one or more of the recommended values, providing alternate recommended values for the rejected values; and
generating variations of a message to test based on the grammar structure and values accepted by the campaign manager.
7 Assignments
0 Petitions
Accused Products
Abstract
A system and method is provided for recommending a grammar for a message campaign used by a message optimization system. A user specifies parameters for a new campaign, from which a set of statistical design budgets is calculated. The user selects a grammar structure, recommended based on the statistical design budgets, for the campaign. The n-most relevant past campaigns are identified. Semantic tags, associated with each previously used value from the n-most relevant past campaigns and each of a plurality of untested values, are identified and ranked based on past performance. The previously used values are ordered by ranked tag group and then within each tag group, while the untested values are ordered by ranked tag group and then randomly within the tag group. Recommended values are selected from the ranked list of previously used values and untested values depending on the degree of exploration/conservatism indicated by the user.
-
Citations
22 Claims
-
1. A method performed by a computer system for recommending a grammar for a message campaign used by a message optimization system, the method comprising:
-
providing a user interface that enables a campaign manager to specify one or more parameters for a new campaign, including audience size, effect size, and expected response rate; calculating a set of statistical design budgets for the message campaign based on the audience size, effect size, and expected response rate specified by the campaign manager, wherein each statistical design budget specifies a number of components in a message and a number of values to test for each component; recommending at least one grammar structure from one or more past campaigns that are within the set of statistical design budgets or from a default grammar that complies with the statistical design budget in the event that none of the past campaigns has a grammar within the set of statistical design budgets, the grammar structure specifying a plurality of message component types; providing a user interface that enables a campaign manager to select one of the recommended grammar structures for the new campaign; for each message component type in the selected grammar structure, generating a ranked list of previously-used values for the component type in the one or more past campaigns, wherein the previously-used values are each associated with a semantic tag and generating the ranked list comprises; identifying the semantic tags associated with the previously-used values in the one or more past campaigns, wherein each semantic tag identifies the semantic meaning of the associated value, creating a list of the previously-used values in the one or more past campaigns grouped by semantic tag, ranking groups of semantic tags based on performance in the one or more past campaigns of the previously-used values within a tag group versus other tag groups, and ordering the previously-used values first by their ranked tag group and second, within each tag group, by the number of times an individual value has been identified as the winning value in the one or more past campaigns; for each message component type in the selected grammar structure, generating a ranked list of untested values for the component type, wherein the untested values are each associated with a semantic tag and generating the ranked list comprises; retrieving the untested values for the component type from a database, wherein each untested value is associated with a semantic tag that identifies the semantic meaning of the associated value and wherein each semantic tag is associated with a ranked tag group of previously-used values in the one or more past campaigns, creating a list of the untested values grouped by semantic tag, and ordering the untested values first by the ranked tag group and second, randomly within each tag group; for each message component type, selecting a plurality of values to recommend testing based at least in part on the ranked list of previously-used values and the ranked list of untested values; enabling the campaign manager to reject one or more of the recommended values; in response to the campaign manager rejecting one or more of the recommended values, providing alternate recommended values for the rejected values; and generating variations of a message to test based on the grammar structure and values accepted by the campaign manager. - View Dependent Claims (2, 3, 4, 5, 6, 7, 8, 9, 10)
-
-
11. A non-transitory computer-readable medium comprising code that, when executed by a computer system, enables the computer system to perform the following method for recommending a grammar for a message campaign used by a message optimization system, the method comprising:
-
enabling a campaign manager to specify one or more parameters for a new campaign, including audience size, effect size, and expected response rate; calculating a set of statistical design budgets for the message campaign based on the audience size, effect size, and expected response rate specified by the campaign manager, wherein each statistical design budget specifies a number of components in a message and a number of values to test for each component; recommending at least one grammar structure from one or more past campaigns that are within the set of statistical design budgets or from a default grammar that complies with the statistical design budget in the event that none of the past campaigns has a grammar within the set of statistical design budgets, the grammar structure specifying a plurality of message component types; enabling a campaign manager to select one of the recommended grammar structures for the new campaign; for each message component type in the selected grammar structure, generating a ranked list of previously-used values for the component type in the one or more past campaigns, wherein the previously-used values are each associated with a semantic tag and generating the ranked list comprises; identifying the semantic tags associated with the previously-used values in the one or more past campaigns, wherein each semantic tag identifies the semantic meaning of the associated value, creating a list of the previously-used values in the one or more past campaigns grouped by semantic tag, ranking groups of semantic tags based on performance in the one or more past campaigns of the previously-used values within a tag group versus other tag groups, and ordering the previously-used values first by their ranked tag group and second, within each tag group, by the number of times an individual value has been identified as the winning value in the one or more past campaigns; for each message component type in the selected grammar structure, generating a ranked list of untested values for the component type, wherein the untested values are each associated with a semantic tag and generating the ranked list comprises; retrieving the untested values for the component type from a database, wherein each untested value is associated with a semantic tag that identifies the semantic meaning of the associated value and wherein each semantic tag is associated with a ranked tag group of previously-used values in the one or more past campaigns, creating a list of the untested values grouped by semantic tag, and ordering the untested values first by the ranked tag group and second, randomly within each tag group; for each message component type, selecting a plurality of values to recommend testing based at least in part on the ranked list of previously-used values and the ranked list of untested values; enabling the campaign manager to reject one or more of the recommended values; in response to the campaign manager rejecting one or more of the recommended values, providing alternate recommended values for the rejected values; and generating variations of a message to test based on the grammar structure and values accepted by the campaign manager. - View Dependent Claims (12, 13, 14, 15, 16, 17, 18, 19, 20, 21)
-
-
22. A computer system for recommending a grammar for a message campaign used by a message optimization system, the system comprising:
-
a processor; a memory coupled to the processor, wherein the memory stores instructions that, when executed by the processor, causes the system to perform the operations of; enabling a campaign manager to specify one or more parameters for a new campaign, including audience size, effect size, and expected response rate; calculating a set of statistical design budgets for the message campaign based on the audience size, effect size, and expected response rate specified by the campaign manager, wherein each statistical design budget specifies a number of components in a message and a number of values to test for each component; recommending at least one grammar structure from one or more past campaigns that are within the set of statistical design budgets or from a default grammar that complies with the statistical design budget in the event that none of the past campaigns has a grammar within the set of statistical design budgets, the grammar structure specifying a plurality of message component types; enabling a campaign manager to select one of the recommended grammar structures for the new campaign; for each message component type in the selected grammar structure, generating a ranked list of previously-used values for the component type in the one or more past campaigns, wherein the previously-used values are each associated with a semantic tag and generating the ranked list comprises; identifying the semantic tags associated with the previously-used values in the one or more past campaigns, wherein each semantic tag identifies the semantic meaning of the associated value, creating a list of the previously-used values in the one or more past campaigns grouped by semantic tag, ranking groups of semantic tags based on performance in the one or more past campaigns of previously-used values within a tag group versus other tag groups, and ordering the previously-used values first by their ranked tag group and second, within each tag group, by the number of times an individual value has been identified as the winning value in the one or more past campaigns; for each message component type in the selected grammar structure, generating a ranked list of untested values for the component type, wherein the untested values are each associated with a semantic tag and generating the ranked list comprises; retrieving the untested values for the component type from a database, wherein each untested value is associated with a semantic tag that identifies the semantic meaning of the associated value and wherein each semantic tag is associated with a ranked tag group of previously-used values in the one or more past campaigns, creating a list of the untested values grouped by semantic tag, and ordering the untested values first by the ranked tag group and second, randomly within each tag group; for each message component type, selecting a plurality of values to recommend testing based at least in part on the ranked list of previously-used values and the ranked list of untested values; enabling the campaign manager to reject one or more of the recommended values; in response to the campaign manager rejecting one or more of the recommended values, providing alternate recommended values for the rejected values; and generating variations of a message to test based on the grammar structure and values accepted by the campaign manager.
-
Specification