Adaptive experimentation and optimization in automated promotional testing

• US 10,438,230 B2
• Filed: 03/31/2014
• Issued: 10/08/2019
• Est. Priority Date: 03/13/2013
• Status: Active Grant

• Current Assignee: Maplebear Inc.
• Original Assignee: Eversight, Inc. (Maplebear Inc.)
• Inventors: Moran, David, Montero, Michael
• Primary Examiner(s): Andrei, Radu
• Application Number:
  US14/231,426
• Publication Number:
  US 20140330633A1
• Time in Patent Office: 2,017 Days
• Field of Search: 705 14
• CPC Class Codes: G06Q 30/02 Marketing; Price estimation or determination; Fundraising
• CPC Class Codes: G06Q 30/0244 Optimization
• CPC Class Codes: G06Q 30/0269 based on user profile or attribute
• CPC Class Codes: G06Q 30/0277 Online advertisement
• CPC Class Codes: G06T 1/0064 Geometric transfor invariant watermarking, e.g. affine transform invariant
• CPC Class Codes: G06T 3/02 Affine transformations for image registration G06T3/147; for image mosaicing G06T3/4038
• CPC Class Codes: H04N 21/41415 involving a public display, viewable by several users in a public space outside their home, e.g. movie theatre, information kiosk

Abstract

Methods and apparatus for implementing forward looking optimizing promotions by administering, in large numbers and iteratively, test promotions automatically formulated using highly granular test variables on subpopulations. The responses from individuals in the subpopulations are received and analyzed. The analysis result is employed to subsequently formulate a general public promotion.

Claims

1. A computer-implemented method for performing adaptive experimentation and optimization for promotional testing that is performed by a promotion optimization system, containing one or more processors, a promotion module, an administration module, and a monitoring module, the method comprising:

generating a plurality of test promotions by receiving a design specification for the promotion, wherein the design specifications include a goal of the promotion, identification of all test variables within the design space, identification of all values possible for the test variables, constraints, and number of times a high performing test variable value needs to be validated by a test promotion;

planning the plurality of test promotions by clustering values for each variable by common themes, comparing numbers of clusters for each variable to determine the variable with a largest number of clusters, setting a testing size to be a number of phases equal to the largest number of clusters, and distributing the test promotions among the phases by keeping all variable values within a given cluster together in each phase;

administering the plurality of test promotions to a plurality of segmented subpopulations of consumers responsive to the planning in real-time, wherein the real-time administration of test promotions includes concurrent testing of the plurality of test promotions at the same time to the plurality of segmented subpopulations, to improve selection of the variable values in future promotions to achieve higher redemption rates;

tracking responses from said segmented subpopulations of consumers and redemption rates; and

generating a general population promotion with values for the variables that have high redemption rates to improve performance effectiveness of the general population promotion.

2. The method of claim 1, wherein the clustering themes include percent off, buy-one-get-one, and graphics versus text.

3. The method of claim 1, wherein the clustering of test promotion values for each test promotion variable is by performance.

4. The method of claim 3, wherein performance value clusters are determined by predictive model or historical performance data.

5. The method of claim 4, wherein the planning the plurality of test promotions includes generating a sequence for exploration of the plurality of test promotions where each variable value is compared against a set of benchmark variable values.

6. The method of claim 1, wherein the planning the plurality of test promotions includes outputting an expected test time for exploring and validating.

7. The method of claim 6, wherein the expected test time is dependent upon a threshold for certainty.

8. The method of claim 6, wherein the test time is for exploring and validating a subset of the variables that are most results effective.

9. The method of claim 1, wherein an optimal order to test the available variables and values includes reduction of permutations, ascertaining effect isolation, incorporating constraints and generating a permutation set that has benchmark variable values and the test variable values different from one another, and for a given benchmark variable value the permutation set has values, that are not benchmark values, for the other variables.

10. The method of claim 9, wherein the reduction of permutations identifies a reduced number of promotional combinations that still effectively test a design space.

11. The method of claim 10, wherein the minimal promotional combinations that still effectively test a design space is determined by identifying total permutations of variables and selecting from the total permutations to a reduced subset where there is an equal appearance for each value per variable, and an equal appearance of pair-wise values across any pair of variables.

12. The method of claim 9, wherein the ascertaining effect isolation includes identifying variables and values that do not cross-contaminate each other.

13. The method of claim 9, wherein the incorporating constraints includes imposing restrictions on values for any given variable, or combination of values for a set of variables.