Automated event correlation to improve promotional testing

• US 9,940,640 B2
• Filed: 03/31/2014
• Issued: 04/10/2018
• Est. Priority Date: 03/13/2013
• Status: Active Grant

• Current Assignee: Maplebear Inc.
• Original Assignee: Eversight, Inc. (Maplebear Inc.)
• Inventors: Moran, David, Montero, Michael
• Primary Examiner(s): Brown, Luis A
• Assistant Examiner(s): Andrei, Radu
• Application Number:
  US14/231,442
• Publication Number:
  US 20140330635A1
• Time in Patent Office: 1,471 Days
• CPC Class Codes: G06F 3/0425 using a single imaging device like a video camera for tracking the absolute position of a single or a plurality of objects with respect to an imaged reference surface, e.g. video camera imaging a display or a projection screen, a table or a wall surface, on which a computer generated image is displayed or projected tracking a projected light spot to determine a position on a display surface G06F3/0386
• CPC Class Codes: G06Q 30/02 Marketing; Price estimation or determination; Fundraising
• CPC Class Codes: G06Q 30/0244 Optimization
• CPC Class Codes: G06Q 30/0269 based on user profile or attribute
• CPC Class Codes: G06Q 30/0277 Online advertisement
• CPC Class Codes: G06T 1/0021 Image watermarking
• CPC Class Codes: G06T 1/0064 Geometric transfor invariant watermarking, e.g. affine transform invariant
• CPC Class Codes: G06T 19/006 Mixed reality object pose determination, tracking or camera calibration for mixed reality G06T7/00
• CPC Class Codes: G06T 3/02 Affine transformations for image registration G06T3/147; for image mosaicing G06T3/4038
• CPC Class Codes: H04N 21/41415 involving a public display, viewable by several users in a public space outside their home, e.g. movie theatre, information kiosk
• CPC Class Codes: H04N 21/8358 involving watermark protecting executable software by watermarking G06F21/16; image watermarking in general G06T1/0021; watermarks inserted in still images for transmission purposes H04N1/32144; inserting watermarks during video coding H04N19/467

Abstract

Methods and apparatus for implementing forward looking optimizing promotions by administering, in large numbers and iteratively, test promotions formulated using highly granular test variables on purposefully segmented subpopulations. The plurality of test promotions automatically account for covariates. The responses from individuals in the subpopulations are received and analyzed. The analysis result is employed to subsequently formulate general public promotion.

Claims

1. A computer implemented promotion-optimization method that is performed by a promotion optimization system, containing a correlation engine, a promotion module, an administration module, a monitoring module, the method comprising:

extracting a plurality of covariants from data feeds, wherein the covariants include referring source, demographic information, search keywords, geographic information, and temporal information;

generating a plurality of test promotions;

administering the plurality of test promotions to a plurality of segmented subpopulations of consumers in a testing time period, wherein the testing time period includes concurrent testing of the plurality of test promotions;

obtaining responses from said segmented subpopulations of consumers via a point of sales terminal, a website and a program;

correlating the extracted covariants to inputs for the test promotions, wherein the correlating is a degree of attribution of experiment performance to the presence of a covariant, and wherein the correlating identifies a range of values for a particular covariant that correlates with experiment performance, and wherein the correlation derives covariant-experiment pairings;

monitoring the covariant-experiment pairings over time for changes in test promotion results from extraneous events, wherein the extraneous events that impact the test promotions is defined as noise;

attributing the noise to covariants based upon covariant-experiment pairing correlations;

improving a signal-to-noise ratio for the responses by factoring out the impact of covariants associated with the extraneous events to generate corrected promotional results; and

generating a general population promotion using the improved signal-to-noise ratio of the test promotion responses responsive to analysis of said corrected promotional results.

2. The method of claim 1, wherein the referring source includes a search engine, a social network, an email, a personal loyalty account and a mobile browser.

3. The method of claim 1, further comprising filtering out covariants that do not have an effect on the test promotions.

4. The method of claim 1, wherein the extracting the plurality of covariants includes identifying the covariants using techniques including k-means clustering, machine learned classification, pre-defined covariants based on domain specific clusters, pre-defined temporal blocks, and hierarchical clustering.

5. The method of claim 4, wherein the technique used for identification of covariants is determined by heuristics based on the data feed type and number of data feeds.

6. The method of claim 1, wherein the correlating is performed by one of general linear models, chi-square analysis and Pearson'"'"'s coefficient.

7. The method of claim 1, wherein the correlating identifies a range of values for a particular covariant that correlate with experiment performance.

8. The method of claim 1, further comprising monitoring covariant-experiment correlation pairs for changes in the degree of attribution of experiment performance.

9. A system for performing promotion optimization comprising:

a correlation engine on a server for extracting a plurality of covariants from data feeds, wherein the covariants include referring source, demographic information, search keywords, geographic information, and temporal information;

a promotion module for generating a plurality of test promotions;

an administration module for administering the plurality of test promotions to a plurality of segmented subpopulations of consumers in a testing time period, wherein the testing time period includes concurrent testing of the plurality of test promotions, and

obtaining responses from said segmented subpopulations of consumers via a point of sales terminal, a website and a program;

a correlation engine for correlating the extracted covariants to inputs for the test promotions, wherein the correlating is a degree of attribution of experiment performance to the presence of a covariant, and wherein the correlating identifies a range of values for a particular covariant that correlates with experiment performance, and wherein the correlation derives covariant-experiment pairings;

a monitoring module for monitoring the covariant-experiment pairings over time for changes in test promotion results from extraneous events, wherein the extraneous events that impact the test promotions is defined as noise;

the correlation engine further for attributing the noise to covariants based upon covariant-experiment pairing correlations, and improving a signal-to-noise ratio for the responses by factoring out the impact of covariants associated with the extraneous events to generate corrected promotional results; and

a general population module for generating a general population promotion using the improved signal-to-noise ratio of the test promotion responses responsive to analysis of said corrected promotional results.

10. The system of claim 9, wherein the referring source includes a search engine, a social network, an email, a personal loyalty account and a mobile browser.

11. The system of claim 9, further comprising a covariant ID kernel for filtering out covariants that do not have an effect on the test promotions.

12. The system of claim 9, wherein the extracting the plurality of covariants includes identifying the covariants using techniques including k-means clustering, machine learned classification, pre-defined covariants based on domain specific clusters, pre-defined temporal blocks, and hierarchical clustering.

13. The system of claim 12, wherein the technique used for identification of covariants is determined by heuristics based on the data feed type and number of data feeds.

14. The system of claim 9, wherein the correlating is performed by one of general linear models, chi-square analysis and Pearson'"'"'s coefficient.

15. The system of claim 9, wherein the correlating identifies a range of values for a particular covariant that correlate with experiment performance.

16. The system of claim 9, further comprising a covariant monitor for monitoring covariant-experiment correlation pairs for changes in the degree of attribution of experiment performance.