Methods, systems, and devices for counterfactual-based incrementality measurement in digital ad-bidding platform

• US 10,467,659 B2
• Filed: 08/02/2017
• Issued: 11/05/2019
• Est. Priority Date: 08/03/2016
• Status: Expired due to Fees

• Current Assignee: Mediamath Acquisition Corporation
• Original Assignee: Mediamath LLC
• Inventors: Chalasani, Prasad, Buchalter, Ari, Winston, Ezra, Thiagarajan, Jaynth
• Primary Examiner(s): Elchanti, Tarek
• Application Number:
  US15/667,507
• Publication Number:
  US 20180040032A1
• Time in Patent Office: 825 Days
• Field of Search: 705 1471
• CPC Class Codes: G06N 7/01 Probabilistic graphical models, e.g. probabilistic networks
• CPC Class Codes: G06Q 30/0243 Comparative campaigns
• CPC Class Codes: G06Q 30/0275 Auctions
• CPC Class Codes: G06Q 30/0277 Online advertisement

Abstract

A digital ad-buying platform uses counterfactual-based incrementality measurement by implementing randomization and/or a correction for auction win bias to avoid the need to identify counterfactual winner types in the control group. This approach can estimate impact at the individual consumer level. Confidence levels can be determined using Gibbs sampling in the context of causal analysis in the presence of non-compliance.

Claims

1. A computer-implemented method for adjusting future bid requests based on a causal ad impact of previous bid requests, the method comprising:

receiving, by a computer system of a demand side platform, a first bid request from an ad exchange, wherein the first bid request is indicative of a first bidding opportunity at the ad exchange;

matching, by the computer system of the demand side platform, ad campaigns of one or more advertisers to the first bid request to identify a list of one or more eligible advertisers to submit a bid to the first bidding opportunity to place one or more advertisements from the ad campaigns of the one or more eligible advertisers on user computing systems of a plurality of users;

executing, by the computer system of the demand side platform, a pre-bid randomization scheme prior to submitting bids, whereby each of a plurality of user identifiers associated with the plurality of users is categorized in a control group or a test group;

logging, by the computer system of the demand side platform, a phantom control impression, the phantom control impression comprising the control group of the plurality of user identifiers, wherein the ad exchange does not transmit the one or more advertisements from the ad campaigns of the one or more eligible advertisers to user computing systems of the control group;

submitting, by the computer system of the demand side platform, one or more bids associated with the ad campaigns of the one or more eligible advertisers for placing the one or more advertisements from the ad campaigns of the one or more eligible advertisers on user computing systems of the test group;

receiving, by the computer system of the demand side platform, results of the one or more submitted bids, wherein the results indicate whether each of the one or more submitted bids was won or lost;

logging, by the computer system of the demand side platform, a test-win impression, the test-win impression comprising a first sub-group of one or more user identifiers of the test group, wherein the ad exchange transmits at least one of the one or more advertisements from the ad campaigns of the one or more eligible advertisers to user computing systems of the first sub-group of one or more user identifiers of the test group;

logging, by the computer system of the demand side platform, a test-lost impression, the test-lost impression comprising a second sub-group of one or more user identifiers of the test group, wherein the ad exchange does not transmit the one or more advertisements from the ad campaigns of the one or more eligible advertisers to user computing systems of the second sub-group of one or more user identifiers of the test group;

accessing, by the computer system of the demand side platform, cookie data associated with the control group of the plurality of user identifiers of the phantom control impression, the first sub-group of the test group of the plurality of user identifiers of the test-win impression, and the second sub-group of the test group of the plurality of user identifiers of the test-lost impression, wherein the cookie data comprises a cookie ID that can be utilized to identify browser actions of associated users;

identifying, by the computer system of the demand side platform, browser actions associated with the control group of the plurality of user identifiers of the phantom control impression, the first sub-group of the test group of the plurality of user identifiers of the test-win impression, and the second sub-group of the test group of the plurality of user identifiers of the test-lost impression by utilizing the accessed cookie data;

determining, by the computer system of the demand side platform, the causal ad impact based at least in part on the identified browser actions associated with the control group of the plurality of user identifiers of the phantom control impression, the first sub-group of the test group of the plurality of user identifiers of the test-win impression, and the second sub-group of the test of the plurality of user identifiers of the test-lost impression; and

generating, by the computer system, a graphical user interface that allows the one or more advertisers to utilize the determined causal ad impact to adjust one or more bids in a second bidding opportunity,

wherein the computer system comprises a computer processor and an electronic storage medium.

2. The computer-implemented method of claim 1, wherein each of the plurality of user identifiers associated with the plurality of users are categorized as the control group or the test group based on a fraction probability, and wherein the causal ad impact is determined based at least in part on the fraction probability.

3. The computer-implemented method of claim 1, wherein the method further comprises:

identifying, by the computer system of the demand side platform, one or more consumer responses based on the browser actions associated with the control group of the plurality of user identifiers of the phantom control impression, the first sub-group of the test group of the plurality of user identifiers of the test-win impression, and the second sub-group of the test group of the plurality of user identifiers of the test-lost impression, wherein determining the causal ad impact is further based on the one or more consumer responses.

4. The computer-implemented method of claim 3, wherein the one or more consumer responses comprises one or more of:

a site visit, a registration, a subscription, an addition of items to a shopping cart, or a purchase.

5. The computer-implemented method of claim 1, wherein each ad campaign comprises one or more advertisement targets on at least one of:

a search channel, a display channel, a mobile application channel, a video channel, a social media channel, or an email channel.

6. The computer-implemented method of claim 1, wherein the method further comprises:

determining, by the computer system of the demand side platform, a confidence factor for the causal ad impact, wherein the confidence factor comprises an interval and/or a value range associated with a probability percentile.

7. The computer-implemented method of claim 6, wherein the confidence factor is determined using Markov-Chain Monte-Carlo sampling.

8. The computer-implemented method of claim 7, wherein the Markov-Chain Monte-Carlo sampling is a Gibbs sampling scheme.

9. A system for adjusting future bid requests based on a causal ad impact of previous bid requests, wherein the system comprises:

one or more computer readable storage devices configured to store a plurality of computer executable instructions; and

one or more hardware computer processors in communication with the one or more computer readable storage devices and configured to execute the plurality of computer executable instructions in order to cause the system to;

receive a first bid request from an ad exchange, wherein the first bid request is indicative of a first bidding opportunity at the ad exchange;

match ad campaigns of one or more advertisers to the first bid request to identify a list of one or more eligible advertisers to submit a bid to the first bidding opportunity to place one or more advertisements from the ad campaigns of the one or more eligible advertisers on user computing systems of a plurality of users;

execute a randomization scheme prior to submitting bids, whereby each of a plurality of user identifiers associated with the plurality of users is categorized in a control group or a test group based on a fraction probability;

log a phantom control impression, the phantom control impression comprising the control group of the plurality of user identifiers, wherein the ad exchange does not transmit the one or more advertisements from the ad campaigns of the one or more eligible advertisers to user computing systems of the control group;

submit one or more bids associated with the ad campaigns of the one or more eligible advertisers for placing the one or more advertisements from the ad campaigns associated with the one or more eligible advertisers on user computing systems of the test group;

receive results of the one or more submitted bids, wherein the results indicate whether each of the one or more submitted bids was won or lost;

log a test-win impression, the test-win impression comprising a sub-group of one or more user identifiers of the test group, wherein the ad exchange transmits at least one of the one or more advertisements from the ad campaigns of one or more eligible advertisers to user computing systems of the sub-group of one or more user identifiers of the test group;

access cookie data associated with the control group of the plurality of user identifiers of the phantom control impression and the sub-group of the test group of the plurality of user identifiers of the test-win impression, wherein the cookie data comprises a cookie ID that can be utilized to identify browser actions of associated users;

identify browser actions associated with the control group of the plurality of user identifiers of the phantom control impression and the sub-group of the test group of the plurality of user identifiers of the test-win impression by utilizing the accessed cookie data;

determine the causal ad impact based at least in part on the identified browser actions associated with the control group of the plurality of user identifiers of the phantom control impression and the sub-group of the test group of the plurality of user identifiers of the test-win impression; and

generate a graphical user interface that allows the one or more advertisers to utilize the determined causal ad impact to adjust one or more bids in a second bidding opportunity.

10. The system of claim 9, wherein user computing systems of the control group of the plurality of user identifiers are sent a general advertisement not associated with the one or more advertisements from ad campaigns of the one or more eligible advertisers.

11. The system of claim 9, wherein to determine the causal ad impact the logs of the phantom control impression and the test-win impression are taken over a predetermined time period.

12. The system of claim 9, wherein the randomization scheme is executed by categorizing each of the plurality of user identifiers in the control group or the test group using a hash function.

13. The system of claim 9, wherein the matching of ad campaigns of one or more advertisers to the first bid request is based on at least one of:

targeting requirements, types of consumers, targeting contexts, governing campaign criteria, available budget, desired frequency of exposure, or publisher restrictions on types of advertisers.

14. A computer-implemented method for adjusting future bid requests based on a causal ad impact of previous bid requests, the method comprising:

receiving, by a computer system of the demand side platform, a first bid request from an ad exchange, wherein the first bid request is indicative of a first bidding opportunity at the ad exchange;

matching, by the computer system of the demand side platform, ad campaigns of one or more advertisers to the first bid request to identify a list of one or more eligible advertisers to submit a bid to the first bidding opportunity to place one or more advertisements from the ad campaigns of the one or more eligible advertisers on user computing systems of a plurality of users;

submitting, by the computer system of the demand side platform, one or more bids associated with the ad campaigns of the one or more eligible advertisers;

receiving, by the computer system of the demand side platform, results of the one or more submitted bids, wherein the results indicate whether each of the one or more bids was won or lost;

executing, by the computer system of the demand side platform, a randomization scheme, whereby each of a plurality of user identifiers associated with the plurality of users is categorized in a control group or a test group;

logging, by the computer system of the demand side platform, a test group impression, the test group impression comprising the test group of the plurality of user identifiers, wherein the ad exchange transmits at least one or more advertisements from the ad campaigns of the one or more eligible advertisers to user computing systems of the test group of the plurality of user identifiers;

logging, by the computer system of the demand side platform, a control group impression, the control group impression comprising the control group of the plurality of user identifiers, wherein the ad exchange does not transmit the one or more advertisements from the ad campaigns of the one or more eligible advertisers to user computing systems of the control group of the plurality of user identifiers;

accessing, by the computer system of the demand side platform, cookie data associated with the test group of the plurality of user identifiers of the test group impression and the control group of the plurality of user identifiers of the control group impression, wherein the cookie data comprises a cookie ID that can be utilized to identify browser actions of associated users;

identifying, by the computer system of the demand side platform, browser actions associated with the test group of the plurality of user identifiers of the test group impression and the control group of the plurality of user identifiers of the control group impression by utilizing the accessed cookie data;

determining, by the computer system of the demand side platform, the causal ad impact based at least in part on the identified browser actions associated with the test group of the plurality of the user identifiers of the test group impression and the control group of the plurality of user identifiers of the control group impression; and

generating, by the computer system, a graphical user interface that allows the one or more advertisers to utilize the determined causal ad impact to adjust one or more bids in a second bidding opportunity,

wherein the computer system comprises a computer processor and an electronic storage medium.

15. The computer-implemented method of claim 14, wherein one or more integrated identifiers are used to determine the causal ad impact, wherein each of the one or more integrated identifiers is associated with a particular user identifier.

16. The computer-implemented method of claim 15, wherein over a million user identifiers are assessed to create the one or more integrated identifiers.

17. The computer-implemented method of claim 15, further comprising:

determining, by the computer system of the demand side platform, a confidence factor for the causal ad impact, wherein the confidence factor is based on a number of total user identifiers assessed to create the integrated identifiers.

18. The computer-implemented method of claim 14, wherein the causal ad impact is used to redirect browser actions.

19. The computer-implemented method of claim 14, wherein the causal ad impact is used to prospect new customers.

20. The computer-implemented method of claim 14, wherein the randomization scheme is executed subsequent to submitting the one or more bids.