Selection bias correction for paid search in media mix modeling

• US 10,445,388 B2
• Filed: 09/19/2017
• Issued: 10/15/2019
• Est. Priority Date: 09/19/2017
• Status: Active Grant

• Current Assignee: Google LLC (Alphabet Inc.)
• Original Assignee: Google LLC (Alphabet Inc.)
• Inventors: Chan, David, Wang, Yueqing, Chen, Aiyou, Koehler, James, Jin, Yuxue, Perry, Michael, Sun, Yunting
• Primary Examiner(s): Gortayo, Dangelino N
• Application Number:
  US15/708,592
• Publication Number:
  US 20190087497A1
• Time in Patent Office: 756 Days
• Field of Search: 707709, 707688, 707724, 707725, 707728, 707737, 705 1441, 705 141
• CPC Class Codes: G06F 16/29 Geographical information databases
• CPC Class Codes: G06F 16/9535 Search customisation based on user profiles and personalisation
• CPC Class Codes: G06F 16/955 using information identifiers, e.g. uniform resource locators [URL]
• CPC Class Codes: G06Q 30/0201 Market modelling; Market analysis; Collecting market data
• CPC Class Codes: G06Q 30/0242 Determining effectiveness of advertisements
• CPC Class Codes: G06Q 30/0243 Comparative campaigns

Abstract

Systems, methods, and computer-readable storage media that may be used to generate causal models and calculate a selection bias in mixed media. In some embodiments, the selection bias calculation is in search sponsored content in the context of mixed media modeling. In some embodiments, a method for search bias correction is based on the back-door criterion from causal inference.

Claims

1. A method comprising:

identifying, by one or more processors, a first Uniform Resource Locator (URL) associated with an incremental value change of a first metric;

identifying, by the one or more processors, a plurality of URLs associated with the first URL;

receiving search query data comprising queries from a target geographical region in a first time window;

partitioning, by the one or more processors, the received search query data to a plurality of groups comprising a first group associated with the first URL, a second group associated with one or more of the plurality of URLs, and a third group associated with a business category, each respective association defined by equaling or exceeding a predetermined threshold of a second metric;

generating, by the one or more processors, a plurality of search query subsets based on the plurality of groups;

generating, by the one or more processors, an additive regression model based on a causal diagram that comprises as identification of a causal effect associated with the incremental value change of the first metric; and

calculating, by the one or more processors, a bias corrected estimate of the incremental value change of the first metric by fitting the additive regression model to the plurality of search query subsets.

2. The method of claim 1, further comprising:

determining a first search term of the received search query data is associated with search results comprising the first URL;

determining a second search term of the received search query data is associated with search results comprising one of the one or more plurality of URLs;

partitioning, by the one or more processors, the first search term to the first group and the second search term to the second group.

3. The method of claim 1, further comprising:

determining the first time window;

calculating, by the one or more processors, a first number of searches in the first group, a second number of searches in the second group, and a third number of searches in the third group;

calculating, by the one or more processors, a total search volume of the first group, the second group, and the third group; and

determining at least one of the first number of searches in the first group, the second number of searches in the second group, and the third number of searches in the third group compared to the total search volume exceeds a second predetermined threshold.

4. The method of claim 1, further comprising:

determining, a number of search queries in the received search query data exceeds a predetermined search query threshold; and

generating the additive regression model using a 3-dimension full tensor product smooth consequent to determining the number of search queries in the received search query data exceeds the predetermined search query threshold.

5. The method of claim 1, wherein the first metric comprises a return on advertising spend.

6. The method of claim 1, further comprising reformulating the additive regression model using a restricted maximum likelihood (REML) algorithm.

7. The method of claim 1, wherein the search query data comprises all queries selected using at least one of a criteria selected from a group comprising the target geographical region, the time window, and a user demographic.

8. A system comprising:

at least one computing device operably coupled to at least one memory and configured to;

identify a first Uniform Resource Locator (URL) associated with an incremental value change of a first metric;

identify a plurality of URLs associated with the first URL;

receive search query data comprising queries from a target geographical region in a first time window;

partition the received search query data to a plurality of groups comprising a first group associated with the first URL, a second group associated with one or more of the plurality of URLs, and a third group associated with a business category, each respective association defined by equaling or exceeding a predetermined threshold of a second metric;

generate a plurality of search query subsets based on the plurality of groups;

generate an additive regression model based on a causal diagram that comprises as identification of a causal effect associated with the incremental value change of the first metric; and

calculate a bias corrected estimate of the incremental value change of the first metric by fitting the additive regression model to the plurality of search query subsets.

9. The system of claim 8, further comprising the at least one computing device configured to:

determine a first search term of the received search query data is associated with search results comprising the first URL;

determine a second search term of the received search query data is associated with search results comprising one of the one or more plurality of URLs;

partition the first search term to the first group and the second search term to the second group.

10. The system of claim 8, further comprising the at least one computing device configured to:

determine the first time window;

calculate a first number of searches in the first group, a second number of searches in the second group, and a third number of searches in the third group; and

calculate a total search volume of the first group, the second group, and the third group.

11. The system of claim 8, further comprising the at least one computing device configured to:

determine, a number of search queries in the received search query data exceeds a predetermined number of search queries threshold; and

generate the additive regression model using a 3-dimension full tensor product smooth consequent to determining the number of search queries in the received search query data exceeds the predetermined number of search queries threshold.

12. The system of claim 8, wherein the first metric comprises a return on ad spend.

13. The system of claim 8, further comprising the at least one computing device configured to reformulate the additive regression model using a restricted maximum likelihood (REML) algorithm.

14. The system of claim 8, wherein the search query data comprises all queries selected using at least one of a criteria selected from a group comprising the target geographical region, the time window, and a user demographic.

15. A non-transitory computer-readable media having computer-executable instructions embodied therein that, when executed by one or more processors of a computing system, cause the computing system to perform a process comprising:

identifying a first content item;

determining a media mix environment is associated with the first content item;

determining a causal relationship between a first media of the media mix environment and a second media of the media mix environment;

identifying a first Uniform Resource Locator (URL) associated with the first content item;

identifying a plurality of URLs associated with the first URL;

receiving search query data comprising queries from a target geographical region in a first time window;

partitioning the received search query data to a plurality of groups comprising a first group associated with the first URL, a second group associated with one or more of the plurality of URLs, and a third group associated with a business category, each respective association defined by equaling or exceeding a predetermined threshold of a second metric;

generating a plurality of search query subsets based on the plurality of groups;

calculating a bias factor between the first media of the media mix environment and the second media of the media mix environment based on the causal relationship;

generating an additive regression model using the bias factor between the first media of the media mix environment and the second media of the media mix environment; and

calculating a bias corrected estimate of the incremental value change of the first metric by fitting the additive regression model to the plurality of search query subsets.

16. The media of claim 15, the process further comprising:

determining a first search term of the received search query data is associated with search results comprising the first URL;

determining a second search term of the received search query data is associated with search results comprising one of the one or more plurality of URLs;

partitioning the first search term to the first group and the second search term to the second group.

17. The media of claim 15, the process further comprising:

determining a first time window;

calculating a first number of searches in the first group, a second number of searches in the second group, and a third number of searches in the third group; and

calculating a total search volume of the first group, the second group, and the third group.

18. The media of claim 15, the process further comprising:

determining a number of search queries in the received search query data exceeds a predetermined search query threshold; and

generating the additive regression model using a 3-dimension full tensor product smooth consequent to determining the number of search queries in the received search query data exceeds the predetermined search query threshold.

19. The media of claim 15, the process further comprising reformulating the additive regression model using a restricted maximum likelihood (REML) algorithm.

20. The media of claim 15, wherein the search query data comprises all queries selected using at least one of a criteria selected from a group comprising a geographical target region, a time window, and a user demographic.