Web page experiments with fragmented section variations

• US 8,239,489 B1
• Filed: 01/28/2011
• Issued: 08/07/2012
• Est. Priority Date: 02/25/2008
• Status: Expired due to Fees

• Current Assignee: Google LLC (Alphabet Inc.)
• Original Assignee: Google Inc. (Alphabet Inc.)
• Inventors: Vasilik, Kenneth Eric
• Primary Examiner(s): Chan, Wing
• Assistant Examiner(s): Zong, Ruolei
• Application Number:
  US13/016,018
• Time in Patent Office: 557 Days
• Field of Search: 709/218, 709/203, 709/217, 709/245, 705/14.4, 705/14.1, 705/10, 705/26, 705/400, 715/234, 715/253, 715/229, 715/264, 715/277
• US Class Current: 709/218
• CPC Class Codes: G06Q 30/02 Marketing; Price estimation or determination; Fundraising
• CPC Class Codes: G06Q 30/0241 Advertisements

Abstract

In one general aspect, a page identifier of a test display environment having code defining a content section, the content section comprising a plurality of content section fragments, is received. A content item group is selected from a collection of content item groups, each content item group defining an association of content items with content section fragments of the content section. Each identified content item in the selected content item group is transmitted for placement on the test display environment in the associated content section fragment.

Claims

1. A computer-implemented method for delivering content to a user device executing an application that displays the content in a display environment, comprising:

receiving a identifier of a test display environment for an experiment, the test display environment including a content section, wherein the content section comprises a plurality of content section fragments, wherein the content section fragments for the content section are portions of the content section that have been identified as being replaceable by related content during the experiment;

selecting, with one or more computers, content items for the content section fragments, the selecting comprising selecting a content item group for the content section from a plurality of content item groups for the content section, wherein each content item group specifies a content item for each fragment of the content section;

transmitting, with the one or more computers, each content item in the selected content item group for placement on the test display environment in the content section fragments; and

measuring the performance of the selected content item group in the test display environment,

wherein the experiment is run during a period of time that is a subset of the time the test display environment is displayed, and only during the period of time, displayed content items for the test display environment are varied to determine an effectiveness of particular different combinations of content items associated with the experiment.

2. The computer-implemented method of claim 1, wherein selecting the content item group comprises pseudo-randomly selecting the content group from the plurality of content item groups for the content section.

3. The computer-implemented method of claim 1, further comprising:

determining that, after the test display environment is presented to a user, the user reached a conversion page for the selected content item group, and recording the determination; and

determining, for each content item group, a respective number of times a conversion page for the content item group was reached.

4. The method of claim 1, wherein the test display environment includes a second content section, the method further comprising selecting one or more content items for the second content section independently of the selection of the content items for the first content section.

5. The method of claim 1, wherein the test display environment is a web page.

6. The method of claim 1, wherein transmitting each content item comprises transmitting each content item from an experiment system to a web browser executing on a user device.

7. A computer-implemented method, comprising:

receiving a test display environment for an experiment;

parsing the test display environment to extract an experiment identifier for the experiment from the test display environment;

parsing the test display environment to identify a content section in the test display environment, wherein the content section comprises a plurality of content section fragments, wherein the content section fragments for the content section are portions of the content section that have been identified as being replaceable by related content during the experiment, and then extracting a section identifier for the content section and extracting a fragment identifier for each of the plurality of content section fragments;

storing the section and fragment identifiers in association with the experiment identifier;

presenting a user interface requesting a group of content items for the content section corresponding to the extracted section identifier;

receiving, through the user interface, a first group of content items for the content section corresponding to the extracted section identifier, the first group of content items including a respective content item for each of the content section fragments of the content section;

storing the first group of content items in a record in association with the section identifier and fragment identifiers; and

measuring the performance of the first group of content items in the test display environment,

wherein the experiment is run during a period of time that is a subset of the time the test display environment is displayed, and only during the period of time, displayed content items for the test display environment are varied to determine an effectiveness of particular different combinations of content items associated with the experiment.

8. The computer-implemented method of claim 7, further comprising:

receiving a second group of content items for the content section corresponding to the extracted section identifier, the second group of content items including a respective content item for each of the content section fragments of the content section; and

storing the second group of content items in a record in association with the section identifier and fragment identifiers.

9. The computer-implemented method of claim 8, further comprising measuring the performance of the test display environment with respect to the second content item group.

10. The computer-implemented method of claim 9, further comprising:

receiving an identifier of a conversion page for the delivered content item group; and

determining that the web browser accessed the conversion page after displaying the delivered content item group on the test display environment; and

recording that the web browser accessed the conversion page for the delivered content item group.

11. The computer-implemented method of claim 10, further comprising presenting the number of times one or more web browsers accessed a conversion page for the first content item group when the first content item group was delivered to the one or more web browsers and presenting the number of times one or more web browsers accessed a conversion page for the second content item group when the second content item group was delivered to the one or more web browsers.

12. The computer-implemented method of claim 11, where the conversion page for the first content item group is the same as the conversion page for the second content item group.

13. The computer-implemented method of claim 7, wherein the test display environment is a web page.

14. A system, comprising:

one or more computers configured to perform operations comprising;

receiving a page identifier of a test display environment for an experiment, the test display environment having code defining a content section, wherein the content section comprises a plurality of content section fragments, wherein the content section fragments for the content section are portions of the content section that have been identified as being replaceable by related content during the experiment;

selecting content items for the content section fragments, the selecting comprising selecting a content item group for the content section from a plurality of content item groups for the content section, wherein each content item group specifies a content item for each fragment of the content section;

transmitting, with the one or more computers, each content item in the selected content item group for placement on the test display environment in the content section fragments; and

means for measuring the performance of the selected content item group in the test display environment,

wherein the experiment is run during a period of time that is a subset of the time the test display environment is displayed, and only during the period of time, displayed content items for the test display environment are varied to determine an effectiveness of particular different combinations of content items associated with the experiment.

15. The system of claim 14, wherein selecting the content item group comprises pseudo-randomly selecting the content group from the plurality of content item groups for the content section.

16. The system of claim 14, further comprising:

determining that, after the test display environment is presented to a user, the user reached a conversion page for the selected content item group, and recording the determination; and

determining, for each content item group, a respective number of times a conversion page for the content item group was reached.

17. The system of claim 14, wherein the test display environment includes a second content section, the method further comprising selecting one or more content items for the second content section independently of the selection of the content items for the first content section.

18. The system of claim 14, wherein the test display environment is a web page.

19. The system of claim 14, wherein transmitting each content item comprises transmitting each content item from an experiment system to a web browser executing on a user device.