FACILITATING CONSISTENT A/B TESTING ASSIGNMENT

• US 20180020065A1
• Filed: 07/13/2016
• Published: 01/18/2018
• Est. Priority Date: 07/13/2016
• Status: Active Grant

• Current Assignee: Adobe Inc.
• Original Assignee: Adobe Systems Incorporated (Adobe Inc.)
• Inventors: Kucera, John
• Granted Patent: US 10,630,789 B2
• Time in Patent Office: Days
• CPC Class Codes: G06Q 30/0246 Traffic
• CPC Class Codes: H04L 67/02 based on web technology, e.g. hypertext transfer protocol [HTTP]
• CPC Class Codes: H04L 67/146 Markers for unambiguous identification of a particular session, e.g. session cookie or URL-encoding
• CPC Class Codes: H04L 67/535 Tracking the activity of the user network monitoring arrangements H04L43/00; recording of computer activity G06F11/34

Abstract

The present disclosure is directed toward systems and methods for providing consistent test versions of an A/B tested web page in an A/B test. For example, one or more embodiments described herein generate an A/B test mapping that associates ranges of identifiers with test versions of an A/B tested web page. Additionally, embodiments described herein generate a web page visitor token based on a random number that corresponds to a single identifier within the mapping. Each time the web page visitor requests the A/B tested web page, systems and methods described herein utilize the visitor token to consistently provide the same test version of the A/B tested web page to the web page visitor.

Claims

1. In a digital medium environment for serving web pages over a network, a method of conducting A/B testing comprising:

receiving, at a web server and from a client computing device, a first request for a web page;

generating, by the web server and using a random number generator, a token for the client-computing device;

identifying, based on the generated token for the client-computing device, a first test version of a plurality of test versions of the requested web page;

providing the first test version of the requested web page and the generated token to the client-computing device;

receiving, at the web server and from the client-computing device, a second request for the web page, wherein the second request comprises the token;

identifying the first test version of the requested web page based on the token; and

providing, to the client-computing device, the first test version of the requested web page.

2. The method as recited in claim 1, further comprising generating a mapping for an A/B test comprising:

receiving A/B test parameters;

generating, utilizing the received A/B test parameters, an A/B test mapping.

3. The method as recited in claim 2, wherein receiving A/B test parameters comprises receiving a listing of the plurality of test versions, and a desired distribution of web page traffic to each of the plurality of test versions.

4. The method as recited in claim 3, wherein generating the A/B test mapping comprises:

generating a plurality of mapping identifiers; and

associating each of the plurality of mapping identifiers to one of the plurality of test versions based on the desired distribution of web page traffic.

5. The method as recited in claim 3, wherein generating a token for the client-computing device comprises:

receiving, from the random number generator, a random number;

generating the token for the client-computing device based on the random number and the A/B test parameters.

6. The method as recited in claim 3, wherein identifying the first test version of the plurality of test versions of the requested web page comprises:

identifying, within the A/B test mapping, a mapping identifier from the plurality of mapping identifiers corresponding to the generated token; and

identifying a test version from the plurality of test versions associated with the mapping identifier.

7. The method as recited in claim 1, wherein providing the first test version of the requested web page and the generated token to the client-computing device comprises generating a browser cookie based on the generated token.

8. In a digital medium environment for serving web pages over a network, a method of conducting A/B testing comprising:

receiving, at a first web server, A/B test parameters that indicate an A/B test mapping of one or more ranges of mapping identifiers associated with one or more test versions of a first A/B tested web page or allow the first web server to generate the mapping;

receiving a first web page request from a client-computing device for the first A/B tested web page, wherein the first web page request is for the first A/B tested web page and comprises a token associated with the client-computing device;

identifying, within the A/B test mapping and based on the token, one of the one or more test versions of the first A/B tested web page; and

providing the identified test version of the first A/B tested web page to the client-computing device.

9. The method as recited in claim 8, wherein identifying one of the one or more test versions of the first A/B tested web page comprises:

identifying a mapping identifier within the A/B test mapping corresponding to the token; and

identifying a test version of the first A/B tested web page associated with the mapping identifier.

10. The method as recited in claim 9, wherein identifying a mapping identifier within the A/B test mapping corresponding to the token comprises identifying a mapping identifier within the A/B test mapping that matches the token.

11. The method as recited in claim 9, wherein identifying a mapping identifier within the A/B test mapping corresponding to the token comprises creating a hash of the token, and identifying the mapping identifier within the A/B test mapping based on the hash.

12. The method as recited in claim 8, further comprising:

identifying, prior to receiving the first web page request from the client-computing device, a plurality of additional web servers sharing a common network with the first web server;

distributing the received A/B test mapping to the additional web servers;

wherein receiving the first web page request is at one of the plurality of additional web servers.

13. The method as recited in claim 9, wherein identifying one of the one or more test versions of the first A/B tested web page further comprises:

identifying a personalization algorithm based on the mapping identifier;

generating a test version of the first A/B tested web page using the personalization algorithm.

14. The method as recited in claim 8, further comprising:

receiving a subsequent A/B test mapping for a second A/B tested web page;

receiving a second web page request from the client-computing device for the second A/B tested web page, wherein the second web page request is for the second A/B tested web page and comprises the token associated with the client-computing device;

identifying, within the subsequent A/B test mapping and based on the token, a test version of the second A/B tested web page; and

providing the identified test version of the second A/B tested web page to the client-computing device.

15. A system for conducting A/B testing comprising:

at least one processor; and

at least one non-transitory computer-readable storage medium storing instructions thereon that, when executed by the at least one processor, cause the system to;

receive, from a client computing device, a first request for a web page;

generate, using a random number generator, a token for the client-computing device;

identify, based on the generated token for the client computing device, a first test version of the requested web page;

provide the first test version of the requested web page and the generated token to the client-computing device;

receive, from the client-computing device, a second request for the web page, wherein the second request comprises the token;

identify the first test version of the requested web page based on the token; and

provide, to the client-computing device, the first test version of the requested web page.

16. The system as recited in claim 15, further storing instructions thereon that, when executed by the at least one processor, cause the system to:

receive A/B test parameters, wherein the A/B test parameters comprise a listing of a plurality of algorithms, and a desired distribution of web page traffic to each of the plurality of algorithms;

generate, utilizing the received A/B test parameters, an A/B test mapping.

17. The system as recited in claim 16, wherein generating the A/B test mapping comprises:

generating a plurality of mapping identifiers; and

associating each of the plurality of mapping identifiers to one of the plurality of algorithms based on the desired distribution of web page traffic.

18. The system as recited in claim 17, wherein generating a token for the client-computing device comprises:

receiving, from the random number generator, a random number; and

generating the token for the client-computing device based on the random number and the A/B test parameters.

19. The system as recited in claim 18, wherein identifying the first test version of the requested web page comprises:

identifying, within the A/B test mapping, a mapping identifier from the plurality of mapping identifiers corresponding to the generated token;

identifying a personalization algorithm from the plurality of algorithms associated with the mapping identifier; and

generating the first test version of the requested web page using the personalization algorithm.

20. The system as recited in claim 19, wherein generating the first test version of the requested web page using the personalization algorithm comprises selecting one or more products, advertisements, images, or videos to include in the first test version of the web page based on user information and one or more parameters specified in the personalization algorithm.