System and method for assigning computer users to test groups

• US 8,214,804 B2
• Filed: 01/14/2008
• Issued: 07/03/2012
• Est. Priority Date: 12/31/2007
• Status: Active Grant

• Current Assignee: Overstock.com Incorporated
• Original Assignee: Overstock.com Incorporated
• Inventors: Robertson, Ian B.
• Primary Examiner(s): Zhen, Wei
• Assistant Examiner(s): Wu, Junchun
• Application Number:
  US12/014,007
• Publication Number:
  US 20090183084A1
• Time in Patent Office: 1,632 Days
• Field of Search: 717/124
• US Class Current: 717/124
• CPC Class Codes: G06Q 10/06 Resources, workflows, human or project management; Enterprise or organisation planning; Enterprise or organisation modelling

Abstract

A system and method to efficiently and randomly place users into a test group associated with a test running on a website being hosted on a server is disclosed. When a user first accesses the website, a random value is transmitted to the user in such a manner that the random value is stored in a computer-readable memory local to the user. Each subsequent time the user accesses the website, the random value is returned to the server from the user. The server may then place the user into a test group for a test to which the user will be exposed on the website based upon the result of an operation that includes the random value received from the user and a unique test value previously associated with the test.

Claims

1. A computer-implemented method of conducting a test on a testable characteristic of a webpage transmitted to a plurality of users, said method comprising:

assigning the test a unique identification value;

generating a plurality of random numbers using a number generation module;

transmitting one of the plurality of random numbers to each of the plurality of users over a computer network;

receiving a random value and a request for the webpage from each of the plurality of users over the computer network;

and

determining a function that uses the random value received from the user and the unique identification value for the test to which the user is exposed to place into a test group;

serving a version of the webpage to the each user that includes a variant of the testable characteristic as dictated by the test group to which the user pertains.

2. The method of claim 1, wherein said unique test value is a prime number.

3. The method of claim 2, wherein said unique test value comprises at least five digits.

4. The method of claim 1, wherein the function comprises (n mod p), where n is the random value received from the user and p is the unique test value associated with the test.

5. The method of claim 1, wherein the function comprises:

6. The method of claim 1, wherein the function comprises:

7. The method of claim 1, further comprising the step of transmitting a random value to each of the plurality of users such that said random value is stored in a computer readable memory of said user.

8. The method of claim 1, further comprising the step of transmitting a random value to each of the plurality of users as part of a cookie.

9. The method of claim 1, wherein said random value is not generated by the user.

10. The method of claim 1, further comprising the step of notifying a tracking system that each of the plurality of users has been exposed to the variant of the testable characteristic corresponding to the test group to which the user pertains.

11. A computer-implemented method of running a plurality of tests on a website, a plurality of users accessing said website, said method comprising the steps of:

(a) associating a unique test value with each of the plurality of tests;

(b) defining test groups for each of the plurality of tests;

(c) receiving a random value from each user accessing the website; and

(d) placing each of the plurality of users that will be exposed to a given test into a test group for the given test based upon the random value received from that user and the unique test value associated with the given test;

associating a numerical range with each of the plurality of tests; and

assigning the test groups to a portion of the numerical range of the test to which they pertain.

12. The method of claim 11, further comprising the step of transmitting a random value to each of the plurality of users accessing the website and causing the random value to be stored in a computer-readable memory of the user.

13. The method of claim 12, further comprising the step of causing the random value to be stored in memory at each of the plurality of users in a cookie.

14. The method of claim 11, wherein step (d) further comprises the steps of:

calculating a test placement value from a function that uses the random value and the unique test value for the given test;

placing the user into the test group assigned to the portion of the numerical range into which the test placement value falls.

15. The method of claim 14, wherein the function comprises (n mod p), where n is the random value and p is the unique test value.

16. The method of claim 14, wherein the function comprises:

17. The method of claim 14, wherein the function comprises:

18. The method of claim 11, further comprising the step of notifying a tracking system of each of the plurality of tests to which each of the plurality of users has been exposed.

19. A computer program on a non-transitory machine-readable storage medium for running a test on a website, said computer program comprising:

a test configuration module for associating a unique test value with the test;

a number generation module for generating a random number;

a server module for transmitting the random value to a user accessing the website and causing the random value to be stored in a computer-readable memory local to the user;

said server module further operable to receive the random value back from the user each time the user accesses the website;

a calculation module for determining a test placement value based on the random value received back from the user and the unique test value using a predefined function; and

a placement module for assigning the user to a test group of the test based upon the test placement value;

wherein said server module is further operable to send a version of a webpage to the user that includes a variant of the test corresponding to the test group to which the user has been assigned.

20. The computer program of claim 19, wherein the predefined function comprises (n mod p), where n is the random value and p is the unique test value.

21. The computer program of claim 19, wherein the predefined function comprises:

22. The computer program of claim 19, wherein the predefined function comprises:

23. The computer program of claim 19, further comprising a tracking module for tracking an exposure of the user to the test.

24. The computer program of claim 19, wherein the unique test value is a prime number.

25. The computer program of claim 19, wherein the number generation module is further operable to generate the unique test value associated with the test.

26. A computer-implemented method of conducting a test on a testable characteristic of a webpage, said method comprising:

receiving a random value and a request for the webpage from a user;

determining a test group for the user using said random value;

serving a version of the webpage to the user that includes a variant of the testable characteristic as dictated by the test group to which the user pertains;

determining the test group for the user using said random value and a unique test value associated with said test in a function;

wherein the function comprises;

27. A computer-implemented method of conducting a test on a testable characteristic of a webpage, said method comprising:

receiving a random value and a request for the webpage from a user;

determining a test group for the user using said random value;

serving a version of the webpage to the user that includes a variant of the testable characteristic as dictated by the test group to which the user pertains;

determining the test group for the user using said random value and a unique test value associated with said test in a function;

wherein the function comprises;