A/B testing

• US 9,077,780 B2
• Filed: 11/08/2013
• Issued: 07/07/2015
• Est. Priority Date: 01/27/2005
• Status: Active Grant

• Current Assignee: FMR LLC
• Original Assignee: FMR LLC
• Inventors: Dixon, Eleri, Enos, Emily, Brodmerkle, Scott
• Primary Examiner(s): Pollack, Melvin H
• Application Number:
  US14/075,382
• Publication Number:
  US 20140115039A1
• Time in Patent Office: 606 Days
• Field of Search: 709/203, 709/217, 709/219
• US Class Current: 1/1
• CPC Class Codes: G06Q 30/02 Marketing; Price estimation or determination; Fundraising
• CPC Class Codes: G06Q 30/0242 Determining effectiveness of advertisements
• CPC Class Codes: G06Q 30/0243 Comparative campaigns
• CPC Class Codes: G06Q 30/0254 based on statistics
• CPC Class Codes: H04L 67/01 Protocols
• CPC Class Codes: H04L 67/02 based on web technology, e.g. hypertext transfer protocol [HTTP]
• CPC Class Codes: H04L 67/56 Provisioning of proxy services store-and-forward switching systems in data switching networks H04L12/54
• CPC Class Codes: Y10S 715/962 Operator interface for marketing or sales

Abstract

Performing A/B testing of content of a webpage includes selecting one of at least two different versions of the webpage to be displayed to a user through an interactive user interface (e.g., a web browser) based on an identification code, such as an Internet Protocol address or tracker identifier supplied by a cookie, associated with the user.

Claims

1. A method comprising:

sending, by a client computer system to one or more server computer systems, a request for an electronic communication which is subject to a test to identify factors that affect usability of the electronic communication;

receiving, by the client computer system from the one or more server computer systems, a version of the electronic communication;

wherein the received version of the electronic communication comprises at least one of a first version of an electronic communication that satisfies the request with the first version promoting identification of first factors that affect the usability, or a second, different version of the electronic communication that satisfies the request, with the second version promoting identification of second, different factors that affect the usability; and

sending, by the client computer system to the one or more server computer systems, one or more responses to the received electronic communication to promote identification by the one or more server computer systems of one or more factors that increase the usability of the requested electronic communication relative to the usability of the requested electronic communication based on other factors.

2. The method of claim 1, wherein the electronic communication is a web page.

3. The method of claim 1, wherein the version of the electronic communication that is received is selected by the one or more server computer systems from at least the first version of the electronic communication that satisfies the request with the first version promoting identification of first factors that affect the usability, and the second, different version of the electronic communication that satisfies the request, with the second version promoting identification of second, different factors that affect the usability and with selecting being at least partly based on one or more bits of information included in the request.

4. The method of claim 3 wherein the two different versions of the electronic communication are different with respect to a visible interactive element.

5. The method of claim 3, wherein the request includes an identification code with one or more bits of information, with the selected version of the electronic communication is at least partly based on whether one or more values for the one or more bits in the identification code match one or more of a first value associated with the first version of the electronic communication and a second, different value associated with the second version of the electronic communication.

6. The method of claim 5, wherein the one or more bits of information comprise an address of the client computer system.

7. The method of claim 1 wherein receiving the version comprises:

receiving a webpage comprising a script;

executing the script; and

requesting, based on execution of the script, the first version of the electronic communication from a Uniform Resource Location.

8. The method of claim 1 wherein the client computer system comprises one or more of a mobile device, a personal data assistant, and a personal computer.

9. One or more machine-readable hardware storage devices storing instructions that are executable by one or more processing devices of a client computer system to perform operations comprising:

sending, by the client computer system to one or more server computer systems, a request for an electronic communication which is subject to a test to identify factors that affect usability of the electronic communication;

receiving, by the client computer system from the one or more server computer systems, a version of the electronic communication;

wherein the received version of the electronic communication comprises at least one of a first version of an electronic communication that satisfies the request with the first version promoting identification of first factors that affect the usability, or a second, different version of the electronic communication that satisfies the request, with the second version promoting identification of second, different factors that affect the usability; and

sending, by the client computer system to the one or more server computer systems, one or more responses to the received electronic communication to promote identification by the one or more server computer systems of one or more factors that increase the usability of the requested electronic communication relative to the usability of the requested electronic communication based on other factors.

10. The one or more machine-readable hardware storage devices of claim 9, wherein the electronic communication is a web page.

11. The one or more machine-readable hardware storage devices of claim 9, wherein the version of the electronic communication that is received is selected by the one or more server computer systems from at least the first version of an electronic communication that satisfies the request with the first version promoting identification of first factors that affect the usability, and the second, different version of the electronic communication that satisfies the request, with the second version promoting identification of second, different factors that affect the usability and with selecting being at least partly based on one or more bits of information included in the request.

12. The one or more machine-readable hardware storage devices of claim 11, wherein the two different versions of the electronic communication are different with respect to a visible interactive element.

13. The one or more machine-readable hardware storage devices of claim 11, wherein the request includes an identification code with one or more bits of information, with the selected version of the electronic communication is at least partly based on whether one or more values for the one or more bits in the identification code match one or more of a first value associated with the first version of the electronic communication and a second, different value associated with the second version of the electronic communication.

14. The one or more machine-readable hardware storage devices of claim 13, wherein the one or more bits of information comprise an address of at least one of the client computer system.

15. An electronic system comprising:

a client computer; and

one or more machine-readable hardware storage devices storing instructions that are executable by the client computer to perform operations comprising;

sending, by the client computer to one or more server computer systems, a request for an electronic communication which is subject to a test to identify factors that affect usability of the electronic communication;

receiving, by the client computer from the one or more server computer systems, a version of the electronic communication;

wherein the received version of the electronic communication comprises at least one of a first version of an electronic communication that satisfies the request with the first version promoting identification of first factors that affect the usability, or a second, different version of the electronic communication that satisfies the request, with the second version promoting identification of second, different factors that affect the usability; and

sending, by the client computer to the one or more server computer systems, one or more responses to the received electronic communication to promote identification by the one or more server computer systems of one or more factors that increase the usability of the requested electronic communication relative to the usability of the requested electronic communication based on other factors.

16. The electronic system of claim 15, wherein the electronic communication is a web page.

17. The electronic system of claim 15, wherein the version of the electronic communication that is received is selected by the one or more server computer systems from at least the first version of an electronic communication that satisfies the request with the first version promoting identification of first factors that affect the usability, and the second, different version of the electronic communication that satisfies the request, with the second version promoting identification of second, different factors that affect the usability and with selecting being at least partly based on one or more bits of information included in the request.

18. The electronic system of claim 17, wherein the two different versions of the electronic communication are different with respect to a visible interactive element.

19. The electronic system of claim 17, wherein the request includes an identification code with one or more bits of information, with the selected version of the electronic communication is at least partly based on whether one or more values for the one or more bits in the identification code match one or more of a first value associated with the first version of the electronic communication and a second, different value associated with the second version of the electronic communication.

20. The electronic system of claim 19, wherein the one or more bits of information comprise an address of the client computer.