Application testing

• US 10,284,664 B2
• Filed: 10/13/2014
• Issued: 05/07/2019
• Est. Priority Date: 10/13/2014
• Status: Active Grant

• Current Assignee: Microsoft Technology Licensing LLC (Microsoft Corporation)
• Original Assignee: Microsoft Technology Licensing LLC (Microsoft Corporation)
• Inventors: Slivkins, Aleksandrs, Agarwal, Alekh, Langford, John, Bird, Sarah L., Sen, Siddhartha, Li, Lihong, Dudik, Miroslav, Bottou, Leon
• Primary Examiner(s): St Leger, Geoffrey R
• Assistant Examiner(s): Luu, Cuong V
• Application Number:
  US14/513,012
• Publication Number:
  US 20160105351A1
• Time in Patent Office: 1,667 Days
• CPC Class Codes: H04L 67/02 based on web technology, e.g. hypertext transfer protocol [HTTP]
• CPC Class Codes: H04L 67/34 involving the movement of software or configuration parameters network booting or remote initial program loading [RIPL] G06F9/4416
• CPC Class Codes: H04L 67/535 Tracking the activity of the user network monitoring arrangements H04L43/00; recording of computer activity G06F11/34

Abstract

The claimed subject matter includes techniques for providing an application testing service. An example method includes receiving context information from a client system, the context information comprising parameters that describe details of a user interaction with an application under test (AUT). The method also includes receiving a set of potential actions from the client system. The method also includes identifying a selected action from the set of potential actions and sending the selected action to the client system, wherein the AUT is customized based on the selected action. The method also includes receiving reward data from the client system based on a user'"'"'s interaction with the AUT. The method also includes storing the context information, the selected action, and the reward data to a log of application test data.

Claims

1. A method, comprising:

receiving context information from a client system at an application testing service located remotely from the client system, wherein the client system is running an application under test (AUT) that is accessible to a user from a user system, the context information comprising parameters that describe details of an interaction with the AUT by the user accessing the AUT from the user system;

receiving a set of potential actions from the client system at the application testing service;

identifying a selected action from the set of potential actions and sending the selected action from the application testing service to the client system, wherein the AUT is customized to present the selected action to the user;

receiving reward data from the client system at the application testing service wherein the reward data is generated by the client system and describes one or more details about the user'"'"'s interaction with the customized AUT, wherein the reward data is used to evaluate an effectiveness of the selected action based on the user'"'"'s interaction with the customized AUT; and

storing the context information, the selected action, a probability of the selected action describing a likelihood that the selected action will be selected by the user, and the reward data to a log of application test data.

2. The method of claim 1, comprising:

receiving a hypothetical policy from the client system;

obtaining data from the log of application test data, the data comprising log entries that include context information that matches a context parameter of the hypothetical policy; and

sending the data to the client system.

3. The method of claim 1, comprising:

receiving a hypothetical policy and a reward function from the client system, wherein the reward function is a mathematical formula for computing a reward statistic that describes an effectiveness of the hypothetical policy;

obtaining data from the log of application test data, the data comprising log entries that include context information that matches a context parameter of the hypothetical policy;

computing the reward statistic from the data according to the reward function; and

sending the reward statistic to the client system.

4. The method of claim 1, wherein identifying the selected action comprises executing a fixed algorithm that does not adapt to previously acquired reward data.

5. The method of claim 1, wherein identifying the selected action comprises choosing the selected action randomly from the set of potential actions.

6. The method of claim 1, wherein identifying the selected action comprises eliminating actions from the set of potential actions to generate a reduced action set and choosing the selected action randomly from the reduced action set.

7. The method of claim 1, wherein identifying the selected action comprises choosing the selected action based, at least in part, on previously received reward data.

8. The method of claim 1, comprising receiving an identification of a selected decision algorithm from the client system, and using the selected decision algorithm for identifying the selected action.

9. The method of claim 1, comprising:

receiving two or more hypothetical policies and a reward function from the client system, wherein the reward function is a mathematical formula for computing a reward statistic that describes an effectiveness of each hypothetical policy;

obtaining reward data relevant to the two or more hypothetical policies from the log of application test data;

computing the reward statistic for each of the two or more hypothetical policies from the reward data according to the reward function; and

identifying a effectiveness of each of the hypothetical policies.

10. The method of claim 1, comprising:

receiving a default policy, a threshold performance level, and an exploratory budget from the AUT; and

if the performance of the AUT falls below the threshold performance level or exhausts the exploratory budget, using the default policy to identify the selected action.

11. The method of claim 1, wherein the selected action identifies content to be presented to the user and the reward data describes whether the user clicked on a link associated with the content.

12. The method of claim 1, wherein the selected action identifies a news article to be presented to the user and the reward data describes whether the user clicked on a link associated with the news article.

13. The method of claim 1, wherein the selected action identifies an advertisement to be presented to the user and the reward data describes whether the user clicked on a link associated with the advertisement.

14. The method of claim 1, wherein the selected action identifies a product to be presented to the user, and the reward data indicates whether the user purchased the product.

15. A system for testing an application, comprising:

a processor; and

a system memory comprising code to direct the processor to;

receive context information from a client system, wherein the client system is running an application under test (AUT) that is accessible to a user from a user system, the context information comprising parameters that describe details of an interaction with the AUT by the user accessing the AUT from the user system;

receive a set of potential actions from the client system;

identify a selected action from the set of potential actions and send the selected action to the client system, wherein the AUT is customized to present the selected action to the user;

receive reward data from the client system wherein the reward data is generated by the client system and describes one or more details about the user'"'"'s interaction with the customized AUT, wherein the reward data is used to evaluate an effectiveness of the selected action based on the user'"'"'s interaction with the customized AUT; and

store the context information, the selected action, a probability of the selected action describing a likelihood that the selected action will be selected by the user, and the reward data to a log of application test data.

16. The system of claim 15, the system memory comprising code to direct the processor to:

receive a hypothetical policy from the client system;

obtain data from the log of application test data, the data comprising log entries that include context information that matches a context parameter of the hypothetical policy; and

send the data to the client system.

17. The system of claim 15, the system memory comprising code to direct the processor to:

receive a hypothetical policy and a reward function from the client system, wherein the reward function is a mathematical formula for computing a reward statistic that describes an effectiveness the hypothetical policy;

obtain reward data from the log of application test data, the reward data comprising log entries that include context information that matches a context parameter of the hypothetical policy;

compute the reward statistic from the reward data according to the reward function; and

send the reward statistic to the client system.

18. The system of claim 15, wherein the code to identify the selected action directs the processor to execute a fixed algorithm that does not adapt to previously acquired reward data.

19. The system of claim 15, wherein the code to identify the selected action directs the processor to choose the selected action randomly from the set of potential actions.

20. The system of claim 15, wherein the code to identify the selected action directs the processor to:

eliminate actions from the set of potential actions based, at least in part, on previously received reward data to generate a reduced action set; and

and choose the selected action randomly from the reduced action set.

21. The system of claim 15, the system memory comprising code to receive an identification of a selected decision algorithm from the client system, and use the selected decision algorithm to identify the selected action.

22. The system of claim 15, the system memory comprising code to:

receive two or more hypothetical policies and a reward function from the client system, wherein the reward function is a mathematical formula for computing a reward statistic that describes an effectiveness of a policy;

obtain reward data relevant to the two or more hypothetical policies from the log of application test data;

compute the reward statistic for each of the two or more hypothetical policies from the reward data according to the reward function; and

identify the effectiveness of each of the hypothetical policies.

23. The system of claim 15, the system memory comprising code to:

receive a default policy, a threshold performance level, and an exploratory budget from the AUT; and

if the performance of the AUT falls below the threshold performance level or exhausts the exploratory budget, use the default policy to identify the selected action.

24. One or more computer-readable memory storage devices for storing computer-readable instructions that, when executed, instruct one or more processing devices to:

receive context information from a client system, wherein the client system is running an application under test (AUT) that is accessible to a user from a user system, the context information comprising parameters that describe details of an interaction with the AUT by the user accessing the AUT from the user system;

receive a set of potential actions from the client system;

identify a selected action from the set of potential actions and send the selected action to the client system, wherein the AUT is customized to present the selected action to the user;

receive reward data from the client system wherein the reward data is generated by the client system and describes one or more details about the user'"'"'s interaction with the customized AUT, wherein the reward data is used to evaluate an effectiveness of the selected action based on the user'"'"'s interaction with the customized AUT; and

store the context information, the selected action, a probability of the selected action describing a likelihood that the selected action will be selected by the user, and the reward data to a log of application test data.

25. The one or more computer-readable memory storage devices of claim 24, wherein the computer-readable instructions that instruct the one or more processing devices to identify a selected action, randomly choose an action from the set of potential actions.

26. The one or more computer-readable memory storage devices of claim 24, comprising computer-readable instructions that instruct the one or more processing devices to receive an identification of a decision algorithm from the client system, wherein the identified decision algorithm is used to identify the selected action.

27. The one or more computer-readable memory storage devices of claim 24, comprising computer-readable instructions that instruct the one or more processing devices to compare previously stored context information, selected actions, and reward data to identify the selected action based on which action is more likely to elicit a useful response from the user.

28. The one or more computer-readable memory storage devices of claim 24, comprising computer-readable instructions that instruct the one or more processing devices to:

receive a hypothetical policy and a reward function from a client system, wherein the reward function is a mathematical formula for computing a reward statistic that describes an effectiveness of the hypothetical policy;

obtain reward data from the log of application test data, the data comprising log entries that include context information that matches a context parameter of the hypothetical policy;

compute the reward statistic from the reward data according to the reward function; and

send the reward statistic to the client system.

29. The one or more computer-readable memory storage devices of claim 24, comprising computer-readable instructions that instruct the one or more processing devices to:

receive two or more hypothetical policies and a reward function from the client system, wherein the reward function is a mathematical formula for computing a reward statistic that describes an effectiveness of each of the hypothetical policies;

obtain reward data relevant to the two or more hypothetical policies from the log of application test data;

compute the reward statistic for each of the two or more hypothetical policies from the reward data according to the reward function; and

identify the effectiveness of each of the hypothetical policies.

30. The one or more computer-readable memory storage devices of claim 24, comprising computer-readable instructions that instruct the one or more processing devices to:

receive a default policy, a threshold performance level, and an exploratory budget from the AUT; and

if the performance of the AUT falls below the threshold performance level or exhausts the exploratory budget, use the default policy to identify the selected action.