Automated testing of applications in cloud computer systems

• US 8,826,068 B2
• Filed: 11/23/2011
• Issued: 09/02/2014
• Est. Priority Date: 11/23/2011
• Status: Active Grant

• Current Assignee: Microsoft Technology Licensing LLC (Microsoft Corporation)
• Original Assignee: Microsoft Corporation
• Inventors: Somendra, Aruna
• Primary Examiner(s): PATEL, KAMINI B
• Application Number:
  US13/303,175
• Publication Number:
  US 20130132774A1
• Time in Patent Office: 1,014 Days
• Field of Search: 714/32
• US Class Current: 714/6.3
• CPC Class Codes: G06F 11/3688 for test execution, e.g. scheduling of test suites
• CPC Class Codes: G06F 11/3692 for test results analysis
• CPC Class Codes: H04L 41/046 comprising network management agents or mobile agents therefor
• CPC Class Codes: H04L 41/0806 for initial configuration or provisioning, e.g. plug-and-play
• CPC Class Codes: H04L 41/40 using virtualisation of network functions or resources, e.g. SDN or NFV entities
• CPC Class Codes: H04L 43/20 the monitoring system or the monitored elements being virtualised, abstracted or software-defined entities, e.g. SDN or NFV
• CPC Class Codes: H04L 43/50 Testing arrangements

Abstract

A system and method for performing automated testing of an application in a cloud environment. A controller initializes an manages a number of virtual machines (VM), each VM including a test engine. The controller retrieves configuration data, determines a number of VMs to deploy, and initializes the VMs. The controller manages each VM by providing test commands and monitoring the results. Each VM receives and executes the test commands. The system may be used to test interactive applications or non-interactive applications.

Claims

1. A computer-based method of performing an automated test of an computer application in a cloud environment, the method comprising:

a) receiving a specification of a test scenario, the test scenario including a plurality of commands;

b) automatically performing a calculation to determine a number of virtual machines to deploy based on a hardware limitation, a number of connections supported, or a resource limitation of each virtual machine;

c) deploying the determined number of virtual machines;

d) initializing a test agent on each deployed virtual machine; and

e) for each test agent, providing to the test agent one or more commands from the test scenario and selectively providing one or more additional commands based on a result of executing the one or more commands.

2. The computer-based method of claim 1, further comprising:

a) receiving a specification of a second test scenario, the second test scenario including a plurality of commands;

b) for each test agent, providing a set of one or more commands from the second test scenario.

3. The computer-based method of claim 1, further comprising storing the test scenario specification in a cloud storage system.

4. The computer-based method of claim 1, providing to each test agent the one or more commands comprises storing into a cloud storage system each of the one or more commands.

5. The computer-based method of claim 1, the computer application being an interactive application, at least one of the one or more commands is an interactive command.

6. A computer-based system, comprising:

a) a cloud computing platform including a cloud storage system; and

b) a controller configured to perform actions, including;

i) receiving a specification of a test scenario that includes a set of commands, each command of the set of commands having a specification of whether to execute the command serially;

ii) performing a calculation to determine a number of virtual machines to deploy based on a hardware limitation, a number of connections supported, or a resource limitation of each virtual machine;

iii) using the cloud computing platform to deploy the number of virtual machines;

iv) providing, to a test agent on each of the number of virtual machines, the set of commands, by storing each command of the set of commands in the cloud storage system; and

v) retrieving a result of each command by each test agent by retrieving the result from the cloud storage system; and

c) one or more physical processors configured to implement the cloud computing platform or the controller.

7. The computer-based system of claim 6, further comprising a set of virtual machines having the determined number of virtual machines, each virtual machine programmed to retrieve the set of commands from the cloud storage system, execute each command, and store the result of each command in the cloud storage system.

8. The computer-based system of claim 6, one or more of the test commands are interactive commands.

9. The computer-based system of claim 6, each test command having a corresponding specification of whether to execute the test command in parallel with one or more other test commands.

10. A computer-readable hardware storage medium having stored thereon computer program instructions for performing an automated test of an application program, the program instructions executable by one or more processors to perform actions including:

a) performing a calculation to determine a number of virtual machines to deploy based on a hardware limitation, a number of connections supported, or a resource limitation of each virtual machine;

b) using a cloud computer platform to deploy a set of virtual machines based on the determined number;

c) using the cloud computer platform to provide test commands to each virtual machine of the set of virtual machines by storing the test commands in cloud storage; and

d) employing the cloud storage to monitor results of each of the test command.

11. The computer-readable hardware storage medium of claim 10, the cloud computer platform is Windows Azure.

12. The computer-readable hardware storage medium of claim 10, determining the number of virtual machines to deploy is based on a virtual processor speed or a virtual memory capacity, or resource limitation of each virtual machine.

13. The computer-readable hardware storage medium of claim 10, the actions further comprising retrieving, from the cloud storage, a scenario specification that includes the test commands.

14. The computer-readable hardware storage medium of claim 10, one or more of the test commands are interactive commands.

15. The computer-readable hardware storage medium of claim 10, the actions further comprising:

a) retrieving, from the cloud storage, a first scenario specification that includes the test commands;

b) retrieving, from the cloud storage, a second scenario specification that includes additional test commands; and

c) performing another automated test by providing the additional test commands to each virtual machine of the set of virtual machines.