CLOUD-BASED TEST EXECUTION

• US 20150006966A1
• Filed: 09/15/2014
• Published: 01/01/2015
• Est. Priority Date: 09/14/2011
• Status: Active Grant

• Current Assignee: Amazon Technologies, Inc. (Amazon.com, Inc.)
• Original Assignee: Amazon Technologies, Inc. (Amazon.com, Inc.)
• Inventors: DIMITROVICH, SLAVIK, NAGA, VENKATA UDAY KUMAR KALEPALLI, IRANTASH, SIAMAK, COLLADO, MICHAEL L., SINGH, VIJAY P., MOORE, MIKE C.
• Granted Patent: US 9,454,469 B2
• Time in Patent Office: Days
• US Class Current: 714/38.14
• CPC Class Codes: G06F 11/3688 for test execution, e.g. scheduling of test suites
• CPC Class Codes: G06Q 10/0631 Resource planning, allocation, distributing or scheduling for enterprises or organisations
• CPC Class Codes: H04L 67/01 Protocols
• CPC Class Codes: H04L 67/025 for remote control or remote monitoring of applications
• CPC Class Codes: H04L 67/10 in which an application is distributed across nodes in the network software deployment G06F8/60; multiprogramming arrangements G06F9/46

Abstract

In some implementations, a testing service receives a test execution request for executing test operations on a test target. The testing service may map the test execution request to a particular type of supported test framework from among a plurality of types of supported test frameworks. The testing service may obtain a test package provided by a user that requested the testing, such as from a cloud storage location. The testing service determines a computing capacity for executing the testing and appropriates a plurality of workers in a cloud computing service. The testing service configures the plurality of workers for executing the test operations based on at least one of the test framework, the test execution request or the test package. The testing service provides test execution chunks from the test package to the plurality of workers for executing the testing on the test target.

Claims

1. A method comprising:

under control of one or more processors specifically configured with executable instructions,

receiving, by a testing service, a test execution request for executing test operations on a test target;

mapping the test execution request to a supported test framework out of a plurality of supported test frameworks;

obtaining a test package from a cloud storage location, the test package containing one or more test execution chunks;

based on at least one of the test execution request or the test package, determining a computing capacity for executing the test operations;

identifying a plurality of workers in a cloud computing service for use in executing the test operations based, at least in part, on the computing capacity determined for executing the test operations;

configuring the plurality of workers for executing the test operations based on the supported test framework and at least one of the test execution request or the test package; and

providing at least one of the test execution chunks to each worker of the plurality of workers for executing the test operations on the test target.

2. The method as recited in claim 1, the configuring the plurality of workers further comprising configuring the plurality of workers with at least one operating system (OS) and at least one browser using an OS and browser image received with at least one of the test execution request or the test package.

3. The method as recited in claim 1, the configuring the plurality of workers further comprising configuring the plurality of workers with a plurality of different operating systems (OSs) and a plurality of different browsers based, at least in part, on a default browser/OS matrix providing a statistical distribution of browsers and OSs among a designated population or geographic region.

4. The method as recited in claim 3, wherein the test execution request establishes a threshold of statistical likelihood with respect to the default browser/OS matrix to limit a number of browser and OS combinations applied during the configuring the plurality of workers.

5. The method as recited in claim 1, the configuring the plurality of workers further comprising configuring the plurality of workers with one or more plug-ins for the supported test framework identified by the mapping.

6. A method comprising:

under control of one or more processors specifically configured with executable instructions,

receiving a capacity request and test requirements for executing testing on a test target;

appropriating a plurality of workers in a cloud computing service for use in executing the testing based, at least in part, on the capacity request;

configuring the plurality of workers for executing the testing based, at least in part, on the test requirements; and

providing at least one test execution chunk to each worker of the plurality of workers for executing the testing on the test target.

7. The method as recited in claim 6, the appropriating the plurality of workers further comprising at least one of:

appropriating idle workers already allocated;

or

requesting one or more additional workers from the cloud computing service.

8. The method as recited in claim 7, wherein:

at least some of the workers are configured as virtual machines; and

requesting one or more additional workers results in allocation of one or more additional virtual machines at the cloud computing service.

9. The method as recited in claim 6, wherein the capacity request and the test requirements are received from a dispatcher, the dispatcher being configured for execution of a particular test framework from among a plurality of supported test frameworks.

10. The method as recited in claim 6, wherein the test execution chunks are distributed to particular workers of the plurality of workers based, at least in part, on particular configurations of the workers.

11. The method as recited in claim 6, further comprising:

monitoring the workers during the testing; and

sending a notification when a worker malfunction is detected.

12. The method as recited in claim 11, further comprising:

in response to the notification of the worker malfunction, appropriating a new worker from the cloud computing service;

configuring the new worker in a configuration of a malfunctioning worker; and

performing failover from the malfunctioning worker to the new worker to resume the testing.

13. The method as recited in claim 6, the configuring the plurality of workers further comprising configuring at least some of the plurality of workers with at least one operating system (OS) and at least one browser using an OS and browser image provided by a user that requested the testing.

14. The method as recited in claim 6, the configuring the plurality of workers further comprising configuring the plurality of workers with a plurality of different operating systems (OSs) and a plurality of different browsers based, at least in part, on a default browser/OS matrix providing a statistical distribution of browsers and OSs among a designated population or geographic region.

15. The method as recited in claim 14, further comprising configuring the plurality of workers based, at least in part, on a threshold set by a user that requested the testing with respect to the default browser/OS matrix to limit a number of browser and OS combinations applied during the configuring the plurality of workers.

16. The method as recited in claim 6, further comprising:

determining computing resources used during the testing; and

billing a user that requested the testing based on the computing resources used during the testing.

17. The method as recited in claim 16, wherein the billing further takes into consideration at least one of:

a number of cores requested by the user for one or more workers;

an amount of memory requested by the user for one or more workers;

or

an input/output performance requested by the user for one or more workers.

18. One or more computing devices comprising:

one or more processors;

one or more computer-readable media;

a plurality of dispatchers maintained on the computer-readable media and executed on the one or more processors, each dispatcher configured to administer testing in a supported testing framework of a particular type from among a plurality of types of supported testing frameworks; and

an intake component maintained on the computer-readable media and executed on the one or more processors to receive a test execution request and map the test execution request to a particular supported testing framework, the intake component further executed to pass the test execution request to a selected dispatcher corresponding to the particular supported testing framework,

wherein the selected dispatcher is executed on the one or more processors to determine a computing capacity for performing the testing and sending a request for scheduling the computing capacity.

19. The one or more computing devices as recited in claim 18, further comprising a worker manager maintained on the computer readable media and executed on the one or more processors to:

receive the request for scheduling the computing capacity; and

appropriate the computing capacity from a cloud computing service.

20. The one or more computing devices as recited in claim 19, wherein the worker manager is further executed to:

receive test requirements from the selected dispatcher; and

configure the computing capacity at the cloud computing service as a plurality of workers according to the test requirements.

21-25. -25. (canceled)