Cloud-based test execution

• US 9,454,469 B2
• Filed: 09/15/2014
• Issued: 09/27/2016
• Est. Priority Date: 09/14/2011
• Status: Active Grant

• Current Assignee: Amazon Technologies, Inc. (Amazon.com, Inc.)
• Original Assignee: Amazon Technologies, Inc. (Amazon.com, Inc.)
• Inventors: Naga, Venkata Uday Kumar Kalepalli, Collado, Michael L., Moore, Mike C., Dimitrovich, Slavik, Irantash, Siamak, Singh, Vijay P.
• Primary Examiner(s): Lottich, Joshua P
• Application Number:
  US14/486,734
• Publication Number:
  US 20150006966A1
• Time in Patent Office: 743 Days
• Field of Search: 714/25, 714/26, 714/27, 714/38.1
• US Class Current: 1/1
• CPC Class Codes: G06F 11/3688 for test execution, e.g. scheduling of test suites
• CPC Class Codes: G06Q 10/0631 Resource planning, allocation, distributing or scheduling for enterprises or organisations
• CPC Class Codes: H04L 67/01 Protocols
• CPC Class Codes: H04L 67/025 for remote control or remote monitoring of applications
• CPC Class Codes: H04L 67/10 in which an application is distributed across nodes in the network software deployment G06F8/60; multiprogramming arrangements G06F9/46

Abstract

In some implementations, a testing service receives a test execution request for executing test operations on a test target. The testing service may map the test execution request to a particular type of supported test framework from among a plurality of types of supported test frameworks. The testing service may obtain a test package provided by a user that requested the testing, such as from a cloud storage location. The testing service determines a computing capacity for executing the testing and appropriates a plurality of workers in a cloud computing service. The testing service configures the plurality of workers for executing the test operations based on at least one of the test framework, the test execution request or the test package. The testing service provides test execution chunks from the test package to the plurality of workers for executing the testing on the test target.

Claims

1. A system, comprising:

one or more processors; and

a memory coupled to the one or more processors and storing program instructions that when executed by the one or more processors cause the one or more processors to;

receive, by a testing service, a test execution request to perform a test on a test target;

determine, in response to receiving the test execution request, a test execution framework of a plurality of supported test execution frameworks to use in performing the test on the test target;

determine, in response to receiving the test execution request, one or more respective combinations of an operating system and a browser to be used to perform the test on the test target, wherein the operating system is determined from a plurality of operating systems supported by the testing service and the browser is determined from a plurality of browsers supported by the testing service;

configure respective workers of a plurality of workers with respective combinations of the one or more determined combinations of operating system and browser; and

configure the plurality of workers to perform the test on the test target according to the determined test execution framework.

2. The system of claim 1, wherein the one or more combinations of operating system and browser are determined based on a user specification included in the test execution request.

3. The system of claim 1, wherein the one or more combinations of operating system and browser are determined based at least in part on one or more combinations of operating systems and browsers associated with a geographic region indicated in the test execution request.

4. The system of claim 1, wherein the test execution framework to be used to perform the test is determined based at least in part on previously stored test instructions.

5. A non-transitory, computer-readable storage medium storing program instructions that when executed by one or more computers cause the one or more computers to:

receive, by a testing service, a test execution request to perform a test on a test target;

determine, in response to receiving the test execution request, one or more combinations of an operating system and a browser to be used to perform the test on the test target, wherein the operating system is determined from a plurality of supported operating systems and the browser is determined from a plurality of supported browsers; and

configure respective workers of a plurality of workers with respective combinations of the one or more determined combinations of operating system and browser.

6. The non-transitory, computer-readable storage medium of claim 5, wherein to determine the one or more combinations of an operating system and a browser the program instructions further cause the one or more computers to:

select the one or more combinations of operating system and browser based on specified combinations of operating systems and browsers included in the test execution request.

7. The non-transitory, computer-readable storage medium of claim 6, wherein the test execution request further includes one or more specified characteristics of a worker that is to execute the test using a particular one of the one or more combinations of an operating system and a browser, and

wherein the program instructions further cause the one or more computers to configure a worker with the one or more specified characteristics to perform the test with the particular combination of operating system and browser.

8. The non-transitory, computer-readable storage medium of claim 5, wherein to determine the one or more combinations of operating system and browser the program instructions further cause the one or more computers to:

select the one or more combinations of operating system and browser based on combinations of operating systems and browsers associated with respective geographic locations.

9. The non-transitory, computer-readable storage medium of claim 8, wherein the one or more combinations of operating system and browser associated with the respective geographic locations are determined based on combinations of operating systems and browsers used in the respective geographic locations being used above a threshold amount.

10. The non-transitory, computer-readable storage medium of claim 5, wherein the one or more combinations of operating system and browser to be used to perform the test are based on one or more browser images received in the test execution request.

11. The non-transitory, computer-readable storage medium of claim 5, wherein the test execution request specifies a future time or schedule for performing the test, and wherein the program instructions further cause the test to be performed at the future time or according to the schedule.

12. The non-transitory, computer-readable storage medium of claim 5, wherein the program instructions further cause the one or more computers to:

configure one or more respective workers with a plug-in, wherein the plug-in allows for a test execution service to control a respective operating system and browser combination configured on the one or more respective workers.

13. A method, comprising:

receiving, by a testing service implemented by one or more computers, a test execution request to perform a test on a test target;

determining, by the testing service, a test execution framework from among a plurality of supported test execution frameworks supported by the testing service, wherein the determined test execution framework is to be used in performing the test; and

configuring, by the testing service, a plurality of workers to perform the test on the test target according to the determined test execution framework.

14. The method of claim 13, wherein said determining a test execution framework from among a plurality of supported test execution frameworks is based at least in part on one or more test execution frameworks specified in the test execution request.

15. The method of claim 14, wherein the one or more specified test execution frameworks in the test execution request are stored by the testing service are and used to determine a test execution framework for performing future tests.

16. The method of claim 13, further comprising:

determining, subsequent to determining the test execution framework to be used to perform the test, one or more dispatchers configured to conduct the test according to the determined test execution framework of a plurality of dispatchers, wherein the plurality of dispatchers comprises respective dispatchers configured to perform testing according to respective test execution frameworks.

17. The method of claim 13, wherein the plurality of test execution framework includes at least one of:

TestNG, Selenium, or Junit.

18. The method of claim 13, further comprising:

performing the test based on a predetermined schedule.

19. The method of claim 13, further comprising:

determining a plurality of dispatchers to concurrently perform the test according to a plurality of test execution frameworks, wherein individual ones of the plurality of dispatchers are configured to perform the test according to different respective test execution frameworks.

20. The method of claim 13, further comprising:

configuring one or more additional workers to perform the test on the test target in response to determining one or more of the plurality of workers is not performing the test on the test target.