Automated Test Management System and Method
First Claim
1. A computer-implemented method for managing selection of a test for execution on a test device in a network, comprising:
- receiving information of a test queue comprising a plurality of tests for execution on the test device, each of the plurality of tests having a required device configuration associated therewith;
determining a first device configuration of the test device, the first device configuration being a present configuration of the test device;
querying the test queue to identify a test that can be executed on the test device;
receiving information of an identified first test in the queue that can be executed in its entirety by the test device in the first device configuration; and
receiving information of an second test in the queue that can be executed by the test device in a second device configuration if the identified first test is not present in the queue, the second device configuration being a new configuration of the test device, the second device configuration including at least one of the required operating system, the required language, and the required application of the required device configuration.
4 Assignments
0 Petitions
Accused Products
Abstract
A test management application on a test management server includes a user interface on a Web-based portal by which a user can define one or more tests, selecting any desired configuration of operating system, connection type, and/or application, which are then saved in a test management database in the central server. Multiple tests involving the same configuration can be defined and saved for later selection, either individually or as a group of tests. A client agent engine on a test device can query the test management server for tests that can be conducted using the device'"'"'s current configuration. If no such tests are found, the device can then query the test management server for the next available test. Upon allocation of the next available test to the device, the necessary system configuration for that test can be automatically retrieved, installed, and verified by the device. The device under test is automatically rebuilt to have the proper configuration for the test to be run.
311 Citations
32 Claims
-
1. A computer-implemented method for managing selection of a test for execution on a test device in a network, comprising:
-
receiving information of a test queue comprising a plurality of tests for execution on the test device, each of the plurality of tests having a required device configuration associated therewith; determining a first device configuration of the test device, the first device configuration being a present configuration of the test device; querying the test queue to identify a test that can be executed on the test device; receiving information of an identified first test in the queue that can be executed in its entirety by the test device in the first device configuration; and receiving information of an second test in the queue that can be executed by the test device in a second device configuration if the identified first test is not present in the queue, the second device configuration being a new configuration of the test device, the second device configuration including at least one of the required operating system, the required language, and the required application of the required device configuration. - View Dependent Claims (2, 3, 4, 5, 6)
-
-
7. A computer-implemented method for managing testing of a test device in a network, comprising:
-
receiving, at the test device, information regarding a test for execution on the test device, the test having a required configuration associated therewith, the required configuration including at least one of a required operating system, a required language, and a required application; determining, at the test device, a first configuration of the test device; determining whether the first configuration of the test device conforms to the required configuration; automatically reconfiguring the test device to the required configuration if the first configuration does not conform to the required configuration; and executing the test on the test device. - View Dependent Claims (8)
-
-
9. A computer-implemented method for automatically managing testing in a network, comprising:
-
creating, at a test management server, a test suite comprising a plurality of test cases, each of the plurality of test cases being configured to test a specified aspect of a test device; and creating, at the test management server, a test database comprising a plurality of test cycles, each of the test cycles comprising a plurality of test configurations comprising a respective test suite and respective required device configuration of the test device; receiving, at the test management server, a query of the test database from the test device for one of the plurality of test configurations for execution by the test device, the query including information of a present device configuration of the test device; returning an answer to the query, the answer comprising a selected test configuration, the selected test configuration comprising a corresponding selected test suite and a corresponding required device configuration; and transmitting the selected test configuration to the test device for execution, the selected test configuration including components of the required device configuration if the device configuration of the test device does not include all components of the required device configuration. - View Dependent Claims (10, 11)
-
-
12. A computer-implemented method for validating performance of a test device under a test, comprising:
-
receiving, at the test device, a test script for the test, the test script including information of a predefined validation criterion for the test; receiving, at the test device, information of a performance of the test device in response to the test; and applying, at the test device, the validation criterion to the performance information to determine a result of the test. - View Dependent Claims (13, 14, 15, 16)
-
-
17. A system for automatic end-to-end management of testing of a device in a network, comprising:
-
a central test management server and a test device connected to the test management server via a communications link; the test management server including a test management database of test suites, each test suite comprising a plurality of test cases configured to test a specified aspect of a test device, the test management database further comprising a plurality of test configurations; and the test device including a client agent application configured to determine a current configuration of the test device and to query the test management database for a test configuration for execution on the test device, the query including information of a current configuration of the test device; wherein the test management server processes the query from the test device and in response to the query returns an allocated test configuration comprising an allocated test suite and information of an associated required device configuration, an identity of the allocated test configuration being based on the current configuration of the test device and the required device configuration; and wherein the allocated test configuration comprises a first allocated test suite if the current configuration of the test device conforms to the required device configuration; and further wherein the allocated test configuration comprises a second allocated test suite and at least one component of a required device configuration associated with the second allocated test suite if the current configuration of the test device does not conform to the required device configuration of the second allocated test suite. - View Dependent Claims (18, 19, 20)
-
-
21. A system for automated end-to-end management of testing of a device in a network, comprising:
-
a central test management server and a test device connected to the central test management server by a communications link; the test management server including a user interface, a test management database, a central file system, and a central scripts database; and the test device including a client agent engine, a test device re-imaging tool, a local file system, a scripting engine, and a local scripts database; the user interface being configured to receive a selection of at least one of a test case, a test suite, and a test configuration for execution on the test device, each of the test case, test suite, and test configuration having a required device including at least one of a required operating system, a required language, and a required application associated therewith; the test management database being configured to store information of the at least one test case, test suite, and test configuration therein, the stored information including the associated required device configuration for each test case, test suite, and test configuration; the central file system being configured to store files of the required operating system, language, and application for each required device configuration and being further configured to communicate with a local file system on the test device; the client agent engine being configured to select one of the test case, test suite, and test configuration from the test database and being further configured to determine whether a present configuration of the test device conforms to the required device configuration of the selected test case, test suite, or test configuration; the test device re-imaging tool being configured to re-image the test device according to the required device configuration if the present configuration of the test device does not conform to the required device configuration of the selected test case, test suite, or test configuration, the re-imaging including an installation of at least one of the required operating system, language, and application, the files therefor being transferred from the central file system to the local file system on the test device; and the scripting engine being configured to execute the selected one of a test case, test suite, test configuration on the test device, a script for the selected test case, test suite, or test configuration being transferred from the central scripts database on the central server to the local scripts database on the test device, the scripting engine being further configured to report a result of the executed test to the test management server; wherein the test device executes a test from the test management database in accordance with a device configuration of the test device and reports the result of the executed test to the test management server. - View Dependent Claims (22)
-
-
23. A computer program product including a computer storage medium comprising one of volatile media and non-volatile media, and a computer program code mechanism embedded in the computer storage medium for facilitating automated management of testing of a test device in a computer network, comprising:
-
a computer code device configured to store a plurality of tests in a test queue, each of the plurality of tests having a required device configuration associated therewith; a computer code device configured to determine a first device configuration of the test device, the first device configuration being a present configuration of the device; a computer code device configured to query the test queue to identify a test that can be executed on the test device, the query including information of the first device configuration of the test device; a computer code device configured to identify a first test in the queue that can be executed by the test device in the first device configuration; a computer code device configured to identify a second test in the queue that can be executed by the test device in a second device configuration if there is not a first test that can be identified; a computer code device configured to download the identified test from a central server for the test device; and a computer code device configured to reconfigure the test device from the first device configuration to the second device configuration if the second is downloaded; wherein the test device can execute the downloaded test.
-
-
24. A computer-implemented method for managing testing of a test device in a network, comprising:
-
storing, at a central test server, a plurality of test queues of tests for execution on a plurality of test devices; receiving, at the central test server, a query from one of the plurality of test devices for a test for execution on the test device, the query including authentication information regarding one of the test device and a user of the test device; verifying, at the central test server, the authentication information; determining an identity of a test queue from the plurality of test queues that contains a test that can be executed on the test device; determining an identity of a test to be allocated to the test device; and allocating the identified test to the test device in accordance with the authentication information. - View Dependent Claims (25, 26, 27, 28)
-
-
29. A computer-implemented method for automatically managing distributed testing on a plurality of test devices in a network, comprising:
-
creating, at a central test management server, a plurality of test assets, the test assets comprising at least one of a test case, a test suite, a test configuration, a test cycle, and a test queue; and organizing the plurality of test assets into at least one group in accordance with an organization scheme, the organization scheme further having at least one associated permission; wherein the test assets can be accessed only in accordance with the permission associated with the group. - View Dependent Claims (30, 31)
-
-
32. A test management system for managing testing of test devices in a hosted environment, comprising:
-
a central hosted test management server, the server including a central test database comprising a plurality of tests; and a plurality of test sites, each of the test sites including at least one test device for execution of tests from the central test database, each of the test sites further including a site-specific central file system for receiving information from the central test database, the information including a plurality of test scripts and build information for reconfiguration of the test device in accordance with a required device configuration associated with at least one of the test scripts; wherein each of the plurality of test sites is allocated site-specific tests from the central test database based on an identity of the test site; wherein the test devices in each of the plurality of test sites can be reconfigured in accordance with the required device configuration using information from the site-specific central file system; and wherein the test devices in each of the plurality of test sites can execute at least one of the plurality of test scripts in the site-specific central file system.
-
Specification