SOFTWARE OR HARDWARE TEST APPARATUS AND METHOD

• US 20070244663A1
• Filed: 04/12/2006
• Published: 10/18/2007
• Est. Priority Date: 04/12/2006
• Status: Active Grant

• Current Assignee: ATI Technologies ULC (Advanced Micro Devices, Inc.)
• Original Assignee: ATI Technologies Incorporated (Advanced Micro Devices, Inc.)
• Inventors: Waters, Zack, Haemel, Nicholas
• Granted Patent: US 7,945,416 B2
• Time in Patent Office: Days
• US Class Current: 702/121
• CPC Class Codes: G06F 11/26 Functional testing
• CPC Class Codes: G06F 11/3688 for test execution, e.g. scheduling of test suites

Abstract

A software or hardware test system and method repeatedly obtains testing status of a plurality of test units in a group while the test units are testing hardware or software being executed on the test units. The system and method provides for display of the current testing status of the plurality of units of the group while the plurality of test units is performing software testing. In another embodiment, a test system and method compiles heuristic data for a plurality of test units that are assigned to one or more groups of test units. The heuristic data may include, for example, data representing a frequency of use on a per-test unit basis over a period of time, and other heuristic data. The test system and method evaluates job queue sizes on a per-group basis to determine whether there are under-utilized test units in the group and determines on a per-group of test unit basis whether a first group allows for dynamic reassignment of a test unit in the group based on at least the compiled heuristic data. If the unit is allowed to be dynamically reassigned, the test system and method reassigns the test unit whose heuristic data meets a specific set of criteria.

Claims

1. A software or hardware test apparatus comprising:

one or more processors; and

memory containing executable instructions that when executed cause the one or more processors to;

repeatedly obtain testing status of a plurality of test units in a group while the test units are testing software being executed thereon or hardware; and

provide for display, the current testing status of the plurality of units of the group while the plurality of test units are performing software or hardware testing.

2. The software or hardware test apparatus of claim 1 wherein the memory contains executable instructions that when executed cause the one or more processors to at least determine testing status as to whether a test unit is idle for testing software or hardware and determine, on a per test unit basis, an amount of time remaining for a particular test set to complete and provide the testing status and data representing the amount of time remaining for the particular software or hardware test to complete for display on a display.

3. The software or hardware test apparatus of claim 1 wherein the memory contains executable instructions that when executed cause the one or more processors to generate a real time status registry that contains asynchronously received status data from all test units in the group and is updated continuously during testing.

4. The software or hardware test apparatus of claim 1 wherein the memory contains executable instructions that when executed cause the one or more processors to:

determine whether a first test unit assigned to a first group of test units is available for reassignment to a second group of test units;

if so update a test coordinator registry with an identifier associated with the second group for the first test unit; and

cause the first unit to perform a test as a test unit of the second group.

5. A software or hardware test apparatus comprising:

one or more processors; and

memory containing executable instructions that when executed cause the one or more processors to;

compile heuristic data for a plurality of test units that are assigned to one or more groups of test units wherein the heuristic data comprises at least data representing a frequency of use on a per test unit basis over a period of time;

evaluate job queue sizes on a per group basis to determine whether there are under utilized test units in the group;

determine on a per group of test unit basis whether a first group allows dynamic reassignment of a test unit in the group based on at least the compiled heuristic data; and

if so, reassigning the test unit whose heuristic data meets a specific set of criteria.

6. The software or hardware test apparatus of claim 5 wherein the memory contains executable instructions that when executed cause the one or more processors to:

determine on a per group of test unit basis whether a first group allows dynamic reassignment of a test unit in the group also based on at least whether a test unit has hardware, software and an operating system compatible for a job of interest.

7. The software or hardware test apparatus of claim 5 wherein the memory contains executable instructions that when executed cause the one or more processors to reassign the test unit whose heuristic data meets a specific set of criteria to a different group of test units by updating a test coordinator registry with a new group identifier for the test unit to be reassigned.

8. The software or hardware test apparatus of claim 5 wherein the memory contains executable instructions that when executed cause the one or more processors to determine if a test unit has undergone a software or hardware malfunction and in response remotely cycle power on the test unit using a remotely addressable power supply.

9. The software or hardware test apparatus of claim 5 wherein the memory contains executable instructions that when executed cause the one or more processors to determine whether a new job to be assigned to a group of test units has a higher priority than a current job being executed;

if so halting the current job and storing state information of the current job;

then executing the new job and restoring the state information of the current job after the new job has executed.

10. The software or hardware test apparatus of claim 5 wherein the memory contains executable instructions that when executed cause the one or more processors to compile heuristic data for the plurality of test units wherein the heuristic data comprises at least data representing:

how often a test unit is used relative to other test units in the same group, an average test runtime for all tests run on a test unit in a group, an average time a test unit is idle in a group, and a per group average job queue lengths.

11. The software or hardware test apparatus of claim 5 wherein the memory contains executable instructions that when executed cause the one or more processors to:

store last known good results based on a prior software test;

using stored last known good results as a test result baseline for a subsequent test;

running the subsequent test and determining whether there are any newly passing portions of the test;

if so updating the last known good results to include the newly passing portions on a per group basis; and

using the updated last known good result as a baseline for yet another subsequent test that is run by a test unit in the group.

12. The software or hardware test apparatus of claim 5 wherein the memory contains executable instructions that when executed cause the one or more processors to provide selectable test control data to select test replication control data that replicates a test job across a plurality of test units in a group, resulting in each test unit executing each test in the job, or to select test distribution data to allow distribution of tests from a test job to different test units in a group, resulting in each test in the job being executed by one test unit.

13. The software or hardware test apparatus of claim 5 wherein the memory contains executable instructions that when executed cause the one or more processors to:

present a graphic user interface on a display wherein the graphic user interface comprises data representing potential tests to be selected for execution on a group of test units and data representing which of the potential tests are currently available to be run by a current group of test units and data representing selected tests from the potential tests that are to be run on different groups of test units; and

uploading new user specified tests and displaying them as being available for use by other users.

14. A method for testing software or hardware comprising:

repeatedly obtaining testing status of a plurality of test units in a group while the test units are testing software being executed thereon; and

providing for display, the current testing status of the plurality of units of the group while the plurality of test units are performing software or hardware testing.

15. The method of claim 14 comprising at least determining testing status as to whether a test unit is idle or testing software or hardware and determine, on a per test unit basis, an amount of time remaining for a particular software test to complete and providing the testing status and data representing the amount of time remaining for the particular software test to complete for display on a display.

16. The method of claim 14 comprising asynchronously generating a real time status registry that contains status data for all test units in the group and is updated continuously during testing.

17. The method of claim 14 comprising:

determining whether a first test unit assigned to a first group of test units is available for reassignment to a second group of test units;

if so updating a test coordinator registry with an identifier associated with the second group for the first test unit; and

causing the first unit to perform tests as a test unit of the second group.

18. A method for testing software or hardware comprising:

compiling heuristic data for a plurality of test units that are assigned to one or more groups of test units wherein the heuristic data comprises at least data representing a frequency of use on a per test unit basis over a period of time;

evaluating job queue sizes on a per group basis to determine whether there are under utilized test units in the group;

determining on a per group of test unit basis whether a first group allows dynamic reassignment of a test unit in the group based on at least the compiled heuristic data; and

if so, reassigning the test unit whose heuristic data meets a specific set of criteria.

19. The method of claim 18 comprising determining on a per group of test unit basis whether a first group allows dynamic reassignment of a test unit in the group also based on at least whether a test unit is hardware and operating system compatible for a job of interest.

20. The method of claim 18 comprising reassigning the test unit, whose heuristic data meet certain criteria, to a different group of test units by updating a test coordinator registry with a new group identifier for the test unit to be reassigned.

21. The method of claim 18 comprising determining if a test unit has experienced a software or hardware malfunction and in response remotely cycle power on the test unit using a remotely addressable power supply.

22. The method of claim 18 comprising determining whether a new job to be assigned to a group of test units has a higher priority than a current job being executed;

if so halting the current job and storing state information of the current job;

then executing the new job and restoring the state information of the original job after the new job has executed.

23. The method of claim 18 comprising compiling heuristic data for the plurality of test units wherein the heuristic data comprises at least data representing:

how often a test unit is used relative to other test units in the same group, an average test runtime for all tests run on a test unit in a group, an average time a test unit is idle in a group, and a per group average job queue lengths.

24. The method of claim 18 comprising:

storing last known good results based on a prior software or hardware test set;

using stored last known good results as a test set result baseline for a subsequent tests;

running the subsequent test set and determining whether there are any newly passing portions of the test set;

if so updating the last known good results to include the newly passing portions on a per group basis; and

using the updated last known good result as a baseline for yet another subsequent test set that is run by a test unit group.

25. The method of claim 18 comprising providing selectable test control data to select test replication control data that replicates a test job across a plurality of test units in a group, resulting in each test unit executing each test in the job, or to select test distribution data to allow distribution of tests from a test job to different test units in a group, resulting in each test in the job being executed by one test unit.

26. The method of claim 18 comprising:

presenting a graphic user interface on a display wherein the graphic user interface comprises data representing potential tests to be selected for execution on a group of test units and data representing which of the potential tests are currently available to be run by a current group of test units and data representing selected tests from the potential tests that may be run on different groups of test units; and

uploading new user specified tests and displaying them as being available for use by other users.