Automated system that tests software on multiple computers

• US 20030131285A1
• Filed: 01/10/2002
• Published: 07/10/2003
• Est. Priority Date: 01/10/2002
• Status: Active Grant

• Current Assignee: Microsoft Technology Licensing LLC (Microsoft Corporation)
• Original Assignee: Microsoft Corporation
• Inventors: Taylor, Jason A., Mitchell, David C., Dibee, Joseph M., Kirkpatrick, Brad P., Elkins, Donald R., Aaron, Freddie L., Stathakopoulos, George N., Lai, Richie, Ingman, Eric W., Beardsley, Eric A.
• Granted Patent: US 7,437,713 B2
• Time in Patent Office: Days
• US Class Current: 714/38
• CPC Class Codes: G06F 11/3664 Environments for testing or debugging software
• CPC Class Codes: G06F 11/3672 Test management
• CPC Class Codes: H04L 41/024 using relational databases for representation of network management data, e.g. managing via structured query language [SQL]
• CPC Class Codes: H04L 43/50 Testing arrangements

Abstract

A system by which a software product may be tested on multiple client computers on various platforms. Product developers submit requests for tests on their products, in the form of test packets, to a test component, via an API. For each platform and language (i.e., group) on which a product developer wants a product tested, the product developer provides a test packet that defines tests that the product developer wants conducted on the product in that group. The test component searches, via an autolab component, for an available client machine for performing the tests in the test packet. The autolab component finds an available client computer, and the client machine is assigned the test packet. The client machine performs the tasks in the test packet, and forwards the results back to the test component.

Claims

1. A computer test system, comprising:

an interface configured to receive a request for performance of test jobs on multiple machines, each of the test jobs including a defined platform for performance of the test jobs; and

an autolab component configured to select one of the multiple machines as a selected machine based upon a platform on the selected machine, and to act on the request by assigning at least one of the test jobs to the selected machine.

2. The computer test system of claim 1, wherein the autolab component comprises a management component that is configured to separate one of the test jobs into subtasks, and to order the subtasks into a reordered job.

3. The computer test system of claim 2, wherein the management component is configured to separate a plurality of the test jobs into subtasks, and to order the subtasks of the plurality of test jobs into a reordered job.

4. The computer test system of claim 3, wherein the management component is configured to add a subtask corresponding to a computing environment.

5. The computer test system of claim 1, wherein the test component is configured to create a personalized test package for the selected machine based upon the platform and applications available at the client machine.

6. The computer test system of claim 5, further comprising a component for defining a time limit for execution of the test job, and wherein the autolab component is configured to reconfigure the test job to execute within the defined time limit.

7. The computer test system of claim 1, further comprising a database component associated with the test component for storing the test jobs.

8. The computer test system of claim 7, wherein the database is configured to store a particular test job in a pending status prior to the particular test job being assigned to one of the multiple machines.

9. The computer test system of claim 7, wherein the database is configured to store a particular test job in an assigned status while the particular test job is assigned to one of the multiple machines.

10. The computer test system of claim 7, wherein the database is configured to store a particular test job in a completed status after the particular test job has been run by one of the multiple machines.

11. The computer test system of claim 1, further comprising a message queue for the selected machine and that is associated with the autolab component, the message queue for storing information about test jobs that have been assigned to the selected machine.

12. The computer test system of claim 1, further comprising a high-level interface that permits direct access between the autolab component and at least one of the multiple machines.

13. The computer test system of claim 12, further comprising a thin client that is configured for communicating between the high-level interface and the multiple machines, the thin client being configured to translate information from a client machine to information that may be utilized by the high-level interface.

14. The computer test system of claim 1, wherein the autolab component selects the selected machine based upon the availability thereof.

15. The computer test system of claim 1, wherein the autolab component selects the selected machine based upon the present imaging of the selected machine.

16. A computer test system comprising:

storage a request for performance of test jobs on multiple machines, each of the test jobs including a defined platform for performance of the test jobs; and

an autolab component configured to select one of the multiple test machines as a selected machine based upon availability thereof, and to act on the request by assigning at least one of the test jobs to the selected machine.

17. The computer test system of claim 16, wherein the autolab component comprises a management component that is configured to separate one of the test jobs into subtasks, and to order the subtasks into a reordered job.

18. The computer test system of claim 17, wherein the management component is configured to separate a plurality of the test jobs into subtasks, and to order the subtasks of the plurality of test jobs into a reordered job.

19. The computer test system of claim 18, wherein the management component is configured to add a subtask corresponding to a computing environment.

20. The computer test system of claim 16, wherein the test component is configured to create a personalized test package for the selected machine based upon the platform and applications available at the client machine.

21. The computer test system of claim 20, further comprising a defined time limit for execution of the test job, and wherein the autolab component is configured to reconfigure the test job to execute within the defined time limit.

22. The computer test system of claim 16, wherein the storage is configured to store a particular test job in a pending status prior to the particular test job being assigned to one of the multiple machines.

23. The computer test system of claim 16, wherein the storage is configured to store a particular test job in an assigned status while the particular test job is assigned to one of the multiple machines.

24. The computer test system of claim 16, wherein the storage is configured to store a particular test job in a completed status after the particular test job has been run by one of the multiple machines.

25. The computer test system of claim 16, further comprising a component for storing a message queue for the selected machine that is associated with the autolab component, the component for storing the message queue being configured to store information about test jobs that have been assigned to the selected machine.

26. The computer test system of claim 16, further comprising a high-level interface component that permits direct access between the autolab component and at least one of the multiple machines.

27. The computer test system of claim 26, further comprising a thin client that is configured for communicating between the high-level interface component and the multiple machines, the thin client being configured to translate information from a client machine to information that may be utilized by the high-level interface component.

28. A computer system comprising:

a plurality of test machines;

a computer test system comprising;

storage for a request for the performance of test jobs on multiple machines, each of the test jobs including a defined platform for performance of the test jobs; and

an autolab component configured to select one of multiple test machines based upon the platform thereon, and to act on the request by assigning at least one of the test jobs to the selected machine.

29. The computer system of claim 28, wherein at least one of the plurality of machines is located remotely from the test system.

30. A computer system, comprising:

a test component configured to receive a request for the performance of a test job on a computer;

a manager component for causing the test job to be performed; and

an image component in which the test job is conducted.

31. The computer system of claim 30, wherein the test component comprises a communication component configured to communicate with a test system and receiving the test job from the test system.

32. The computer system of claim 31, wherein the communication component is configured to ping the test component for a new test job when idle.

33. The computer system of claim 31, wherein the communication component is configured to send the results of the test job to the test component upon completion of the test job.

34. The computer system of claim 31, wherein the computer system is located remote of the test system, and wherein the communication component is configured to communicate with the test component through a high-level interface.

35. The computer system of claim 31, wherein the computer system is located remote of the test system, and wherein the communication component is configured to communicate with the test component through an Internet protocol.

36. The computer system of claim 30, wherein the test job includes a status request for an indications of a portion of the test job being completed, and wherein the manager component is configured to send a message to the test component upon the portion being completed.

37. The computer system of claim 30, wherein the manager component is configured to send a message to the test component upon completion of the test job.

38. The computer system of claim 37, wherein the message includes the results of the test.

39. The computer system of claim 30, wherein the test component, manager component, and the image component are located in a single computer.

40. The computer system of claim 39, wherein the test component, manager component, and the image component are each located on separate partitions of the single computer.

41. The computer system of claim 39, wherein the test component and the image component are located on separate partitions of the single computer.

42. A computer-implemented method comprising,

receiving a request for performance of test jobs on multiple machines, each of the test jobs including a defined platform for performance of the test jobs;

selecting one of the multiple machines as a selected machine based upon a platform on the selected machine; and

acting on the request by assigning one of the test jobs to the selected machine.

43. The method of claim 42, further comprising separating one of the test jobs into subtasks, and ordering the subtasks into a reordered job.

44. The method of claim 43, further comprising separating a plurality of the test jobs into subtasks, and ordering the subtasks of the plurality of test jobs into a reordered job.

45. The method of claim 43, further comprising adding a subtask corresponding to a computing environment.

46. The method of claim 42, further comprising creating a personalized test package for the selected machine based upon the platform and applications available at the client machine.

47. A computer-readable medium having computer-executable instructions for performing the method of claim 42.