Distributed test framework for clustered systems

• US 7,165,189 B1
• Filed: 12/19/2003
• Issued: 01/16/2007
• Est. Priority Date: 12/19/2003
• Status: Active Grant

• Current Assignee: Oracle America, Inc. (Oracle Corporation)
• Original Assignee: Sun Microsystems Incorporated (Oracle Corporation)
• Inventors: Kesavan, Ambujavalli, Lakkapragada, Sekhar G.
• Primary Examiner(s): Iqbal; Nadeem
• Application Number:
  US10/741,112
• Time in Patent Office: 1,124 Days
• Field of Search: 714/31, 714/43, 714/48, 714/32, 714/33, 709/220, 709/221
• US Class Current: 714/31
• CPC Class Codes: G06F 11/2294 by remote test

Abstract

Methods and apparatus, including computer program products, implementing and using techniques for testing a computing cluster having several computing nodes. Configuration information regarding a test system is received and used to generate a configuration file that profiles the test system. The configuration file may be passed to each computing machine in the test system whereupon an appropriate test suite is launched on each computing machine in the test system to perform the tests contemplated by the test suite.

Claims

1. A method for testing a computing cluster having a plurality of computing nodes, the method comprising:

receiving selected configuration information regarding a test system including the identification of at least one node of a computing cluster that is part of the test system;

using systems operations to query selected components of the test system, including the computing cluster to obtain additional configuration information regarding the test system, including additional configuration information regarding the computing cluster;

generating a configuration file based on the received and obtained configuration information;

passing the configuration file to each computing machine in the test system; and

launching a test suite on each computing machine in the test system required to perform the tests contemplated by the test suite.

2. A method as recited in claim 1 wherein the test system further includes at least one client computer, the method further comprising querying each client computer to obtain an IP address associated with the client computer.

3. A method as recited in claim 1 wherein the additional configuration information regarding the computing cluster includes a host name for each node in the cluster and a node ID for each node in the cluster.

4. A method as recited in claim 3 wherein each cluster node has a plurality of associated network interface cards and the additional configuration information further includes an identification of the type of each network interface card associated with each cluster node and a count of the number of network interface cards associated with each of the cluster nodes.

5. A method as recited in claim 3 wherein the additional configuration information further includes an identification of any quorum devices associated with the cluster and a count of the cluster nodes with which such quorum device is connected.

6. A method as recited in claim 3 wherein the additional configuration information further includes an identification of a private network switch configuration associated with the cluster.

7. A method as recited in claim 3 wherein the additional configuration information further includes an identification of a public network switch configuration associated with the cluster.

8. A method as recited in claim 1 wherein the test system further includes at least one storage device, the method further comprising querying to determine the number of cluster nodes with which each storage device is connected and to identity which cluster nodes serve as the primary and secondary nodes for each storage device.

9. A method as recited in claim 1 wherein the test system further includes at least one logical host name, the method further comprising validating the logical name.

10. A method as recited in claim 1 wherein the test system further includes at least one external client machine, the method further comprising validating the external client machine and obtaining an IP address for each external client machine.

11. A method as recited in claim 1 wherein the test system fiber includes a reporting tool for reporting the results of the test suite launched and a comparison tool for comparing the results of the test suite launched with a predetermined data set.

12. A test suite support module comprising:

a plurality of global information-gathering tools configured to collect global information associated with a computer cluster testing environment;

a plurality of test suite specific information-gathering tools configured to collect test suite specific information associated with the computer cluster testing environment;

a plurality of generic tools, wherein the generic tools include a summary report generation tool, a plurality of start-up tools, a synchronization tool, and a plurality of cleanup tools.

13. The test suite support module of claim 12 wherein the global information further comprises;

a count of cluster nodes in the computer cluster environment, a host name and a node identification for each of the cluster nodes in the computer cluster environment;

an identification of the type of each network interface card associated with each cluster node and a count of the number of network interface cards associated with each of the cluster nodes an identification of any quorum devices associated with the cluster nodes and a count of the cluster nodes with which such quorum devices are connected;

an identification of a private network switch configuration associated with the cluster; and

an identification of a public network switch configuration associated with the cluster.

14. The test suite support module of claim 12 wherein the test suite specific information further comprises:

a count of cluster nodes with which each storage device is connected and an identification of which cluster nodes serve as the primary and secondary nodes for each storage device;

a determination of a logical host name, and

a determination of an external client machine and an IP address corresponding with the external client machine.

15. The test suite support module of claim 14 wherein the storage devices, the logical host name, and the external client machine are validated.

16. The test suite support module of claim 14 wherein the plurality of start-up tools comprises:

a global start-up tool, a test suite start-up tool, a global storage start-up tool, a logical host name start-up tool, and a client start-up tool.

17. The test suite support module of claim 14 wherein the plurality of cleanup tools include a test suite cleanup tool, and a global cleanup tool.

18. The test suite support module of claim 16 wherein the test suite start-up tool is configured to:

a) create a file system of a predetermined size and type;

b) mount the file system;

c) create files and directories under the mounted file system;

d) export the file system to a client machine;

e) mount the exported file system on the client machine; and

f) load a kernel.

19. The test suite support module of claim 14 wherein the plurality of start-up tools further comprises:

a reporting tool for reporting the results of the test suite launched; and

a comparison tool for comparing the results of the test suite launched with a predetermined data set.

20. A computer-implemented method for testing a computing cluster having a plurality of computing nodes, the method comprising:

automatically receiving selected configuration information regarding a test system including the identification of at least one node of a computing cluster that is part of the test system;

automatically using systems operations to query selected components of the test system, including the computing cluster to obtain additional configuration information regarding the test system, including additional configuration information regarding the computing cluster;

automatically generating a configuration file based on the received and obtained configuration information;

automatically passing the configuration file to each computing machine in the test system; and

automatically launching a test suite on each computing machine in the test system required to perform the tests contemplated by the test suite.

21. A computer program product for testing a computing cluster having a plurality of computing nodes, stored on a machine-readable medium, comprising instructions operable to cause a computer to:

receive selected configuration information regarding a test system including the identification of at least one node of a computing cluster that is part of the test system;

use systems operations to query selected components of the test system, including the computing cluster to obtain additional configuration information regarding the test system, including additional configuration information regarding the computing cluster;

generate a configuration file based on the received and obtained configuration information;

pass the configuration file to each computing machine in the test system; and

launch a test suite on each computing machine in the test system required to perform the tests contemplated by the test suite.

22. A method for a cluster node of a computing cluster having a plurality of cluster nodes being tested, the method comprising:

receiving, by the cluster node, at least one query for configuration information from a test administration machine arranged to test the computer node;

passing, by the cluster node, selected node configuration information to the test administration machine in response to the at least one query received;

receiving, by the cluster node, a configuration file generated by the test administration machine; and

launching, by the cluster node in response to a test administration machine command, a selected test suite using the configuration file generated by the test administration machine.

23. A method as recited in claim 22 wherein the receiving at least one query is generated from systems operations to query selected component of the cluster node to obtain additional configuration information regarding the cluster node.

24. A method as recited in claim 23 wherein the additional configuration information regarding the cluster node includes a host name for the node and a node ID for the node.

25. A method as recited in claim 24 wherein the additional configuration information regarding the cluster node further includes:

an identification of the type of each network interface card associated with the cluster node;

an identification of any quorum devices associated with the cluster node;

an identification of any private network switch configurations associated with the cluster node; and

an identification of any public network switch configurations associated with the cluster node.

26. A method as recited in claim 22 wherein the cluster node is attached with at least one storage device, the method further comprising receiving a plurality of queries from the test administration machine to determine the cluster nodes to which the storage device is attached and to identify which nodes serve as primary and secondary nodes for the storage device.