System and method for measuring performance with distributed agents

• US 20040199815A1
• Filed: 04/02/2003
• Published: 10/07/2004
• Est. Priority Date: 04/02/2003
• Status: Active Grant

• Current Assignee: Oracle America, Inc. (Oracle Corporation)
• Original Assignee: Sun Microsystems Incorporated (Oracle Corporation)
• Inventors: Dinker, Darpan, Bhoopathy, Kannan, Pandey, Dhirendra
• Granted Patent: US 7,178,065 B2
• Time in Patent Office: Days
• US Class Current: 714/21
• CPC Class Codes: G06F 11/3428 Benchmarking
• CPC Class Codes: G06F 11/3433 for load management allocation of a server based on load conditions G06F9/505; load rebalancing G06F9/5083; redistributing the load in a network by a load balancer H04L67/1029
• CPC Class Codes: G06F 11/3457 Performance evaluation by simulation
• CPC Class Codes: G06F 11/3495 for systems
• CPC Class Codes: G06F 2201/87 Monitoring of transactions

Abstract

Various systems and methods for testing one or more servers using a distributed test system may involve a master agent synchronously transitioning multiple test agents through several state changes. In some embodiments, a method may involve configuring multiple test agents to execute a test by initiating a state change to a first state at each of the test agents. Each of the test agents is prepared to execute the test when in the first state. Each of the test agents simulates multiple clients of a server under test when executing the test. In response to each of the test agents confirming the state change to the first state, a state change to a second state may be initiated at each of the plurality of test agents. Each of the test agents executes the test when in the second state.

Claims

1. A method, comprising:

configuring a plurality of test agents to execute a test by initiating a state change to a first state at each of the plurality of test agents, wherein each of the test agents is prepared to execute the test when in the first state, and wherein each of the test agents simulates a plurality of clients of a server under test when executing the test;

in response to each of the test agents confirming the state change to the first state, initiating a state change to a second state at each of the plurality of test agents, wherein each of the test agents executes the test when in the second state.

2. The method of claim 1, further comprising accessing a cluster configuration file to identify each of the plurality of test agents and the master agent.

3. The method of claim 2, further comprising the plurality of test agents confirming each state change by sending a communication to the master agent via a server port identified in the cluster configuration file.

4. The method of claim 1, wherein said initiating the state change to the first state is performed by a master agent included in a same cluster as the plurality of test agents sending a state change request and configuration information associated with the test to each of the plurality of test agents.

5. The method of claim 4, wherein the configuration information includes one or more class names, a number of threads, a number of iterations, and a maximum execution time.

6. The method of claim 4, further comprising the master agent reading the configuration information from a configuration file prior to sending the state change request.

7. The method of claim 6, further comprising the master agent reading the configuration file subsequent to completion of the test;

and

if the configuration file includes other configuration information for an other test, the master agent sending the state change request and the other configuration information to the plurality of test agents in order to configure the plurality of test agents to perform the other test.

8. The method of claim 4, further comprising each of the plurality of test agents confirming the state change to the second state.

9. The method of claim 4, further comprising each of the plurality of test agents sending the master agent test results in response to completing execution of the test.

10. The method of claim 9, further comprising the master agent storing the test results received from each of the plurality of test agents in a location specified in a configuration file.

11. The method of claim 9, wherein the test results received from each of the plurality of test agents include a number of iterations of the test executed by each of a plurality of test threads and a total execution time of each of the plurality of test threads.

12. The method of claim 9, further comprising the master agent aggregating the test results from each of the plurality of test agents.

13. The method of claim 9, further comprising the master agent performing statistical analysis on the test results received from each of the plurality of test agents.

14. The method of claim 1, wherein the plurality of test agents simulates a plurality of clients of a plurality of servers under test when executing the test.

15. The method of claim 1, each of the test agents simulates the plurality of clients of the server under test when executing the test by communicating with the server under test via the Internet.

16. The method of claim 1, further comprising the plurality of test agents simulating a plurality of clients of a different server under test in response to being configured to execute a different test.

17. The method of claim 1, wherein each test agent simulates the plurality of clients by executing a corresponding plurality of test threads.

18. The method of claim 1, further comprising ending execution of the test at each other one of the plurality of test agents in response to one of the plurality of test agents failing to confirm the state change to the second state.

19. A method, comprising:

testing a server under test by configuring each of a plurality of test agents to execute a test and initiating execution of the test at each of the plurality of test agents, wherein each of the plurality of test agents simulates a plurality of clients of the server under test when executing the test;

detecting an error in at least one of the plurality of test agents during said testing;

in response to said detecting, failing said testing prior to each test agent completing execution of the test.

20. A computing device, comprising:

at least one processor; and

a memory coupled to the at least one processor, wherein the memory stores program instructions executable by the at least one processor to implement a master agent;

wherein the master agent is configured to;

initiate a state change to a first state at each of a plurality of test agents in order to configure the plurality of test agents to execute a test, wherein each of the plurality of test agents is prepared to execute the test when in the first state, and wherein at least one of the test agents is implemented on a different computing device;

in response to each of the test agents confirming the state change, initiate a state change to a second state at each of the plurality of test agents, wherein each of the plurality of test agents is configured to simulate a plurality of clients of a server under test by executing the test when in the second state.

21. The computing device of claim 20, wherein the master agent is configured to initiate the state change to the first state at each of the plurality of test agents by sending a state change request and configuration information associated with the test to each of the plurality of test agents.

22. The computing device of claim 21, wherein the configuration information includes one or more class names, a number of threads, a number of iterations, and a maximum execution time.

23. The computing device of claim 21, wherein the master agent is configured to read the configuration information from a configuration file stored in the memory prior to sending the state change request.

24. The computing device of claim 21, wherein the master agent is configured to access the configuration file subsequent to completion of the test and, if the configuration file includes other configuration information for an other test, the master agent is configured to send the state change request and the other configuration information to the plurality of test agents in order to configure the plurality of test agents to perform the other test.

25. The computing device of claim 21, wherein in response to each of the test agents completing execution of the test, the master agent is configured to receive test results from each of the plurality of test agents.

26. The computing device of claim 25, wherein the master agent is configured to store the test results in a location specified in a configuration file stored within the memory.

27. The computing device of claim 25, wherein the test results received from each of the plurality of test agents include a number of iterations of the test executed by each of a plurality of test threads and a total execution time of each of the plurality of test threads.

28. The computing device of claim 25, wherein the master agent is configured to aggregate the test results received from each of the plurality of test agents.

29. The computing device of claim 25, wherein the master agent is configured to perform statistical analysis on the test results received from each of the plurality of test agents.

30. The computing device of claim 20, wherein the master agent is configured to end execution of the test at each other one of the plurality of test agents in response to one of the plurality of test agents failing to confirm the second state change.

31. A computing device, comprising:

at least one processor; and

a memory coupled to the at least one processor, wherein the memory stores program instructions executable by the at least one processor to implement a master agent;

wherein the master agent is configured to test a server under test by distributing test parameters to each of a plurality of test agents and subsequently initiating execution of a test at each of the plurality of test agents, wherein each of the test agents simulates a plurality of clients of the server under test when executing the test;

wherein the master agent is configured to fail testing in response to detecting an error in one of the plurality of test agents while testing the server under test.

32. A system, comprising:

a plurality of test agents each configured to simulate a plurality of clients of a server under test during execution of a test;

a master agent coupled to the plurality of test agents and configured to perform testing of the server under test by initiating execution of the test at each of the plurality of test agents;

wherein the master agent is configured to fail testing of the server under test in response to detecting an error in one of the plurality of test agents.

33. A system, comprising:

a plurality of test agents each configured to simulate a plurality of clients of a server under test during execution of a test;

a master agent coupled to the plurality of test agents and configured to initiate a state change to a first state at each of the plurality of test agents, wherein each of the plurality of test agents is prepared to perform the test when in the first state;

wherein in response to each of the plurality of test agents confirming the state change to the first state, the master agent is configured to initiate a state change to a second state at each of the plurality of test agents, wherein each of the plurality of test agents is configured to execute the test when in the second state.

34. The system of claim 33, wherein the plurality of test agents are configured to confirm each state change by sending a communication to the master agent via a server port identified in a cluster configuration file.

35. The system of claim 33, wherein the master agent is configured to initiate the state change to the first state by sending a state change request and configuration information associated with the test to each of the plurality of test agents.

36. The system of claim 35, wherein the configuration information includes one or more class names, a number of threads, a number of iterations, and a maximum execution time.

37. The system of claim 35, wherein the master agent is configured to read the configuration information from a configuration file prior to sending the state change request.

39. The system of claim 33, wherein in response to each of the test agents completing execution of the test, the master agent is configured to initiate a state change to a third state at each of the plurality of test agents, wherein when in the third state, each of the plurality of test agents is configured to send the master agent test results generated by executing the test.

40. The system of claim 39, wherein the master agent is configured to store the test results in a location specified in a configuration file.

41. The system of claim 39, wherein the test results received from each of the plurality of test agents include a number of iterations of the test executed by each of a plurality of test threads and a total execution time of each of the plurality of test threads.

42. The system of claim 39, wherein the master agent is configured to aggregate the test results received from each of the plurality of test agents.

43. The system of claim 39, wherein the master agent is configured to perform statistical analysis on the test results received from each of the plurality of test agents.

44. The system of claim 33, wherein the plurality of test agents is configured to simulate a plurality of clients of a plurality of servers under test.

45. The system of claim 33, wherein the plurality of test agents are each configured to simulate the plurality of clients of the server under test by communicating with the server under test via the Internet.

46. The system of claim 33, wherein the plurality of test agents is configurable to simulate clients of a different server under test by being configured to perform a different test.

47. The system of claim 33, wherein each test agent is configured to run multiple test threads during execution of the test.

48. The system of claim 33, wherein the master agent is configured to end execution of the test at each other one of the plurality of test agents in response to one of the plurality of test agents failing to confirm a state change initiated by the master agent.

49. The system of claim 33, wherein the master agent is configured to initiate execution of a single iteration of a single instance of the test in a single one of the plurality of test agents.

50. The system of claim 33, wherein the plurality of test agents is implemented from a generic test framework useable to perform a plurality of different tests.

38. The system of claim 38, wherein the master agent is configured to read the configuration file subsequent to completion of the test;

and

if the configuration file includes other configuration information for an other test, the master agent is configured to send the state change request and the other configuration information to the plurality of test agents in order to configure the plurality of test agents to perform the other test.

51. A computer accessible medium comprising program instructions, wherein the program instructions are executable to:

configure a plurality of test agents to execute a test by initiating a state change to a first state at each of the plurality of test agents, wherein each of the plurality of test agents is prepared to execute a test when in the first state, and wherein each of the plurality of test agents simulates a plurality of clients of a server under test during execution of the test;

in response to each of the test agents confirming the state change to the first state, initiate execution of the test at each of the plurality of test agents by initiating a state change to a second state at each of the plurality of test agents, wherein each of the plurality of test agents executes the test when in the second state.