Method and system for software object testing

• US 6,934,934 B1
• Filed: 01/12/2000
• Issued: 08/23/2005
• Est. Priority Date: 08/30/1999
• Status: Expired due to Term

• Current Assignee: Oracle International Corporation (Oracle Corporation)
• Original Assignee: Empirix Incorporated (Empirix Holdings I, Inc.)
• Inventors: Friedman, George, Glik, Michael V., Osborne, Theodore M. II, Baker, Caren H., Vahey, Walter G.
• Primary Examiner(s): Nguyen-Ba, Antony
• Assistant Examiner(s): STEELMAN, MARY J
• Application Number:
  US09/482,178
• Time in Patent Office: 2,050 Days
• Field of Search: 717/124, 717/125, 717/127, 717/128, 717/130, 717/135, 717/106, 717/109, 717/126, 709/224, 718/100, 719/315
• US Class Current: 717/126
• CPC Class Codes: G06F 11/3414 Workload generation, e.g. scripts, playback
• CPC Class Codes: G06F 11/3419 by assessing time
• CPC Class Codes: G06F 11/3495 for systems
• CPC Class Codes: G06F 2201/875 Monitoring of systems including the internet

Abstract

A system for testing middleware of applications in the N-tiered model. The test system contains test code generators, test engines to execute multiple copies of the test code and a data analyzer to analyze and present the results to a human user. The system is able to automatically generate test code to exercise components of the middleware using information about these components that would otherwise be available to the application under test. Multiple copies of the test code are executed in a synchronized fashion. Execution times of multiple events are recorded and then presented in one of several formats. With the system, an application developer can identify components that represent performance bottlenecks or can gather information on deployment properties of individual components that can be used to enhance the performance of the application under test.

Claims

1. A method of testing a technology based object on oriented software component of a computerized application under test that allows simultaneous users over a computer network, the method comprising the steps of:

a) providing test code automatically generated from analysis of the technology based object oriented software component that exercises said technology based object oriented software component of the application under test;

b) synchronizing and simultaneously executing a plurality of instances of the test code remotely located at a server, wherein said test code tests said technology based object oriented software component of the application under test over said computer network and recording performance data on said technology based object oriented software component of the application under test;

c) repeating step b) multiple times, with a different number of instances of the test code; and

d) analyzing the recorded performance data to indicate a performance characteristic of said technology based object oriented software component of the application under test in response to load.

2. The method of claim 1 wherein the application under test is an object oriented language and the step of providing test code comprises providing test code to test one object oriented software component of the application.

3. The method of claim 1 wherein the step of synchronizing comprises starting each instance of the test code at the same time.

4. The method of claim 1 wherein the step of synchronizing and executing comprises executing a portion of the plurality of instances of test code remotely located on a first computer and a portion of the plurality of instance of test code remotely located on a second computer connected to the network.

5. The method of claim 1 wherein the step of analyzing includes preparing a graphical display having as an independent variable the number of instances of the test code and the dependent variable is the performance data.

6. The method of claim 1 wherein the step of analyzing includes preparing a graphical display having as an independent variable the number of instances of the test code and the dependent variable is derived from the performance data.

7. The method of claim 1 wherein the application under test is resident on a first server within the network and the application has a remote interface and the test code is resident on at least a second computer within the network and exercises the object oriented software component of the application under test using the remote interface of the application under test.

8. The method of claim 1 wherein the step of analyzing includes displaying the analyzed data to a human user using a graphical user interface.

9. A method of testing a technology based object oriented software component of a computerized application under test that allows simultaneous users over a computer network, the method comprising the steps of:

a) specifying test conditions through a user interface to a test system;

b) initiating, across a network, through a user interface to the test system the gathering of test data on the performance of at least one technology based object oriented software component of the application under test at a plurality of load conditions, the load conditions including simultaneously running a plurality of instances of test code automatically generated from analysis of the technology based software component;

c) specifying through a user interface to the test system the output format of the test data; and

d) displaying in the specified format the response of said at least one technology based object oriented software component of the application under test to load.

10. The method of claim 9 wherein the specified format is a graphical format indicating response time as a function of load conditions.

11. The method of claim 10 wherein the specified graphical format is a Hi-Lo plot.

12. The method of claim 10 wherein the step of gathering data under a plurality of load conditions comprises initiating the execution of a plurality of copies of a test program, with the number of copies executing simultaneously relates to the load condition.

13. The method of claim 12 wherein the step of specifying an output format includes specifying a method by which response is measured.

14. The method of claim 12 wherein the step of gathering test data includes recording the execution time between selected points in the test program for each simultaneously executing copy of the test program and analyzing the recorded execution times for all copies of the test program.

15. The method of claim 14 wherein the step of analyzing comprises determining the average and maximum execution times for each of the load conditions.

16. The method of claim 9 wherein:

a) the computerized application under test comprises software resident on a server controlling access to a computerized database;

b) the server is connected to a network and the application under test is simultaneously accessed by a plurality of clients over the network; and

c) the test system is resident on at least a second server connected to the network.

17. A method of testing a technology based object oriented software component of a computerized application under test that allows simultaneous users over a computer network, the application under test having a plurality of technology based object oriented software components, the method comprising the steps of:

a) providing test code automatically generated from analysis of the technology based object oriented software component to exercise a selected technology based object oriented software component;

b) creating a first plurality of copies of the test code;

c) simultaneously executing the first plurality of copies of test code remotely located at a first server while recording times between events in each of the first plurality of copies of test code, wherein said test code accesses said technology based object oriented software component over the computer network;

d) creating a second plurality of copies of test code;

e) simultaneously executing the second plurality of copies of test code remotely located at a second server while recording times between events in each of the second plurality of copies of test code;

f) repeating a predetermined number of times the steps of creating plural copies of the test code and simultaneously executing the plural copies while recording event times; and

g) analyzing the recorded times to present information on the performance of the technology based object oriented software component of the application under test as a function of load.

18. The method of claim 17 wherein each object oriented software component has a plurality of functions therein and the test code exercises functions of the object oriented software components.

19. The method of claim 18 wherein the events at which times are recorded includes times at which commands are issued to access functions of the software components and times at which execution of the commands are completed.

20. A system for determining performance of a technology based object oriented software component of an application under test in response to load, the system comprising:

a) coordination software;

b) at least one code generator, receiving as an input commands from the coordination software and having as an output client test code automatically generated from analysis of the technology based object oriented software component;

c) at least one test engine, receiving as an input commands from the coordination software, the test engine comprising a computer server having a plurality of threads thereon, each thread simultaneously executing an instance of the client test code, wherein said client test code is remotely located at a first server and accesses said technology based object oriented software component over a computer network;

d) at least one data log having computerized memory, the memory holding timing data created by the instances of the client test code in the plurality of threads; and

e) at least one data analyzer software, operatively connected to the data log, having an output that represents performance of the technology based object oriented software component of the application under test in response to load.