System and method for automated performance testing in a dynamic production environment

• US 8,756,586 B2
• Filed: 12/10/2010
• Issued: 06/17/2014
• Est. Priority Date: 12/10/2009
• Status: Active Grant

• Current Assignee: TATA Consultancy Services Limited (Tata Sons Pvt Ltd.)
• Original Assignee: TATA Consultancy Services Limited (Tata Sons Pvt Ltd.)
• Inventors: Khanapurkar, Amol, Parab, Omkar, Malan, Suresh
• Primary Examiner(s): Vu, Tuan A
• Application Number:
  US12/964,759
• Publication Number:
  US 20110145795A1
• Time in Patent Office: 1,285 Days
• Field of Search: 717/126, 717/101, 717/103, 717/106, 717/130, 714/38.1, 714/46, 714/47.2, 709/223, 709/230, 709/224, 715/736, 705/2, 705/7.39, 702/186, 703/13, 707/999.002
• US Class Current: 717/130
• CPC Class Codes: G06F 11/3409 for performance assessment
• CPC Class Codes: G06F 11/3414 Workload generation, e.g. scripts, playback
• CPC Class Codes: G06F 11/3419 by assessing time
• CPC Class Codes: G06F 11/3433 for load management allocation of a server based on load conditions G06F9/505; load rebalancing G06F9/5083; redistributing the load in a network by a load balancer H04L67/1029
• CPC Class Codes: G06F 11/3466 Performance evaluation by tracing or monitoring
• CPC Class Codes: G06F 2201/805 Real-time

Abstract

The present invention relates to a system and method for automated performance testing in a dynamic production environment. More particularly the present invention relates to an automated system that enables the determination of whether a software application can meet the scalability and performance requirements in a real time use environment, wherein the performance requirement test comprises test for workload, reliability and resource usage. The system of the present invention provides an automated system for performing testing method that can convert a static environment i.e. fixed workload, one application state and one hardware representation into a dynamic one and predict accurate performance results for said constantly changing application. The system enables less effort, time and consequently money, and meets the performance scalability and requirements in real-time.

Claims

1. A system having a scalable and pluggable framework for automated performance testing of an enterprise application, the system comprising:

an agent coupled to a framework;

a plurality of interconnected components communicatively coupled to the agent, the interconnected components comprising a plurality of load generators, an emulator, and a profiler; and

a server with a processor that executes instructions stored in memory to perform operations, the server communicatively coupled to a repository, a scheduler, a system modeling tool, a system resource utilization monitor, a run-time duration module, a real-time test monitor, and a reporting engine;

wherein the a plurality of load generators record and enhance a test script for running the test script on the enterprise application, wherein the test script is enhanced by providing wrappers around the test script;

wherein the operations comprise detecting a network packet bottleneck based upon the test script run on the enterprise application;

wherein the operations further comprise measuring a response time and utilization statistics of the enterprise application;

wherein the real-time test monitor monitors network activity based upon the network packet bottleneck, the response time, and the utilization statistics;

wherein the system modeling tool extrapolates network activity data from a static test environment to a dynamic production environment;

wherein the load generator, the emulator, and the profiler validate the network activity data; and

wherein the reporting engine generates a performance output parameter based upon the network activity data.

2. The system of claim 1, wherein the emulator is selected from a group consisting of a WAN emulator and a database production emulator.

3. The system of claim 2, wherein the WAN emulator captures network characteristics, the network characteristics selected from a group consisting of bandwidth, latency, packet loss, jitter and connection losses.

4. The system of claim 2, wherein the database production emulator is integrated within a control level and a data layer in the framework.

5. The system of claim 1, wherein the profiler is selected from a group consisting of a Java profiler and a transaction profiler.

6. The system of claim 5, wherein the transaction profiler analyzes network packets, and identifies network packet bottlenecks.

7. The system of claim 5, wherein the Java profiler collects data at a with overheads of 2 microseconds to 5 microseconds.

8. The system of claim 1, wherein the agent includes a plurality of adapters, wherein each adapter operates on a different server.

9. The system of claim 1, wherein the repository stores inventory and mapping information.

10. The system of claim 1, wherein the reporting outputs data related to a response time, a throughput, a runtime duration, or a memory usage.

11. A method for enabling automated performance testing of an enterprise application by employing a system having a scalable and pluggable framework, the method comprising the steps of:

recording and enhancing a test script using a plurality of load generators for running the test script on the enterprise application, wherein the enhancing comprises providing wrappers around the test script;

extrapolating network activity data from a static test environment to a dynamic production environment using a system modeling tool;

detecting a network packet bottleneck based upon the test script run on the enterprise application;

measuring a response time and utilization statistics of the enterprise application;

monitoring network activity based upon the network packet bottleneck, the response time, and the utilization statistics;

validating network activity data using at least one of a load generator, an emulator, and a profiler;

generating a performance output parameter using a reporting engine based upon the network activity data;

retrieving the performance output parameter from the reporting engine using an agent; and

transmitting the performance output parameter from the agent to a server via the network.

12. The method of claim 11, wherein the load generator is integrated into the framework using a wrapper.

13. The method of claim 11, wherein the emulator is selected from a group consisting of a WAN emulator and a database production emulator.

14. The method of claim 13, further comprising the step of enforcing network characteristics using the WAN emulator.

15. The method of claim 14, wherein the network activity is a function of enforced network characteristics.

16. The method of claim 13, further comprising the step of predicting a query performance for a future volume using the database production emulator.