Method and system for processing application performance data ouside of monitored applications to limit overhead caused by monitoring

• US 7,957,934 B2
• Filed: 05/15/2008
• Issued: 06/07/2011
• Est. Priority Date: 05/15/2007
• Status: Active Grant

• Current Assignee: Dynatrace LLC (Dynatrace, Inc.)
• Original Assignee: dynaTrace Software GmbH (Dynatrace, Inc.)
• Inventors: Greifeneder, Bernd
• Primary Examiner(s): Lau; Tung S
• Application Number:
  US12/120,645
• Publication Number:
  US 20080288212A1
• Time in Patent Office: 1,118 Days
• Field of Search: 702/182
• US Class Current: 702/182
• CPC Class Codes: G06F 11/3466 Performance evaluation by tracing or monitoring
• CPC Class Codes: G06F 2201/86 Event-based monitoring
• CPC Class Codes: G06F 2201/865 Monitoring of software
• CPC Class Codes: G06F 2201/88 Monitoring involving counting

Abstract

The present invention provides a system and method that aggregates and processes monitoring data acquired inside a monitored application outside of the said application to reduce overhead caused by monitoring tasks in the monitored application. Monitoring data acquired by monitoring sensors is stored in a ring buffer after acquisition, which decouples acquisition of data and its transfer, enabling asynchronous sending of performance data to a monitoring server for analysis. Memory consumption of said ring buffer is limited to a fixed size which keeps memory consumption caused by monitoring processes low and predictable. Correlation, analysis, all computation of monitoring data, and bytecode manipulation to place sensors is performed outside of the monitored application, by a monitoring server.

Claims

1. A method to process application performance data, the method performed by at least one computer processor executing computer-readable instructions tangibly stored on at least one computer-readable medium, the method comprising the steps of:

(A) injecting an agent into a monitored application;

(B) at the agent;

(B)(1) capturing original byte code of the monitored application;

(B) (2) sending the captured byte code of the monitored application to a monitoring server external to the monitored application;

(C) at an instrumentation engine in the monitoring server;

(C) (1) inserting sensor bytecode into the captured original bytecode to generate instrumented byte code at runtime of the application and without altering source code of the application;

(C) (2) transmitting the instrumented byte code to the monitored application;

(D) at the monitored application;

(D) (1) loading the instrumented byte code instead of the original bytecode;

(D) (2) executing the sensor bytecode to generate measurement data which describes performance characteristics of the original bytecode; and

(D) (3) writing the measurement data to a data set of fixed size.

2. The method of claim 1, wherein (B) (1) comprises capturing the original byte code by interrupting the loading of the byte code.

3. The method of claim 1, wherein (C) further comprises:

(C) (3) generating metadata describing the instrumented byte code;

(C) (4) storing said metadata in a sensor metadata node external to the monitored application;

(C) (5) applying a key to the instrumented byte code which identifies the metadata describing the instrumented byte code; and

(C) (6) correlating measurement data generated by a specific portion of instrumented byte code with the metadata describing the specific portion of instrumented byte code external to the monitored application.

4. The method of claim 1, wherein (D) further comprises:

(D) (4) before (D) (1), receiving the instrumented bytecode from the instrumentation engine; and

(D) (5) before (D) (1), forwarding the instrumented bytecode to a bytecode loading process.

5. The method of claim 1:

wherein (D) (2) comprises executing the sensor bytecode to generate the measurement data without blocking execution of the monitored application; and

wherein (D) further comprises;

(D) (4) tagging the measurement data with a unique sensor identifier (ID) of the sensor bytecode without blocking execution of the monitored application; and

wherein (D) (3) comprises storing a first event record in an event buffer in the application, wherein the first event record includes the measurement data and the sensor ID, without blocking execution of the monitored application.

6. The method of claim 1, wherein the data set of fixed size comprises an event buffer, wherein the event buffer is a ring buffer of fixed size, and wherein the method further comprises:

(E) at the agent in the application;

(E) (1) transmitting the first event record to the monitoring server asynchronously with a second event record being written to the event buffer; and

(E) (2) repeating (E) (1) cyclically;

(F) at the monitoring server;

(F) (1) receiving the first event record; and

(F) (2) extracting the unique sensor ID from the first event record.

7. The method of claim 3, wherein the metadata comprises at least one of:

(1) a class name of a class into which the sensor bytecode was inserted;

(2) a method name of a method into which the sensor bytecode was inserted; and

(3) a unique sensor identifier (ID) associated with the inserted sensor bytecode.

8. The method of claim 3, wherein (C) further comprises:

(C) (7) parameterizing the sensor bytecode with a unique sensor identifier (ID) identifying the metadata about the inserted sensor bytecode; and

wherein (C) (1) comprises inserting the parameterized sensor bytecode into the captured original bytecode to generate the instrumented bytecode.

9. The method of claim 6, wherein (E) further comprises:

(E) (3) reading the second event record from the event buffer simultaneously with storing the first event record in the event buffer.

10. The method of claim 6, wherein (F) further comprises:

(F) (3) determining whether a sensor metadata repository contains the unique sensor ID; and

(F) (4) if the sensor metadata repository is determined to include a sensor metadata node having the unique sensor ID, then;

(F) (4) (a) adding a reference to the sensor metadata node to the first event record to produce a first updated event record; and

(F) (4) (b) inserting the first updated event record into a measurement buffer at the monitoring server.

11. The method of claim 9, wherein the event buffer in the application comprises a ring buffer of fixed size.

12. A computer-readable medium comprising computer-readable instructions tangibly stored on the computer-readable medium, wherein the computer-readable instructions are executable by at least one computer processor to cause a monitoring server to perform a method, the method comprising:

(A) injecting an agent into a monitored application;

(B) at the agent;

(B) (1) capturing original byte code of the monitored application;

(B) (2) sending the captured byte code of the monitored application to a monitoring server external to the monitored application;

(C) at an instrumentation engine in the monitoring server;

(C) (1) inserting sensor bytecode into the captured original bytecode to generate instrumented byte code at runtime of the application and without altering source code of the application;

(C) (2) transmitting the instrumented byte code to the monitored application;

(D) at the monitored application;

(D) (1) loading the instrumented byte code instead of the original bytecode;

(D) (2) executing the sensor bytecode to generate measurement data which describes performance characteristics of the original bytecode; and

(D) (3) writing the measurement data to a data set of fixed size.

13. The computer-readable medium of claim 12, wherein (B) (1) comprises capturing the original byte code by interrupting the loading of the byte code.

14. The computer-readable medium of claim 12, wherein (C) further comprises:

(C) (3) generating metadata describing the instrumented byte code;

(C) (4) storing said metadata in a sensor metadata node external to the monitored application;

(C) (5) applying a key to the instrumented byte code which identifies the metadata describing the instrumented byte code; and

(C) (6) correlating measurement data generated by a specific portion of instrumented byte code with the metadata describing the specific portion of instrumented byte code external to the monitored application.

15. The computer-readable medium of claim 12, wherein (D) further comprises:

(D) (4) before (D) (1), receiving the instrumented bytecode from the instrumentation engine; and

(D) (5) before (D)(1), forwarding the instrumented bytecode to a bytecode loading process.

16. The computer-readable medium of claim 12:

wherein (D) (2) comprises executing the sensor bytecode to generate the measurement data without blocking execution of the monitored application; and

wherein (D) further comprises;

(D) (4) tagging the measurement data with a unique sensor identifier (ID) of the sensor bytecode without blocking execution of the monitored application; and

wherein (D) (3) comprises storing a first event record in an event buffer in the application, wherein the first event record includes the measurement data and the sensor ID, without blocking execution of the monitored application.

17. The computer-readable medium of claim 12, wherein the data set of fixed size comprises an event buffer, wherein the event buffer is a ring buffer of fixed size, and wherein the method further comprises:

(E) at the agent in the application;

(E) (1) transmitting the first event record to the monitoring server asynchronously with a second event record being written to the event buffer; and

(E) (2) repeating (E) (1) cyclically;

(F) at the monitoring server;

(F) (1) receiving the first event record; and

(F) (2) extracting the unique sensor ID from the first event record.

18. The computer-readable medium of claim 14, wherein the metadata comprises at least one of:

(1) a class name of a class into which the sensor bytecode was inserted;

(2) a method name of a method into which the sensor bytecode was inserted; and

(3) a unique sensor identifier (ID) associated with the inserted sensor bytecode.

19. The computer-readable medium of claim 14, wherein (C) further comprises:

(C) (7) parameterizing the sensor bytecode with a unique sensor identifier (ID) identifying the metadata about the inserted sensor bytecode; and

wherein (C) (1) comprises inserting the parameterized sensor bytecode into the captured original bytecode to generate the instrumented bytecode.

20. The computer-readable medium of claim 17, wherein (E) further comprises:

(E) (3) reading the second event record from the event buffer simultaneously with storing the first event record in the event buffer.

21. The computer-readable medium of claim 17, wherein (F) further comprises:

(F) (3) determining whether a sensor metadata repository contains the unique sensor ID; and

(F) (4) if the sensor metadata repository is determined to include a sensor metadata node having the unique sensor ID, then;

(F) (4) (a) adding a reference to the sensor metadata node to the first event record to produce a first updated event record; and

(F) (4) (b) inserting the first updated event record into a measurement buffer at the monitoring server.

22. The computer-readable medium of claim 20, wherein the event buffer in the application comprises a ring buffer of fixed size.

23. A computer-implemented method performed by a monitoring server, the method comprising:

(A) receiving original bytecode of an application from an agent, wherein the agent is in the application and external to the monitoring server;

(B) inserting sensor bytecode into the captured original bytecode to generate instrumented bytecode at runtime of the application and without altering source code of the application;

(C) transmitting the instrumented bytecode to the agent;

(D) generating metadata about the context of the inserted sensor bytecode; and

(E) storing the metadata in a sensor metadata node external to the application.

24. The method of claim 23, wherein the context data comprises at least one of:

(1) a class name of a class into which the sensor bytecode was inserted;

(2) a method name of a method into which the sensor bytecode was inserted; and

a unique sensor identifier (ID) associated with the inserted sensor bytecode.

25. The method of claim 23, further comprising:

(F) before (B), parameterizing the sensor bytecode with a unique sensor identifier (ID) identifying the metadata about the inserted sensor bytecode; and

wherein (B) comprises inserting the parameterized sensor bytecode into the captured original bytecode to generate the instrumented bytecode.

26. A computer-readable medium comprising computer-readable instructions tangibly stored on the computer-readable medium, wherein the computer-readable instructions are executable by at least one computer processor to cause a monitoring server to perform a method, the method comprising:

(A) receiving original bytecode of an application from an agent, wherein the agent is in the application and external to the monitoring server;

(B) inserting sensor bytecode into the captured original bytecode to generate instrumented bytecode at runtime of the application and without altering source code of the application;

(C) transmitting the instrumented bytecode to the agent;

(D) generating metadata about the context of the inserted sensor bytecode; and

(E) storing the metadata in a sensor metadata node external to the application.

27. The computer-readable medium of claim 26, wherein the context data comprises at least one of:

(1) a class name of a class into which the sensor bytecode was inserted;

(2) a method name of a method into which the sensor bytecode was inserted; and

a unique sensor identifier (ID) associated with the inserted sensor bytecode.

28. The computer-readable medium of claim 26, further comprising:

(F) parameterizing the sensor bytecode with a unique sensor identifier (ID) identifying the metadata about the inserted sensor bytecode; and

wherein (B) comprises inserting the parameterized sensor bytecode into the captured original bytecode to generate the instrumented bytecode.

29. A computer-implemented method for use with an application and a monitoring server, wherein the monitoring server is external to the application, wherein the application includes instrumented bytecode, and wherein the instrumented bytecode includes original bytecode and sensor bytecode inserted into the original bytecode, the method comprising:

(A) at an application, without blocking execution of the application, executing the sensor bytecode to;

(A) (1) generate measurement data which describes performance characteristics of the original bytecode into which the sensor bytecode is inserted;

(A) (2) tag the measurement data with a unique sensor identifier (ID) of the sensor bytecode;

(A) (3) storing a first event record in an event buffer in the application, wherein the first event record includes the measurement data and the sensor ID;

(B) at an agent in the application;

(B) (1) transmitting the first event record to the monitoring server asynchronously with a second event record being written to the event buffer; and

(B) (2) repeating (B) (1) cyclically;

(C) at the monitoring server;

(C) (1) receiving the first event record; and

(C) (2) extracting the unique sensor ID from the first event record.

30. The method of claim 29, wherein (B) further comprises:

(B) (3) reading the second event record from the event buffer simultaneously with storing the first event record in the event buffer.

31. The method of claim 29, wherein the event buffer in the application is a ring buffer of fixed size.

32. The method of claim 29, wherein (C) further comprises:

(C) (1) determining whether a sensor metadata repository contains the unique sensor ID; and

(C) (2) if the sensor metadata repository is determined to include a sensor metadata node having the unique sensor ID, then;

(C) (4) (a) adding a reference to the sensor metadata node to the first event record to produce a first updated event record;

(C) (4) (b) and inserting the first updated event record into a measurement buffer at the monitoring server.

33. A computer-readable medium comprising computer-readable instructions tangibly stored on the computer-readable medium, wherein the computer-readable instructions are executable by at least one computer processor to perform a method for use with an application and a monitoring server, wherein the monitoring server is external to the application, wherein the application includes instrumented bytecode, and wherein the instrumented bytecode includes original bytecode and sensor bytecode inserted into the original bytecode, the method comprising:

(A) at an application, without blocking execution of the application, executing the sensor bytecode to;

(A) (1) generate measurement data which describes performance characteristics of the original bytecode into which the sensor bytecode is inserted;

(A) (2) tag the measurement data with a unique sensor identifier (ID) of the sensor bytecode;

(A) (3) storing a first event record in an event buffer in the application, wherein the first event record includes the measurement data and the sensor ID;

(B) at an agent in the application;

(B) (1) transmitting the first event record to the monitoring server asynchronously with a second event record being written to the event buffer; and

(B) (2) repeating (B) (1) cyclically;

(C) at the monitoring server;

(C) (3) receiving the first event record; and

(C) (4) extracting the unique sensor ID from the first event record.

34. The computer-readable medium of claim 33, wherein (B) further comprises:

(B) (3) reading the second event record from the event buffer simultaneously with storing the first event record in the event buffer.

35. The computer-readable medium of claim 33, wherein the event buffer in the application is a ring buffer of fixed size.

36. The computer-readable medium of claim 33, wherein (C) further comprises:

(C) (5) determining whether a sensor metadata repository contains the unique sensor ID; and

(C) (6) if the sensor metadata repository is determined to include a sensor metadata node having the unique sensor ID, then;

(C) (4) (c) adding a reference to the sensor metadata node to the first event record to produce a first updated event record;

(C) (4) (d) and inserting the first updated event record into a measurement buffer at the monitoring server.

37. A computer-implemented method performed by an application, wherein the application includes instrumented bytecode, and wherein the instrumented bytecode includes original bytecode and sensor bytecode inserted into the original bytecode, the method comprising:

(A) without blocking execution of the application, executing the sensor bytecode to;

(A) (1) generate measurement data which describes performance characteristics of the original bytecode into which the sensor bytecode is inserted;

(A) (2) tag the measurement data with a unique sensor identifier (ID) of the sensor bytecode;

(A) (3) storing a first event record in a buffer in the application, wherein the first event record includes the measurement data and the sensor ID;

(B) at an agent in the application;

(B) (1) transmitting the first event record to a monitoring server external to the application asynchronously with a second event record being written to the buffer; and

(B) (2) repeating (B) (1) cyclically.

38. The method of claim 37, wherein (B) further comprises:

(B) (3) reading the second event record from the buffer simultaneously with storing the first event record in the buffer.

39. The method of claim 37, wherein the buffer in the application is a ring buffer of fixed size.

40. A computer-readable medium comprising computer-readable instructions tangibly stored on the computer-readable medium, wherein the computer-readable instructions are executable by at least one computer processor to cause an application to perform a method, wherein the application includes instrumented bytecode, and wherein the instrumented bytecode includes original bytecode and sensor bytecode inserted into the original bytecode, the method comprising:

(A) without blocking execution of the application, executing the sensor bytecode to;

(A) (1) generate measurement data which describes performance characteristics of the original bytecode into which the sensor bytecode is inserted;

(A) (2) tag the measurement data with a unique sensor identifier (ID) of the sensor bytecode;

(A) (3) storing a first event record in a buffer in the application, wherein the first event record includes the measurement data and the sensor ID;

(B) at an agent in the application;

(B) (1) transmitting the first event record to a monitoring server external to the application asynchronously with a second event record being written to the buffer; and

(B) (2) repeating (B) (1) cyclically.

41. The computer-readable medium of claim 40, wherein (B) further comprises:

(B) (3) reading the second event record from the buffer simultaneously with storing the first event record in the buffer.

42. The computer-readable medium of claim 40, wherein the buffer in the application is a ring buffer of fixed size.