Performance monitoring in distributed systems using synchronized clocks and distributed event logs

• US 6,751,573 B1
• Filed: 01/10/2000
• Issued: 06/15/2004
• Est. Priority Date: 01/10/2000
• Status: Expired due to Fees

• Current Assignee: Keysight Technologies, Inc.
• Original Assignee: Agilent Technologies, Inc.
• Inventors: Burch, Jefferson B.
• Primary Examiner(s): Bui, Bryan
• Application Number:
  US09/480,041
• Time in Patent Office: 1,618 Days
• Field of Search: 702/178, 702/182-183, 702/187-188, 375/130, 375/222, 713/400, 713/502, 713/503, 700/14-16, 370/473, 370/516, 370/478, 370/479, 370/498
• US Class Current: 702/178
• CPC Class Codes: G06F 11/0709 in a distributed system consisting of a plurality of standalone computer nodes, e.g. clusters, client-server systems
• CPC Class Codes: G06F 11/0766 Error or fault reporting or storing
• CPC Class Codes: G06F 11/3404 for parallel or distributed programming
• CPC Class Codes: G06F 11/3476 Data logging G06F11/14, G06F11/2205 take precedence
• CPC Class Codes: G06F 11/3495 for systems

Abstract

A distributed system that provides performance monitoring capability across multiple nodes of a distributed application. A distributed system according to the present techniques includes a set of nodes that communicate via a network. A distributed application is performed by a set of cooperating node applications executing in the nodes. The distributed system implements techniques for generating time-stamp records for each of a set of significant events associated with one or more of the node applications. The time-stamp records provides a synchronized time base across the nodes for the significant events thereby enabling temporal ordering of the significant events.

Claims

1. A method of performance monitoring in a distributed system, comprising the steps of:

determining a set of significant events that are to be monitored in a distributed application in the distributed system;

providing each of a set of nodes applications associated with the significant events with a recorder function which when called generates a record that identifies the corresponding significant event and includes a time-stamp for the corresponding significant event obtained from a corresponding synchronized clock;

running an experiment in the distributed application that generates one or more of the significant events;

obtaining the records from the node applications and analyzing the records.

2. The method of claim 1, wherein each record includes an event code associated with the corresponding significant event.

3. The method of claim 1, wherein the step of analyzing the records comprises the step of generating a graphical representation of the records.

4. The method of claim 1, further comprising the step of determining a set of delays in execution of the node applications associated with the generation of the records.

5. The method of claim 4, further comprising the step of correcting the records in response to the delays.

6. A distributed application, comprising:

a set of node applications which communicate via a network, each node application having at least one function which is associated with a significant event in the distributed application and further having a recorder function which when called by the function obtains a time-stamp for the significant event from a corresponding synchronized clock;

event log for each node application, each event log for holding a record of the corresponding significant event and time-stamp such that the time-stamps provide a synchronized time base in the distributed application for evaluating the significant events.

7. The distributed application of claim 6, wherein each function generates an event code which identifies the corresponding significant event.

8. The distributed application of claim 7, wherein each recorder function obtains the event code from the corresponding function and writes the event code into the corresponding event log along with the corresponding time-stamp.

9. The distributed application of claim 6, wherein one or more of the node applications executes on a node that includes the corresponding synchronized clock.

10. The distributed application of claim 6, wherein one or more of the node applications executes on a node that is connected to a companion node that includes the corresponding synchronized clock.

11. The distributed application of claim 6, further comprising means for obtaining the records from the event logs via the network and analyzing the records using the synchronized time base.

12. The distributed application of claim 6, further comprising means for starting and stopping the recording of records in one or more of the event logs.

13. The distributed application of claim 6, wherein one of the significant events is a generation of an HTTP command in the distributed application.

14. The distributed application of claim 6, wherein one of the significant events is a transmission of a message on the network.

15. The distributed application of claim 6, wherein one of the significant events is a receipt of a message on the network.

16. The distributed application of claim 6, wherein one of the significant events is a database access.

17. The distributed application of claim 6, wherein one of the significant events is a generation of a control value in the distributed application.

18. The distributed application of claim 6, wherein one of the significant events is a receipt of sensor data.

19. The distributed application of claim 6, wherein one of the significant events is an location of a control value to an actuator.