Method And System For Real-time Modeling Of Communication, Virtualization And Transaction Execution Related Topological Aspects Of Monitored Software Applications And Hardware Entities

• US 20160105350A1
• Filed: 10/09/2015
• Published: 04/14/2016
• Est. Priority Date: 10/10/2014
• Status: Active Grant

• Current Assignee: Dynatrace LLC (Dynatrace, Inc.)
• Original Assignee: Dynatrace LLC (Dynatrace, Inc.)
• Inventors: Greifeneder, Bernd, AMBICHL, Ernst, SPIEGL, Helmut, LEHOFER, Andreas, SCHWARZBAUER, Gunther, Mlotowski, Rafat
• Granted Patent: US 11,159,599 B2
• Time in Patent Office: Days
• US Class Current: 1/1
• CPC Class Codes: H04L 41/046 comprising network management agents or mobile agents therefor
• CPC Class Codes: H04L 41/12 Discovery or management of network topologies
• CPC Class Codes: H04L 41/122 of virtualised topologies, e.g. software-defined networks [SDN] or network function virtualisation [NFV]
• CPC Class Codes: H04L 41/5058 Service discovery by the service manager
• CPC Class Codes: H04L 67/02 based on web technology, e.g. hypertext transfer protocol [HTTP]
• CPC Class Codes: H04L 67/1001 for accessing one among a plurality of replicated servers
• CPC Class Codes: H04L 67/62 Establishing a time schedule for servicing the requests

Abstract

A system and method for real-time discovery and monitoring of multidimensional topology models describing structural aspects of applications and of computing infrastructure used to execute those applications is disclosed. Different types of agents are deployed to the monitored application execution infrastructure dedicated to capture specific topological aspects of the monitored system. Virtualization agents detect and monitor the virtualization structure of virtualized hardware used in the execution infrastructure, operating system agents deployed to individual operating systems monitor resource utilization, performance and communication of processes executed by the operating system and transaction agents deployed to processes participating in the execution of transactions, providing end-to-end transaction trace and monitoring data describing individual transaction executions. The monitoring and tracing data of the deployed agents contains correlation data that allows to create a topology model of the monitored system that integrates transaction execution, process execution and communication and virtualization related aspects.

Claims

1. A computer-implemented method for monitoring a distributed transaction by a monitoring system across a distributed computing environment, comprising:

receiving, by a transaction agent, transaction trace data that identifies a transaction being executed in part on a host computing device, where the transaction trace data includes an identifier for a thread executed by a process;

receiving, by the transaction agent, service data that identifies a service used by the transaction to enter the process, where the service is executed within the process and the transaction agent is instrumented in the process;

determining, by the transaction agent, identifying information for a process group to which the process belongs and identifying information for an operating system executing on the host computing device and upon which the process is running;

combining, by the transaction agent, the transaction trace data and the service data with the identifying information for the process group and the operating system to form trace and topology correlation data; and

sending, by the transaction agent, the trace and topology correlation data over a network to a monitoring node located remotely from the host computing device.

2. The computer-implemented method for claim 1 further comprises receiving the transaction trace data and the service data from a service sensor instrumented in a method, where the method is configured to receive a request from another process.

3. The computer-implemented method of claim 2 further comprises receiving a service request for the service and parsing the service request to determine the service data, where the service request includes a uniform resource locator (URL).

4. The computer-implemented method of claim 3 wherein the service data includes an identifier for a server computer and an identifier for a protocol for accessing the server computer.

5. The computer-implemented method of claim 1 further comprises

determining, by an OS agent, metadata for the process and metadata for the operating system, where the OS agent is running on the host computing device;

sending, by the OS agent, an OS topology event over the network to the monitoring node, where the OS topology event includes the metadata for the process and the operating system; and

correlating, by a topology processor, the topology correlation data with the metadata for the process and the operating system, where the topology processor resides on the monitoring node.

6. The computer-implemented method of claim 3 further comprises determining, by a topology processor, a vertical relationship between a method servicing a service request and the service from the service data and creating a record in a topology repository for the vertical relationship, where the topology processor resides on the monitoring node.

7. The computer-implemented method of claim 6 wherein the service is further defined as a remote method invocation (RMI) service, and the method is further defined as a specific remote method which is provided by the RMI service for remote execution.

8. The computer-implemented method of claim 3 further comprises determining, by a topology processor, a vertical relationship between the service used by the transaction to enter the process and the process group to which the process belongs.

9. The computer-implemented method of claim 3 further comprises determining, by a topology processor, a horizontal relationship between the service servicing the service request and another service called by the service from the service data and creating a record in a topology repository for the horizontal relationship, where the topology processor resides on the monitoring node.

10. The computer-implemented method of claim 1 wherein the transaction agent interacts with an OS agent to determine the identifying information for a process group to which the process belongs and identifying information for an operating system upon which the process is running, where the OS agent is running on the host computing device.

11. The computer-implemented method of claim 10 further comprises extracting, by the OS agent, identifying information for the process group from at least one of process metadata or from a command which initiates the process.

12. The computer-implemented method of claim 1 wherein processes having the similar functions are mapped to the same process group.

13. The computer-implemented method of claim 1 further comprises determining, by an OS agent, metadata for a given process and metadata for the operating system, where the given process is running on the host computing device and the OS agent is running on the host computing device;

detecting, by the OS agent, a communication with another process running on a second operating system on a second host device, where the given process is participating in the communication; and

sending, by the OS agent, a communication topology event over the network to the monitoring node, where the communication topology event includes identifying information for a process group to which the given process belongs, identifying information for the operating system, a network address and port identifier for host computing device and a network address and port identifier for the second host device.

14. The computer-implemented method of claim 13 further comprises

receiving, by a topology processor, the communication topology event from the host computing device, where the topology processor resides on the monitoring node;

receiving, by the topology processor, a second communication topology event from the second host device, where the second communication topology event includes identifying information for a second process group to which the another process belongs, identifying information for the second operating system, the network address and port identifier for the host computing device and the network address and port identifier for the second host device;

determining, by the topology processor, a horizontal relationship between the process group of the given process and the second process group of the another process using the communication topology event and the second communication topology event; and

creating a record for the horizontal relationship in a topology repository.

15. The computer-implemented method of claim 13 further comprises

sending, by the OS agent, an OS topology event over the network to the monitoring node, where the OS topology event includes metadata for the given process and the operating system and where the metadata for the given process includes identifying information for a process group to which the given process belongs;

determining, by a topology processor, a vertical relationship between the operating system and the process group to which the given process belongs using the OS topology event; and

creating a record for the vertical relationship in a topology repository.

16. The computer-implemented method of claim 1 further comprises

detecting, by a virtualization agent, other host computing devices implemented as virtualized computer systems;

generating, by the virtualization agent, a virtualization topology event for the detected virtualized computer systems, where the virtualization topology event includes an identifier for the virtualized computer system and an identifier for a network interface assigned to the virtualized computer system;

sending, by the virtualization agent, the virtualization topology event over a network to a monitoring node located remotely from the host device;

determining, by an OS agent, identifying information for an operating system running on a particular one of the other host computing devices, wherein the OS agent is running on the operating system;

generating, by the OS agent, an OS topology event, where the OS topology event includes an identifier for the operating system and the identifier for a network interface assigned to the host computing device;

sending, by the OS agent, the OS topology event over the network to the monitoring node; and

correlating, by a topology processor, the particular one of the other host computing devices with the virtualized computer system using the identifier for the network interface of the virtualized computer system and the identifier for the network interface assigned to the host computing device, where the topology processor resides on the monitoring node.

17. A computer-implemented system for monitoring a distributed transaction across a distributed computing environment, comprising:

a transaction agent instrumented in a process executing on a host computing device and executed by a processor of the host computing device, the transaction agent is configured to receive transaction trace data that identifies a transaction being executed in part on the host computing device and service data that identifies a service used by the transaction to enter the process, where the transaction trace data includes an identifier for a thread executing the transaction; and

an OS agent running on the host computing device and executed by the processor of the host computing device, the OS agent determines identifying information for a process group to which the process belongs and identifying information for an operating system executing on the host computing device and upon which the process is running and communicates the identifying information for the process group and identifying information for the operating system to the transaction agent;

wherein the transaction agent combines the transaction trace data and the service data with the identifying information for the process group and the operating system to form trace and topology correlation data, and sends the trace and topology correlation data over a network to a monitoring node located remotely from the host computing device.

18. The computer-implemented system of claim 17 further comprises a service sensor instrumented in a method associated with the service, the service sensor determines the transaction trace data and the service data and sends the transaction trace data and the service data to the transaction agent, wherein the service data includes an identifier for a server computer and an identifier for a protocol for accessing the server computer.

19. The computer-implemented system of claim 18 wherein the OS agent determines metadata for the process and metadata for the operating system and sends an OS topology event over the network to the monitoring node, where the OS topology event includes the metadata for the process, metadata for the operating system and the identifying information for a process group to which the process belongs.

20. The computer-implemented system of claim 19 further comprises a topology processor residing on the monitoring node and configured to receive the trace and topology correlation data, the topology processor determines from trace and topology correlation data a vertical relationship between a method servicing a service request and the service and creates a record in a topology repository for the vertical relationship.

21. The computer-implemented system of claim 19 further comprises a topology processor residing on the monitoring mode and configured to receive the trace and topology correlation data, the topology processor determines from the trace and topology correlation data a vertical relationship between the service used by the transaction to enter the process and the process group to which the process belongs.

22. The computer-implemented system of claim 19 further comprises a topology processor residing on the monitoring mode and configured to receive the trace and topology correlation data, the topology processor determines from the trace and topology correlation data a horizontal relationship between the service and another service called by the service and creating a record in a topology repository for the horizontal relationship.

23. The computer-implemented system of claim 19 wherein the OS agent determines metadata for the process and metadata for the operating system, detects a communication with another process running on a second operating system on a second host device;

and sends a communication topology event over the network to the monitoring node, where the communication topology event includes identifying information for the process group, identifying information for the operating system, a network address and port identifier for host computing device and a network address and port identifier for the second host device.

24. The computer-implemented system of claim 23 further comprises a topology processor residing on the monitoring mode and configured to receive the communication topology event from the host computing device and a second communication topology event from the second host device, where the second communication topology event includes identifying information for a process group to which the another process belongs, identifying information for the second operating system, the network address and port identifier for the host computing device and the network address and port identifier for the second host device, wherein the topology processor determines a horizontal relationship between the process group of the process and the second process group of the another process system using the communication topology event and the second communication topology event;

and creates a record for the horizontal relationship in a topology repository.

25. The computer-implemented system of claim 23 further comprises a virtualization agent, the virtualization agent detects host computing devices implemented as virtualized computer systems, generates a virtualization topology event for the detected virtualized computer system, and sends the virtualization topology event over a network to the monitoring node, where the virtualization topology event includes an identifier for the virtualized computer system and an identifier for a network interface assigned to the virtualized computer system.

26. The computer-implemented system of claim 25 wherein the OS agent determines identifying information for an operating system running on the host computing device, generates an OS topology event and sends the OS topology event over the network to the monitoring node, where the OS topology event includes an identifier for the operating system and the identifier for a network interface assigned to the host computing device;

and a topology processor receives the OS topology event and identifies a virtualized computer system for the host computing device using the identifier for the network interface, where the topology processor resides on the monitoring mode.

27. A computer-implemented system for monitoring a distributed transaction across a distributed computing environment, comprising:

a virtualization agent deployed on a given host computing device and executed by the processor of the given host computing device, the virtualization agent detects other host computing devices implemented as virtualized computer systems, generates a virtualization topology event for the detected virtualized computer systems, and sends the virtualization topology event over a network to a monitoring node, where the virtualization topology event includes an identifier for the virtualized computer systems and an identifier for a network interface assigned to the virtualized computer systems;

an OS agent running on one of the other host computing devices and executed by the processor of the second host computing device, the OS agent determines identifying information for an operating system running on the one other host computing device, generates an OS topology event and sends the OS topology event over the network to the monitoring node, where the OS topology event includes an identifier for the operating system and the identifier for a network interface assigned to the one other host computing device; and

a topology processor residing of the monitoring node, the topology processor configured to receive the virtualization topology event and the OS topology event and correlate the operating system with a virtualized computer system using the identifier for the network interface.

28. The computer-implemented system of claim 27 wherein the virtualization agent determines virtualization infrastructure entities involved in creation of virtualized computer systems, generates data identifying and describing the detected virtualization infrastructure entities and appending the data to the virtualization topology event.

29. A computer-implemented method for building an integrated topology model for a distributed computing environment, comprising:

identifying one or more operating systems to be monitored in the computing environment and installing an OS agent on each of the identified operating systems;

monitoring, by the installed OS agents, processes executed by the identified operating systems;

determining, by the installed OS agents, topology data describing the monitored processes;

reporting, by the installed OS agents, the topology data describing the monitored processes across a network to a monitoring node;

installing a transaction agent into select processes executed by the identified operating systems;

monitoring, by the installed transaction agents, transactions executed at least in part by the monitored processes;

extracting, by the installed transaction agents, topology data describing services that are used by the monitored transactions;

reporting the topology data describing services across a network to the monitoring node; and

combining the topology data describing the monitored processes with the topology data describing services that are used by the monitored transactions to form a topological model.

30. The computer-implemented method of claim 29 further comprises

detecting, by a virtualization agent, a virtualized computer system in the computing environment;

determining topology data describing the virtualized computer system; and

reporting the topology data describing the virtualized computer system across the network to the monitoring node.