System and method for generating application-level dependencies in one or more virtual machines

• US 8,881,145 B2
• Filed: 12/15/2011
• Issued: 11/04/2014
• Est. Priority Date: 12/15/2011
• Status: Active Grant

• Current Assignee: Industrial Technology Research Institute
• Original Assignee: Industrial Technology Research Institute
• Inventors: Chiueh, Tzi-Cker, Lin, Hao-Cheng
• Primary Examiner(s): Bullock, Jr., Lewis A
• Assistant Examiner(s): KEPNANG, GILLES R
• Application Number:
  US13/327,407
• Publication Number:
  US 20130159999A1
• Time in Patent Office: 1,055 Days
• Field of Search: 370/389, 370/395.41, 709/224, 709/223, 712/227, 717/127, 717/136, 717/177, 718/1, 718/100, 719/319, 726/23
• US Class Current: 718/1
• CPC Class Codes: G06F 11/3466 Performance evaluation by tracing or monitoring
• CPC Class Codes: G06F 11/3476 Data logging G06F11/14, G06F11/2205 take precedence
• CPC Class Codes: G06F 2201/815 Virtual middleware or OS functionality using virtual machines to implement generic software techniques for error detection or fault masking G06F11/1484
• CPC Class Codes: G06F 2201/865 Monitoring of software
• CPC Class Codes: G06F 9/45504 Abstract machines for programme code execution, e.g. Java virtual machine [JVM], interpreters, emulators
• CPC Class Codes: G06F 9/45533 Hypervisors; Virtual machine monitors
• CPC Class Codes: H04L 41/069 using logs of notifications; Post-processing of notifications
• CPC Class Codes: H04L 43/04 Processing captured monitoring data, e.g. for logfile generation

Abstract

A technique generates application-level dependencies in one or more virtual machines (VMs). An intercepting module in a VM monitor intercepts a guest operation system (OS) at TCP send and close related operations. An introspecting module in the VM monitor performs VM introspection for TCP connection and thread information by checking running thread and processing socket kernel objects. It generates an application trajectory and exports application dependencies from the application trajectory for a selected application.

Claims

1. A system for generating application-level dependencies in one or more virtual machines (VMs), comprising:

an application performance management (APM) server running on a physical machine, wherein said APM server accepts a request and sends a control command to at least one forward daemon, each of said at least one forward daemon transfers the control command to a corresponding VM manager to turn on at least one inspecting status of at least one VM of said one or more VMs, and said APM server further generates an application trajectory of a selected application and exports application-level dependencies from the application trajectory for said selected application;

one or more intercepting modules, wherein each intercepting module intercepts a guest operating system (OS) at transmission control protocol (TCP) send and closely-related operations on the at least one related VM; and

one or more introspecting modules, wherein each introspecting module performs VM introspection for TCP connection and running thread information;

wherein each intercepting module intercepts either by hardware breakpoint at thread-level or by intercepting TCP-related system calls;

wherein a decision to select an intercept mechanism depends on a number of hardware debug registers supported by a central processing unit (CPU) and a number of TCP send and closely-related functions; and

wherein when the number of hardware debug registers supported by a CPU is greater than the number of TCP send and closely-related functions, intercepting by hardware breakpoint is selected as the intercept mechanism.

3. The system as claimed in claim 1, wherein said each intercepting module and said each introspecting module are both in a corresponding VM monitor on a corresponding physical server, and a corresponding forward daemon of the at least one forward daemon is running on said corresponding physical server.

6. The system as claimed in claim 1, wherein each of the one or more per-thread traffic logs contains time information, thread information, connection information and direction.

8. The system as claimed in claim 1, wherein said application trajectory records message exchanges and time periods between applications that starting from one application.

9. The system as claimed in claim 1, wherein said application trajectory is stored in a tree data structure starting from a root node, and comprises one or more nodes, one or more links and one or more arrows, a node in the application trajectory indicates a message exchange to an application during a time period, a link in the application trajectory indicates a direct message exchange between two applications, an arrow in the trajectory indicates the direction of a connection between two applications.

10. The system as claimed in claim 1, wherein said application dependencies include an application dependency map represented in a tree data structure.

11. The system as claimed in claim 1, wherein said APM server uses an incremental enable module to gradually enable a related VM'"'"'s inspect status until the at least one related VM'"'"'s inspect status is turned on.

2. The system as claimed in 1, wherein said each introspecting module performs said VM introspection by checking the guest OS'"'"'s running kernel data structure, getting one or more running threads and TCP connection information, and generating one or more per-thread traffic logs for sending to said APM server via said forward daemon.

4. The system as claimed in claim 2, wherein said APM server further comprises:

a traffic log conversion module for converting the one or more per-thread traffic logs to one or more inter-thread traffic logs; and

a dependency map generating module for generating said application trajectory from the one or more inter-thread traffic logs and exporting the application dependencies from the application trajectory for the selected application.

5. The system as claimed in claim 4, said system uses a log database to store the one or more inter-thread traffic logs.

7. The system as claimed in claim 4, wherein each of the one or more inter-thread traffic logs contains time information, sender thread information, receiver thread information and direction.

12. A computer-implemented method for generating application-level dependencies in one or more virtual machines (VMs), comprising:

under control of one or more physical servers that provide a virtual environment having said one or more VMs, performing the following;

intercepting a guest operating system (OS) at transmission control protocol (TCP) send and closely-related operations of at least one related VM of said one or more VMs;

performing VM introspection for TCP connection and running thread information;

generating an application trajectory of a selected application and exporting application dependencies from the application trajectory for the selected application; and

implementing the intercepting either by hardware breakpoint at thread-level or by intercepting TCP-related system calls;

wherein a decision to select an intercept mechanism depends on a number of hardware debug registers supported by a central processing unit (CPU) and a number of TCP send and closely-related functions; and

wherein when the number of hardware debug registers supported by a CPU is greater than the number of TCP send and closely-related functions, intercepting by hardware breakpoint is selected as the intercept mechanism.

13. The computer-implemented method as claimed in claim 12, further comprising performing said VM introspection by checking the guest OS'"'"'s running threads, getting one or more running threads and TCP connection information, and generating one or more per-thread traffic logs.

14. The computer-implemented method as claimed in claim 13, further comprising converting the one or more per-thread traffic logs to one or more inter-thread traffic logs.

15. The computer-implemented method as claimed in claim 14, further comprising a learning phase, a discovery phase and a generation phase to generate application-level dependencies.

16. The computer-implemented method as claimed in claim 15, further comprising, in the learning phase, using an incremental approach to identify at least one VM related to the selected application, and generating a related vm-id list and providing to the discovery phase.

17. The computer-implemented method as claimed in claim 15, further comprising, in the discovery phase, collecting the one or more per-thread traffic logs, generating the one or more inter-thread traffic logs and providing to the generation phase.

18. The computer-implemented method as claimed in claim 15, further comprising, in the generation phase, generating the application trajectory and generating an application dependency map from the application trajectory by applying an application dependency algorithm.

19. The computer-implemented method as claimed in claim 18, further comprising using a recursive approach by the application dependency algorithm to find out all indirect dependent threads related to the selected application during a time period.

20. The computer-implemented method as claimed in claim 13, wherein each of the one or more per-thread traffic logs comprises time information, thread information, connection information and direction.

21. The computer-implemented method as claimed in claim 14, wherein each of the one or more inter-thread traffic logs comprises time information, sender thread information, receiver thread information and direction.

22. The computer-implemented method as claimed in claim 12, wherein said selected application runs on multi-process models.

23. The computer-implemented method as claimed in claim 12, wherein said selected application runs on multi-thread models.

24. The computer-implemented method as claimed in claim 12, wherein when the number of hardware debug registers supported by a CPU is not greater than the number of TCP send and closely-related functions, intercepting TCP-related system calls is selected, which includes a shadow service descriptor table (SDT) cloned in a VM monitor, and the SDT table is modified to intercept TCP-related system calls.

25. A system for generating application-level dependencies in one or more virtual machines (VMs), comprising:

an application performance management (APM) server running on a physical machine, wherein said APM server accepts a request and sends a control command to at least one forward daemon, each of said at least one forward daemon transfers the control command to a corresponding VM manager to turn on at least one inspecting status of at least one VM of said one or more VMs, and said APM server further generates an application trajectory of a selected application and exports application-level dependencies from the application trajectory for said selected application;

one or more intercepting modules, wherein each intercepting module intercepts a guest operating system (OS) at transmission control protocol (TCP) send and closely-related operations on the at least one related VM; and

one or more introspecting modules, wherein each introspecting module performs VM introspection for TCP connection and running thread information;

wherein each intercepting module intercepts either by hardware breakpoint or by intercepting TCP-related system calls;

wherein a decision to select an intercept mechanism depends on a number of hardware debug registers supported by a central processing unit (CPU) and a number of TCP send and closely-related functions; and

wherein when the number of hardware debug registers supported by a CPU is not greater than the number of TCP send and closely-related functions, intercepting TCP-related system calls is selected as the intercept mechanism, the intercepting TCP-related system calls including cloning a shadow service descriptor table (SDT) in a VM monitor and modifying the SDT to intercept TCP-related system calls.

26. A computer-implemented method for generating application-level dependencies in one or more virtual machines (VMs), comprising:

under control of one or more physical servers that provide a virtual environment having said one or more VMs, performing the following;

intercepting a guest operating system (OS) at transmission control protocol (TCP) send and closely-related operations of at least one related VM of said one or more VMs;

performing VM introspection for TCP connection and running thread information;

generating an application trajectory of a selected application and exporting application dependencies from the application trajectory for the selected application; and

implementing the intercepting either by hardware breakpoint or by intercepting TCP-related system calls;

wherein a decision to select an intercept mechanism depends on a number of hardware debug registers supported by a central processing unit (CPU) and a number of TCP send and close related functions; and

wherein when the number of hardware debug registers supported by a CPU is not greater than the number of TCP send and closely-related functions, intercepting TCP-related system calls is selected as the intercept mechanism, the intercepting TCP-related system calls including cloning a shadow service descriptor table (SDT) in a VM monitor and modifying the SDT to intercept TCP-related system calls.