Method for Request Profiling in Service Systems with Kernel Events

• US 20140109112A1
• Filed: 03/26/2013
• Published: 04/17/2014
• Est. Priority Date: 03/26/2012
• Status: Active Grant

• Current Assignee: NEC Corporation
• Original Assignee: NEC Laboratories America Inc (NEC Corporation)
• Inventors: Zhang, Hui, Jiang, Guofei, Yoshihira, Kenji, Rhee, Jungwhan
• Granted Patent: US 8,875,158 B2
• Time in Patent Office: Days
• US Class Current: 719/318
• CPC Class Codes: G06F 11/3466 Performance evaluation by tracing or monitoring
• CPC Class Codes: G06F 11/3636 by tracing the execution of the program
• CPC Class Codes: G06F 2201/86 Event-based monitoring
• CPC Class Codes: G06F 2201/865 Monitoring of software
• CPC Class Codes: G06F 9/542 Event management; Broadcasting; Multicasting; Notifications

Abstract

A method for request profiling in service systems with kernel events includes collecting kernel events traces from a target system, the kernel event traces being obtainable from individual service machines by instrumenting core kernel functions, analyzing kernel event traces for constructing end-to-end request profiling traces consisting of kernel events belonging to service processes, and categorizing request traces responsive to the analyzing with the constructing including grouping requests based on marking kernel events used in the analyzing.

Claims

1. A method for request profiling in service systems with kernel events, comprising the steps of:

collecting kernel events traces from a target system, said kernel event traces obtainable from individual service machines by instrumenting core kernel functions;

analyzing kernel event traces for constructing end-to-end request profiling traces consisting of kernel events belonging to service processes; and

categorizing request traces responsive to said analyzing step, said constructing including grouping requests based on marking kernel events used in said analyzing step.

3. The method of claim 1, wherein said end-to-end request profiling comprises:

generating request units from collected kernel events based on a configurable marking event set; and

constructing full-path request traces by composing relevant requests units based on a causality relationship set which is between request units and defined on the marking event set and temporal relationships.

4. The method of claim 3, wherein said step of generating request units comprises:

presenting request unit marks as a 4-tuple record;

begin event type, end event type, owner filter and event data filter; and

providing a set of request unit marks used to detect begins and ends of request units in event record sequences, said request unit marks being predefinable as a default set or dynamically configured upon user requests.

5. The of claim 4, wherein said begin event type comprises defining the event type that the first event of a request unit must exactly match, examples of said begin event type, including but not limited to, are network event types, inter-process communication event types, process event type, system call event types, and application event types.

6. The method of claim 4, wherein said end event type comprises defining the event type that the last event of a request unit must exactly match, examples of said end event type, including but not limited to, are network event types, inter-process communication event types, system call event types, and application event types.

7. The method of claim 5, wherein said owner filter comprises defining the owner ID that the first and last events of a request unit must partially or exactly match, examples of said owner filter, including but not limited to, are a specific process name, a specific process ID, or a wild card matching any owner ID.

8. The method of claim 4, wherein said event data filter comprises defining the event data that the first and last events of a request unit must partially or exactly match, examples of said event data filter, including but not limited to, are a network event, inter-process communication event data, and application event data, a wild card in these fields matching any event data and the event data filters can be different for the first and last events.

9. The method of claim 3, wherein said step of generating request units comprises a request unit slicing process where one request unit is defined through a time ordered event sequence data structure that is an event record subsequence of one process, said request unit slicing comprising:

generating transaction units from per-process kernel event traces;

keeping track of open transaction units and complete transaction units, said open transaction units having beginnings detected but not found in endings yet and complete transaction units having both beginnings and endings detected;

scanning one event after another in a process'"'"'s event sequence;

checking the event with event marks on a begin event type and an end event type;

creating a new open transaction unit if its first beginning event is detected; and

closing an open transaction unit as a complete one when its ending event is detected.

10. The method of claim 3, wherein said step of constructing full-path requests comprises defining a causality relationship set including event association rules each presented as a 5 tuple record:

causing event type, caused event type, time rule, owner rule and event data rule.

11. The method of claim 3, wherein said step of constructing full path requests includes a request unit sequencing procedure comprising for each transaction unit, searching for the transaction units causing it, either in other processes due to certain event-causality rules, or in its own process due to time-causality rules;

and after said searching, constructing a full-path transaction for each transaction unit with no causing transaction unit found;

the transaction being rooted at said transaction unit, and expanded over all transaction units caused by that root in one hop or multiple hops.

12. The method of claim 11, wherein said causing event type comprises defining the type of events that can cause the occurrence of other events, said caused event type comprises defining the type of events that are caused by other events, said time rule comprises defining a rule that a causing event type event and a caused event type event can be associated based on their temporal relationships, said owner rule comprises defining a rule that a causing event type event and a caused event type event can be associated based on their owner IDs;

and said event data rule comprises defining a rule that a causing event type event and a caused event type event can be associated based on their event data.

2. The method of claim 2, wherein said service processes include one of being distributed over multiple machines, residing on the same machine, and belonging to same processes such as a series of threads executing a single request.

13. A system for request profiling in service systems with kernel events, comprising:

kernel event collection for collecting kernel events traces from a target system, said kernel event traces obtainable from individual service machines by instrumenting core kernel functions;

end-to-end request profiling for analyzing kernel event traces for constructing end-to-end request profiling traces consisting of kernel events belonging to service processes; and

request categorization for categorizing request traces responsive to said end-to-end request profiling, said constructing including grouping requests based on marking kernel events used in said end-to-end request profiling.

14. The system of claim 13, wherein said end-to-end request profiling comprises:

request unit generation for generating request units from collected kernel events based on a configurable marking event set; and

request unit sequencing for constructing full-path request traces by composing relevant requests units based on a causality relationship set which is between request units and defined on the marking event set and temporal relationships.

15. The system of claim 14, wherein said request unit generation comprises a defined marking event set comprising:

presenting request unit marks as a 4-tuple record;

begin event type, end event type, owner filter and event data filter; and

providing a set of request unit marks used to detect begins and ends of request units in event record sequences, said request unit marks being predefinable as a default set or dynamically configured upon user requests.

16. The system of claim 15, wherein said begin event type comprises defining the event type that the first event of a request unit must exactly match, examples of said begin event type, including but not limited to, are network event types, inter-process communication event types, process event type, system call event types, and application event types .

17. The system of claim 15, wherein said end event type comprises defining the event type that the last event of a request unit must exactly match, examples of said end event type, including but not limited to, are network event types, inter-process communication event types, system call event types, and application event types.

18. The system of claim 15, wherein said owner filter comprises defining the owner ID that the first and last events of a request unit must partially or exactly match, examples of said owner filter, including but not limited to, are a specific process name, a specific process ID, or a wild card matching any owner ID.

19. The system of claim 15, wherein said event data filter comprises defining the event data that the first and last events of a request unit must partially or exactly match, examples of said event data filter, including but not limited to, are a network event, inter-process communication event data, and application event data, a wild card in these fields matching any event data and the event data filters can be different for the first and last events.

20. The system of claim 14, wherein said request unit generation comprises a request unit slicing process where one request unit is defined through a time ordered event sequence data structure that is an event record subsequence of one process, said request unit slicing comprising:

generating transaction units from per-process kernel event traces;

keeping track of open transaction units and complete transaction units, said open transaction units having beginnings detected but not found in endings yet and complete transaction units having both beginnings and endings detected;

scanning one event after another in a process'"'"'s event sequence;

checking the event with event marks on a begin event type and an end event type;

creating a new open transaction unit if its first beginning event is detected; and

closing an open transaction unit as a complete one when its ending event is detected.