Identification of root cause for a transaction response time problem in a distributed environment

• US 7,487,407 B2
• Filed: 07/12/2005
• Issued: 02/03/2009
• Est. Priority Date: 07/12/2005
• Status: Expired due to Fees

• Current Assignee: International Business Machines Corporation
• Original Assignee: International Business Machines Corporation
• Inventors: Gehman, Byron Christian, Tipton, Sandra Lee
• Primary Examiner(s): Lohn; Joshua A
• Application Number:
  US11/179,241
• Publication Number:
  US 20070016831A1
• Time in Patent Office: 1,302 Days
• Field of Search: 714/26, 714/37, 714/43, 714/51, 714/57
• US Class Current: 714/51
• CPC Class Codes: G06F 11/0724 in a multiprocessor or a multi-core unit multiprocessors per se G06F15/80
• CPC Class Codes: G06F 11/079 Root cause analysis, i.e. error or fault diagnosis in a hardware test environment G06F11/22; in a software test environment G06F11/36
• CPC Class Codes: G06F 11/3419 by assessing time
• CPC Class Codes: G06F 11/3495 for systems
• CPC Class Codes: G06F 2201/87 Monitoring of transactions

Abstract

Method and apparatus for identifying a cause for a response time problem for a transaction in a distributed computing system that includes a central server and a plurality of subsystems. Data is stored at each subsystem relating to sub-transactions of transactions performed by the subsystems. When a problem is discovered in connection with the completion of a particular transaction, each subsystem of the plurality of subsystems that was involved in the particular transaction is identified, and both instance data relating to all of the sub-transactions of the particular transaction stored at each identified subsystem and current hourly aggregate data stored at each identified subsystem is forwarded to the central server. Root-Cause Analysis is then performed using the forwarded instance data and aggregate data to identify the particular subsystem that caused the transaction problem.

Claims

1. A method for identifying a cause for a response time problem for a transaction in a distributed computing system that includes a central server and a plurality of subsystems, where a single transaction is divided into a plurality of sub-transactions and each sub-transaction is sent to a given subsystem of the plurality of subsystems, the method comprising:

storing data, by an agent at each subsystem, relating to the sub-transactions performed by the subsystems, wherein the data is stored at the each subsystem in a queue that contains transaction instance data for multiple different transactions that have been processed by the each subsystem, wherein the transaction instance data is specific data generated for a given sub-transaction processed by the subsystem;

discovering a problem in connection with completion of a particular transaction;

identifying the each subsystem of the plurality of subsystems involved in the particular transaction using at least one identifier stored at the each subsystem, wherein the at least one identifier includes a first identifier uniquely identifying the particular transaction, and a second identifier identifying a policy associated with the particular transaction;

forwarding the data stored at each identified subsystem to the central server, the forwarded data including instance data relating to the sub-transaction of the particular transaction performed by the identified subsystems and aggregate data relating to sub-transactions of transactions performed by the identified subsystems; and

performing a Root-Cause Analysis using the forwarded aggregate data and the forwarded instance data to identify the subsystem that caused the response time problem with the particular transaction.

2. The method according to claim 1, wherein the each subsystem notifies the central server of the second identifier contained in an initial correlator for a given sub-transaction that is received by the each subsystem.

3. The method according to claim 1, wherein the storing step comprises:

storing data at the each subsystem relating to sub-transactions of transactions performed by the subsystems in a circular doubly linked in-memory queue.

4. The method according to claim 3, and further comprising:

specifying a size of the queue at the each subsystem as a function of transaction traffic.

5. The method according to claim 1, wherein the instance data comprises First Failure Data Capture data.

6. The method according to claim 1, wherein the aggregate data comprises current hourly aggregate data.

7. A computer program product, comprising computer executable instructions embodied in a computer usable, recordable-type medium, for identifying a cause for a response time problem for a transaction in a distributed computing system that includes a central server and a plurality of subsystems, where a single transaction is divided into a plurality of sub-transactions and each sub-transaction is sent to a given subsystem of the plurality of subsystems, the computer program product comprising:

first instructions for storing data, by an agent at each subsystem, relating to the sub-transactions performed by the subsystems, wherein the data is stored at the each subsystem in a queue that contains transaction instance data for multiple different transactions that have been processed by the each subsystem, wherein the transaction instance data is specific data generated for a given sub-transaction processed by the subsystem;

second instructions for discovering a problem in connection with completion of a particular transaction;

third instructions for identifying the each subsystem of the plurality of subsystems involved in the particular transaction using at least one identifier stored at the each subsystem, wherein the at least one identifier includes a first identifier uniquely identifying the particular transaction, and a second identifier identifying a policy associated with the particular transaction;

fourth instructions for forwarding the data stored at each identified subsystem to the central server, the forwarded data including instance data relating to the sub-transaction of the particular transaction performed by the identified subsystems and aggregate data relating to sub-transactions of transactions performed by the identified subsystems; and

fifth instructions for performing a Root-Cause Analysis using the forwarded aggregate data and the forwarded instance data to identify the subsystem that caused the response time problem with the particular transaction.

8. The computer program product according to claim 7, wherein the each subsystem notifies the central server of the second identifier contained in an initial correlator for a given sub-transaction that is received by the each subsystem.

9. The computer program product according to claim 7, wherein the first instructions comprises:

instructions for storing data at the each subsystem relating to sub-transactions of transactions performed by the subsystems in a circular doubly linked queue.

10. The computer program product according to 9, and further comprising:

sixth instructions for specifying a size of the queue at the each subsystem as a function of transaction traffic.

11. An apparatus for identifying a cause for a response time problem for a transaction in a distributed computing system that includes a central server and a plurality of subsystems, where a single transaction is divided into a plurality of sub-transactions and each sub-transaction is sent to a given subsystem of the plurality of subsystems, the apparatus comprising:

an agent at each subsystem for storing data relating to the sub-transactions performed by the subsystems, wherein the data is stored at the each subsystem in a queue that contains transaction instance data for multiple different transactions that have been processed by the subsystem, wherein the transaction instance data is specific data generated for a given sub-transaction processed by the subsystem;

a mechanism for discovering a problem in connection with completion of a particular transaction;

an identifying mechanism for identifying the each subsystem of the plurality of subsystems involved in the particular transaction using at least one identifier stored at the each subsystem, wherein the at least one identifier includes a first identifier uniquely identifying the particular transaction, and a second identifier identifying a policy associated with the particular transaction;

a forwarding mechanism for forwarding the data stored at each identified subsystem to the central server, the forwarded data including instance data relating to the sub-transaction of the particular transaction performed by the identified subsystems and aggregate data relating to sub-transactions of transactions performed by the identified subsystems; and

an analyzer at the central server for performing a Root-Cause Analysis using the forwarded aggregate data and the forwarded instance data to identify the subsystem that caused the response time problem with the particular transaction.

12. The apparatus according to claim 11, wherein the each subsystem notifies the central server of the second identifier contained in an initial correlator for a given sub-transaction that is received by the each subsystem.

13. The apparatus according to claim 11, wherein the agent at the each subsystem includes a circular doubly linked in-memory queue for storing the data relating to sub-transactions of transactions performed by the subsystems.

14. The apparatus according to claim 11, wherein the instance data comprises First Failure Data Capture data.

15. The apparatus according to claim 11, wherein the aggregate data comprises current hourly aggregate data.