Systems, methods and computer programs for determining dependencies between logical components in a data processing system or network

• US 8,132,180 B2
• Filed: 04/07/2008
• Issued: 03/06/2012
• Est. Priority Date: 10/20/2003
• Status: Active Grant

• Current Assignee: Daedalus Blue LLC
• Original Assignee: International Business Machines Corporation
• Inventors: Agarwal, Manoj K., Gupta, Manish, Kar, Gautam, Kermani, Parviz, Neogi, Anindya
• Primary Examiner(s): Ho, Andy
• Application Number:
  US12/098,491
• Publication Number:
  US 20080216098A1
• Time in Patent Office: 1,429 Days
• Field of Search: 719/316, 718/100, 718/107, 717/120, 709/223, 709/224
• US Class Current: 719/316
• CPC Class Codes: G06F 11/008 Reliability or availability analysis
• CPC Class Codes: H04L 67/02 based on web technology, e.g. hypertext transfer protocol [HTTP]

Abstract

Described are methods, apparatus and computer programs for determining run-time dependencies between logical components of a data processing environment. Components of the data processing environment are monitored by monitoring agents accessing run-time activity data via APIs of the managed system. A dependency generator identifies correlations between the run-time activity of the monitored components. For synchronous monitored systems, the dependency generator calculates an activity period for monitored components and determines which component'"'"'s activity periods contain the activity periods of other components. Containment is used as an indicator of a likely dependency relationship, and a weighting is computed for each dependency relationship based on the consistency of containment.

Claims

1. A method, implemented in a data processing system, for determining run-time dependencies between logical components of a data processing environment, the method comprising the steps of:

monitoring run-time activity of each of a first logical component and a second logical component of the data processing environment, said monitoring comprising determining a first activity period for said first logical component and a second activity period for said second logical component, wherein said determining comprises determining a first start time and a first end time for said first logical component and a second start time and a second end time for said second logical component;

comparing the monitored run-time activity of the first logical component with the monitored run-time activity of the second logical component to identify correlations between the monitored run-time activity of the first and second logical components, said comparing comprising comparing said first activity period and said second activity period to identify correlations between said first logical component and said second logical component, wherein said comparing comprises determining whether said first start time is before said second start time and whether said first end time is after said second end time; and

in response to identification of a positive correlation between the monitored run-time activity of the first and second logical components, recording the existence of a dependency relationship between the first and second logical components.

2. The method according to claim 1, further comprising the steps of:

monitoring run-time activity data of each of a plurality of logical components additional to said first and second logical components;

comparing the monitored run-time activity of the first logical component with the monitored run-time activity of the plurality of logical components to identify positive correlations between the monitored run-time activity of the first logical component and the monitored run-time activity of any of the plurality of logical components; and

recording the existence of a dependency relationship between the first logical component and any of the plurality of logical components for which a positive correlation is identified.

3. The method according to claim 2, further comprising aggregating the recorded dependency relationships of the first logical component.

4. The method according to claim 2, further comprising:

comparing the monitored run-time activity of the second logical component with the run-time activity of each of the plurality of logical components to identify positive correlations between the monitored run-time activity of the second logical component and the monitored run-time activity of any of the plurality of logical components; and

recording the existence of a dependency relationship between the second logical component and any of the plurality of logical components for which a positive correlation is identified.

5. The method according to claim 1, wherein the comparing step comprises determining whether the activity period of the first logical component contains the activity period of the second logical component.

6. The method according to claim 1, wherein the step of monitoring run-time activity comprises monitoring invocations of the first and second logical components, and the comparing step comprises comparing the number of invocations of each of the first and second logical components within a monitoring period.

7. The method according to claim 1, further comprising:

monitoring run-time activity for a plurality of executions of each of a first and a second logical component,

comparing the monitored run-time activity for the plurality of executions of the second logical component with the monitored run-time activity for the plurality of executions of the first logical component;

determining a proportion of executions of the first logical component for which a positive correlation is identified between the compared run-time activity of the first and second logical components; and

recording in association with the recorded dependency relationship a value representing the determined proportion of executions of the first logical component for which a positive correlation is identified.

8. The method according to claim 1, further comprising:

monitoring run-time activity for a plurality of executions of each of a first and a second logical component,

comparing the monitored run-time activity for the plurality of executions of the second logical component with the monitored run-time activity for the plurality of executions of the first logical component;

determining a proportion of executions of the second logical component for which a positive correlation is identified between the compared run-time activity of the first and second logical components; and

recording in association with the recorded dependency relationship a value representing the determined proportion of executions of the second logical component for which a positive correlation is identified.

9. The method according to claim 7, further comprising:

determining a proportion of executions of the second logical component for which a positive correlation is identified between the compared run-time activity of the first and second logical components; and

recording in association with the recorded dependency relationship a value representing the determined proportion of executions of the second logical component for which a positive correlation is identified; and

generating a weight value comprising a combination function of the determined proportions of executions of the first and second logical components, and storing the weight value in association with the recorded dependency relationship.

10. The method according to claim 1, comprising:

computing a probability value associated with a dependency relationship by determining, for a plurality of executions of the first and second components, the proportion of executions of the first component for which the activity period of the first component contains the activity period of the second component, and recording the probability value in association with the recorded dependency relationship.

11. The method according to claim 1, further comprising:

computing a probability associated with a dependency relationship by determining, for a plurality of executions of the first and second components, the proportion of executions of the second component for which the activity period of the first component contains the activity period of the second component, and recording the probability value in association with the recorded dependency relationship.

12. The method according to claim 1, wherein the events are generated in response to a monitoring agent polling the first and second components for information relating to the processing of requests.

13. The method according to claim 1, wherein the events are generated by a publish/subscribe broker which receives run-time activity data from the first and second logical components, identifies a subscriber by reference to stored subscription information, and sends the generated events to the identified subscriber.

14. The method according to claim 1, for monitoring a data processing system which comprises a monitoring interface for accessing run-time activity data within the data processing system, wherein the step of monitoring run-time activity comprises a monitoring agent accessing run-time activity data via the monitoring interface.

15. The method according to claim 14, wherein the monitoring agent comprises an aggregator for aggregating run-time activity data accessed via the monitoring interface.

16. The method according to claim 14, wherein the monitoring agent comprises program code for computing run-time activity metrics from the run-time activity data accessed via the monitoring interface.

17. The method according to claim 4, wherein the monitoring agent is configured to poll the first and second logical components via the monitoring interface.

18. A method of fault management comprising the steps of:

monitoring run-time activity of each of a first logical component and a plurality of additional logical components of a data processing environment, said monitoring comprising determining a first activity period for said first logical component and additional activity periods for said additional logical components, wherein said determining comprises determining a first start time and a first end time for said first logical component and additional start times and additional end times for said additional logical components;

comparing the monitored run-time activity of the first logical component with the monitored run-time activity of each of the plurality of logical components, to identify positive correlations between the monitored run-time activity of the first logical component and the monitored run-time activity of any of the plurality of logical components, said comparing comprising comparing said first activity period and said additional activity periods to identify correlations between said first logical component and any of said additional logical components, wherein said comparing comprises determining whether said first start time is before said additional start times and whether said first end time is after said additional end times;

recording the existence of a dependency relationship between the first logical component and any of the plurality of logical components for which a positive correlation is identified;

aggregating the recorded dependency relationships to determine a set of logical components having dependency relationships with the first logical component; and

responding to identification of a problem affecting the first logical component by analyzing the set of logical components having dependency relationships with the first logical component.

19. The method according to claim 18, wherein the step of aggregating the recorded dependency relationships comprises sorting the dependencies into an order determined by a sorting heuristic, and the step of analyzing the set of logical components comprises analyzing components of the set of components in said determined order.

20. A method for determining a set of logical components of a data processing environment which are likely to be affected by termination of a first logical component of the data processing environment, the method comprising:

monitoring run-time activity of each of a first logical component and a plurality of additional logical components of a data processing environment, said monitoring comprising determining a first activity period for said first logical component and additional activity periods for said additional logical components, wherein said determining comprises determining a first start time and a first end time for said first logical component and additional start times and additional end times for said additional logical components;

comparing the monitored run-time activity of the first logical component with the monitored run-time activity of each of the plurality of logical components, to identify positive correlations between the monitored run-time activity of the first logical component and the monitored run-time activity of any of the plurality of logical components, said comparing comprising comparing said first activity period and said additional activity periods to identify correlations between said first logical component and any of said additional logical components, wherein said comparing comprises determining whether said first start time is before said additional start times and whether said first end time is after said additional end times; and

recording the existence of a dependency relationship between the first logical component and any of the plurality of logical components for which a positive correlation is identified; and

aggregating the recorded dependency relationships to determine a set of logical components having dependency relationships with the first logical component.

21. A computer program product comprising program code recorded on a recording medium, for controlling the operation of a data processing system on which the program code executes to determine run-time dependencies between logical components of a data processing environment, the program code comprising:

at least one monitoring agent for monitoring run-time activity data of logical components of the data processing environment, said monitoring comprising determining activity periods for said logical components, and for sending the monitored run-time activity data to a correlation identifier;

a correlation identifier for receiving, from the at least one monitor, monitored run-time activity data of each of a first logical component and a second logical component of a data processing environment, wherein said receiving comprises receiving a first start time and a first end time for said first logical component and a second start time and a second end time for said second logical component, and for comparing the monitored run-time activity data of the first logical component with the monitored run-time activity data of the second logical component to identify positive correlations between the monitored run-time activity of the first and second logical components, said comparing comprising comparing said activity periods of said logical components to identify correlations between said first logical component and said second logical component, wherein said comparing comprises determining whether said first start time is before said second start time and whether said first end time is after said second end time; and

a dependency generator for responding to identification of a positive correlation between the monitored run-time activity of the first and second logical components by recording the existence of a dependency relationship between the first and second logical components.

22. A computer program product comprising program code recorded on a recording medium, for controlling the operation of a data processing system on which the program code executes to determine run-time dependencies between logical components of a data processing environment, the program code comprising:

a correlation identifier;

for receiving, from at least one monitoring agent, monitored run-time activity data of each of a first logical component and a second logical component of a data processing environment, said receiving comprising receiving a first activity period for said first logical component and a second activity period for said second logical component, wherein said receiving comprises receiving a first start time and a first end time for said first logical component and a second start time and a second end time for said second logical component, and

for comparing the monitored run-time activity data of the first logical component with the monitored run-time activity data of the second logical component to identify positive correlations between the monitored run-time activity of the first and second logical components, said comparing comprising comparing said first activity period and said second activity period to identify correlations between said first logical component and said second logical component, wherein said comparing comprises determining whether said first start time is before said second start time and whether said first end time is after said second end time; and

a dependency generator for responding to identification of a positive correlation between the monitored run-time activity of the first and second logical components by generating a representation of the existence of a dependency relationship between the first and second logical components.

23. A data processing apparatus comprising:

a data processing unit;

a data storage unit;

a correlation identifier

for receiving, from at least one monitoring agent, monitored run-time activity data of each of a first logical component and a second logical component of a data processing environment, said receiving comprising receiving a first activity period for said first logical component and a second activity period for said second logical component, wherein said receiving comprises receiving a first start time and a first end time for said first logical component and a second start time and a second end time for said second logical component, and

for comparing the monitored run-time activity data of the first logical component with the monitored run-time activity data of the second logical component to identify positive correlations between the monitored run-time activity of the first and second logical components, said comparing comprising comparing said first activity period and said second activity period to identify correlations between said first logical component and said second logical component, wherein said comparing comprises determining whether said first start time is before said second start time and whether said first end time is after said second end time; and

a dependency generator for responding to identification of a positive correlation between the monitored run-time activity of the first and second logical components by recording in the data storage unit the existence of a dependency relationship between the first and second logical components.

24. The data processing apparatus according to claim 23, wherein the dependency generator is configured to generate a CIM_dependency class instance to represent an identified positive correlation, the apparatus further comprising a CIM object manager configured to record dependency information within a CIM repository in the data storage unit.

25. A distributed data processing system comprising:

a first data processing apparatus comprising a set of logical components to be monitored, and at least one monitoring agent for monitoring run-time activity data for the set of logical components and for sending the monitored run-time activity data to a correlation identifier on a second data processing apparatus, said monitoring comprising determining activity periods for said logical components; and

a second data processing apparatus comprising;

a data processing unit;

a data storage unit;

a correlation identifier

for receiving, from the at least one monitoring agent, monitored run-time activity data of each of a first logical component and a second logical component of a data processing environment, said receiving comprising receiving a first activity period for said first logical component and a second activity period for said second logical component, wherein said receiving comprises receiving a first start time and a first end time for said first logical component and a second start time and a second end time for said second logical component, and

for comparing the monitored run-time activity data of the first logical component with the monitored run-time activity data of the second logical component to identify positive correlations between the monitored run-time activity of the first and second logical components, said comparing comprising comparing said first activity period and said second activity period to identify correlations between said first logical component and said second logical component, wherein said comparing comprises determining whether said first start time is before said second start time and whether said first end time is after said second end time; and

a dependency generator for responding to identification of a positive correlation between the monitored run-time activity of the first and second logical components by recording in the data storage unit the existence of a dependency relationship between the first and second logical components.

26. The distributed data processing system according to claim 25, further comprising at least one management application program for analyzing recorded dependency relationships.

27. The distributed data processing system according to claim 25, wherein the at least one management application program comprises a fault management application program.

28. A method for discovering dependencies between monitored components of a managed data processing system, comprising the steps of:

accessing, from the managed system, run-time activity data for the monitored components, said accessing comprising accessing activity periods for said monitored components, wherein said accessing comprises accessing a first start time and a first end time for a first logical component of said monitored components and a second start time and a second end time for a second logical component of said monitored components;

comparing the accessed run-time activity data of the monitored components to identify positive correlations between the run-time activity of the monitored components, said comparing comprising comparing said activity periods to identify correlations between said monitored components, wherein said comparing comprises determining whether said first start time is before said second start time and whether said first end time is after said second end time; and

generating an identification of a dependency relationship between components for which a positive correlation is identified.

29. The method according to claim 28, wherein the step of accessing comprises accessing the run-time activity data via an API provided by the managed system.

30. The method according to claim 28, wherein the step of comparing the accessed run-time activity data comprises comparing a plurality of run-time activity metrics for each monitored component.

31. The method according to claim 30, further comprising:

computing a value representing the consistency of identification of a positive correlation between components for the plurality of run-time activity metrics; and

storing the computed value in association with the generated identification of a dependency relationship.

32. The method according to claim 30, wherein the step of comparing the accessed run-time activity data comprises comparing run-time activity data for a plurality of executions of the monitored components, and wherein the method further comprises:

computing a single value representing the consistency of identification of a positive correlation between components, for the plurality of executions of the components and the plurality of run-time activity metrics; and

storing the computed single value in association with the generated identification of a dependency relationship.

33. The method according to claim 28, herein the step of comparing the accessed run-time activity data comprises comparing run-time activity data for a plurality of executions of the monitored components, and wherein the method further comprises:

computing a value representing the consistency of identification of a positive correlation between components for the plurality of executions of the components; and

storing the computed value in association with the generated identification of a dependency relationship.