Software behavior pattern recognition and analysis

• US 20040163079A1
• Filed: 02/13/2003
• Published: 08/19/2004
• Est. Priority Date: 02/13/2003
• Status: Active Grant

• Current Assignee: PATH Reliability, Inc.
• Original Assignee: Path Communications Incorporated
• Inventors: Noy, Oded, McBride, Jason
• Granted Patent: US 7,269,824 B2
• Time in Patent Office: Days
• US Class Current: 717/154
• CPC Class Codes: G06F 11/3409 for performance assessment
• CPC Class Codes: G06F 11/3419 by assessing time
• CPC Class Codes: G06F 11/3452 Performance evaluation by statistical analysis
• CPC Class Codes: G06F 11/3495 for systems
• CPC Class Codes: G06F 2201/81 Threshold
• CPC Class Codes: G06F 2201/865 Monitoring of software

Abstract

A target software behavior pattern recognition and analysis tool is disclosed that operates during the real-time execution of monitored target software on a computer, derives a subset of measurements indicating the target software performance and activity, automatically derives a baseline from the subset of measurements, and establishes a state of health of the target software based on comparing real-time measurements to the baseline. The monitored target software contains an agent for deriving a subset of measurements indicating the target software performance and activity. The agent includes sensors embedded into the target software to measure specific code segments, and an examiner which receives measurements from the sensors and transmits these examiner values to a central server. The central server automatically derives a baseline from the received examiner values, and automatically establishes a state of health of the target software based on comparing real-time measurements to the baseline.

Claims

1. A target software behavior pattern recognition and analysis method for monitored target software being executed on a computer, the monitored target software including one or more monitored code segments, the method comprising:

automatically deriving a subset of measurements indicating performance and activity of the monitored target software;

automatically deriving a baseline from the subset of measurements; and

establishing a state of health of the monitored target software based on comparing real-time measurements of the monitored target software to the baseline.

2. The method as recited in claim 1, the step of automatically deriving a subset of measurements further comprising:

determining resource utilization for each of the one or more monitored code segments; and

adjusting a number of measurements taken for each of the one or more monitored code segments based on its resource utilization.

3. The method as recited in claim 2, wherein the resource utilization for each of the one or more monitored code segments is determined by:

collecting call rate and execution time data for the monitored code segment;

computing an average call rate and an average execution time for the monitored code segment; and

multiplying the average call rate by the average execution time.

4. The method as recited in claim 3, further comprising adjusting the number of measurements taken for each of the one or more monitored code segments by activating or deactivating the collecting of call rate or execution time data for the monitored code segment.

5. The method as recited in claim 3, further comprising adjusting the number of measurements taken for each of the one or more monitored code segments based on a comparison of the average call rate or average execution time of the monitored code segment to specified thresholds.

6. The method as recited in claim 3, further comprising adjusting the number of measurements taken for each of the one or more monitored code segments by periodically sampling the call rate or execution time data for the monitored code segment.

7. The method as recited in claim 1, further comprising adjusting the number of measurements taken for each of the one or more monitored code segments such that an overhead penalty associated with the subset of measurements is less than a user selectable amount.

8. The method as recited in claim 3, wherein for object-oriented monitored target software, the resource utilization for each of the one or more monitored code segments includes memory utilization data for a selected set of objects.

9. The method as recited in claim 3 wherein, during a data collection time period, the step of automatically deriving a baseline from the subset of measurements further comprises:

computing an average throughput examiner code segment value by computing a first sum of the average execution times of all code segments for which execution time data was measured during the data collection time period, and dividing the first sum by a first number of code segments for which execution time data was measured during the data collection time period;

computing an average responsiveness examiner code segment value by computing a second sum of the average call rate of all code segments for which call rate data was measured during the data collection time period, and dividing the second sum by a second number of code segments for which call rate data was measured during the data collection time period;

computing an average resource utilization examiner code segment value by multiplying the average throughput examiner value by the average responsiveness examiner value;

for a throughput examiner value vector, which comprises the average throughput examiner code segment value computed during the data collection time period,

updating a first database used to create a current throughput examiner code segment value baseline by adding the average throughput examiner code segment value to the first database, and

computing an updated throughput examiner code segment value baseline from the updated first database;

for a responsiveness examiner value vector, which comprises the average responsiveness examiner code segment value computed during the data collection time period,

updating a second database used to create a current responsiveness examiner code segment value baseline by adding the average responsiveness examiner code segment value to the second database, and

computing an updated responsiveness examiner code segment value baseline from the updated second database; and

for a resource utilization examiner value vector, which comprises the average resource utilization examiner code segment value computed during the data collection time period,

updating a third database used to create a current resource utilization examiner code segment value baseline by adding the average resource utilization examiner code segment value to the third database, and

computing an updated resource utilization examiner code segment value baseline from the updated third database.

10. The method as recited in claim 9, further comprising maintaining multiple sets of databases and baselines for each examiner value, each set corresponding to different time periods longer than the data collection time period.

11. The method as recited in claim 9, the step of establishing a state of health of the monitored target software based on comparing real-time measurements of the monitored target software to the baseline further comprising computing a normalized examiner value for each average examiner code segment value by computing (aiAvg−

Ea)/Ea, where Ea is the average examiner code segment value and aiAvg is the baseline for that average examiner code segment value.

12. The method as recited in claim 9 wherein, during the data collection time period, the step of automatically deriving a baseline from the subset of measurements further comprises:

creating a deviant list of code segments having an execution time that exceeds a particular threshold, the code segments in the deviant list being excluded from the computation of the average responsiveness examiner value; and

including the deviant list as part of the examiner value vector.

13. The method as recited in claim 11, further comprising displaying one examiner value gauge representing the normalized examiner value for each of the one or more average examiner values.

14. The method as recited in claim 13, the examiner value gauges including multi-part indicators for displaying information from the central server.

15. The method as recited in claim 13, further comprising:

computing a normalized instance examiner value by averaging the normalized examiner value for all of the one or more average examiner values; and

displaying one instance examiner value gauge representing the normalized instance examiner value.

16. The method as recited in claim 11, further comprising generating an alert when the normalized examiner value exceeds a predetermined limit.

17. In a system for performing target software behavior pattern recognition and analysis on monitored target software being executed on a computer, the monitored target software including one or more monitored code segments, a method for automatically deriving a subset of measurements indicating performance and activity of the monitored target software for use in deriving a baseline from the subset of measurements and establishing a state of health of the monitored target software based on comparing real-time measurements of the monitored target software to the baseline, the method comprising:

determining resource utilization for each of the one or more monitored code segments; and

adjusting a number of measurements taken for each of the one or more monitored code segments based on its resource utilization;

wherein the resource utilization for each of the one or more monitored code segments is determined by

collecting call rate and execution time data for the monitored code segment,

computing an average call rate and an average execution time for the monitored code segment, and

multiplying the average call rate by the average execution time.

18. The method as recited in claim 17, further comprising adjusting the number of measurements taken for each of the one or more monitored code segments by activating or deactivating the collection of call rate or execution time data for the monitored code segment.

19. The method as recited in claim 17, further comprising adjusting the number of measurements taken for each of the one or more monitored code segments based on a comparison of the average call rate or average execution time of the monitored code segment to specified thresholds.

20. The method as recited in claim 17, further comprising adjusting the number of measurements taken for each of the one or more monitored code segments by periodically sampling the call rate or execution time data for the monitored code segment.

21. The method as recited in claim 17, further comprising adjusting the number of measurements taken for each of the one or more monitored code segments such that an overhead penalty associated with the subset of measurements is less than a user selectable amount.

22. The method as recited in claim 17, wherein for object-oriented monitored target software, the resource utilization for each of the one or more monitored code segments includes memory utilization data for a selected set of objects.

23. In a system for performing target software behavior pattern recognition and analysis on monitored target software being executed on a computer, the monitored target software including one or more monitored code segments capable of providing a subset of measurements during a data collection time period indicating performance and activity of the monitored target software, the subset of measurements determinable by collecting call rate and execution time data for each monitored code segment, computing an average call rate and an average execution time for each monitored code segment, multiplying the average call rate by the average execution time to generate a resource utilization for each monitored code segment, and adjusting a number of measurements taken for each monitored code segment based on its resource utilization, a method for automatically deriving a baseline from the subset of measurements during the data collection time period for use in establishing a state of health of the monitored target software by comparing real-time measurements of the monitored target software to the baseline, the method comprising:

computing an average throughput examiner code segment value by computing a first sum of the average execution times of all code segments for which execution time data was measured during the data collection time period, and dividing the first sum by a first number of code segments for which execution time data was measured during the data collection time period;

computing an average responsiveness examiner code segment value by computing a second sum of the average call rate of all code segments for which call rate data was measured during the data collection time period, and dividing the second sum by a second number of code segments for which call rate data was measured during the data collection time period;

computing an average resource utilization examiner code segment value by multiplying the average throughput examiner value by the average responsiveness examiner value;

for a throughput examiner value vector, which comprises the average throughput examiner code segment value computed during the data collection time period,

updating a first database used to create a current throughput examiner code segment value baseline by adding the average throughput examiner code segment value to the first database, and

computing an updated throughput examiner code segment value baseline from the updated first database;

for a responsiveness examiner value vector, which comprises the average responsiveness examiner code segment value computed during the data collection time period,

updating a second database used to create a current responsiveness examiner code segment value baseline by adding the average responsiveness examiner code segment value to the second database, and

computing an updated responsiveness examiner code segment value baseline from the updated second database; and

for a resource utilization examiner value vector, which comprises the average resource utilization examiner code segment value computed during the data collection time period,

updating a third database used to create a current resource utilization examiner code segment value baseline by adding the average resource utilization examiner code segment value to the third database, and

computing an updated resource utilization examiner code segment value baseline from the updated third database.

24. The method as recited in claim 23, further comprising maintaining multiple sets of databases and baselines for each examiner value, each set corresponding to different time periods longer than the. data collection time period.

25. The method as recited in claim 23, further comprising establishing a state of health of the monitored target software by comparing real-time measurements of the monitored target software to the baseline.

26. The method as recited in claim 25, the step of comparing real-time measurements of the monitored target software to the baseline comprising computing a normalized examiner value for each average examiner code segment value by computing (aiAvg−

Ea)/Ea, where Ea is the average examiner value and aiAvg is the baseline for that average examiner value.

27. The method as recited in claim 23, further comprising:

creating a deviant list of code segments having an execution time that exceeds a particular threshold, the code segments in the deviant list being excluded from the computation of the average responsiveness examiner value; and

including the deviant list as part of the examiner value vector.

28. The method as recited in claim 27, further comprising displaying one examiner value gauge representing the normalized examiner value for each of the one or more average examiner values.

29. The method as recited in claim 28, the examiner value gauges including multi-part indicators for displaying information from the central server.

30. The method as recited in claim 28, further comprising:

computing a normalized instance examiner value by averaging the normalized examiner value for all of the one or more average examiner values; and

displaying one instance examiner value gauge representing the normalized instance examiner value.

31. The method as recited in claim 26, further comprising generating an alert when the normalized examiner value exceeds a predetermined limit.

32. An article of manufacture comprising:

a computer usable medium having computer readable program code means embodied therein for target software behavior pattern recognition and analysis of monitored target software being executed on a computer, the monitored target software including one or more monitored code segments, the computer readable program code means in the article of manufacture comprising;

computer readable program code means for causing a computer to automatically derive a subset of measurements indicating performance and activity of the monitored target software;

computer readable program code means for causing the computer to automatically derive a baseline from the subset of measurements; and

computer readable program code means for causing the computer to establish a state of health of the monitored target software based on comparing real-time measurements of the monitored target software to the baseline.

33. The article of manufacture as recited in claim 32, the computer readable program code means for causing a computer to automatically derive a subset of measurements further comprising:

computer readable program code means for causing the computer to determine resource utilization for each of the one or more monitored code segments; and

computer readable program code means for causing the computer to adjust a number of measurements taken for each of the one or more monitored code segments based on its resource utilization.

34. The article of manufacture as recited in claim 33, further comprising:

computer readable program code means for causing the computer to compute the resource utilization for each of the one or more monitored code segments by;

collecting call rate and execution time data for the monitored code segment;

computing an average call rate and an average execution time for the monitored code segment; and

multiplying the average call rate by the average execution time.

35. The article of manufacture as recited in claim 34, further comprising computer readable program code means for causing the computer to adjust the number of measurements taken for each of the one or more monitored code segments by activating or deactivating the collecting of call rate or execution time data for the monitored code segment.

36. The article of manufacture as recited in claim 34, further comprising computer readable program code means for causing the computer to adjust the number of measurements taken for each of the one or more monitored code segments based on a comparison of the average call rate or average execution time of the monitored code segment to specified thresholds.

37. The article of manufacture as recited in claim 34, further comprising computer readable program code means for causing the computer to adjust the number of measurements taken for each of the one or more monitored code segments by periodically sampling the call rate or execution time data for the monitored code segment.

38. The article of manufacture as recited in claim 32, further comprising computer readable program code means for causing the computer to adjust the number of measurements taken for each of the one or more monitored code segments such that an overhead penalty associated with the subset of measurements is less than a user selectable amount.

39. The article of manufacture as recited in claim 34, wherein for object-oriented monitored target software, the computer readable program code means for causing the computer to compute the resource utilization for each of the one or more monitored code segments further comprises:

computer readable program code means for causing the computer to include memory utilization data for a selected set of objects in the resource utilization for each of the one or more monitored code segments.

40. The article of manufacture as recited in claim 34 wherein, during a data collection time period, the computer readable program code means for causing the computer to automatically derive a baseline from the subset of measurements further comprises:

computer readable program code means for causing the computer to compute an average throughput examiner code segment value by computing a first sum of the average execution times of all code segments for which execution time data was measured during the data collection time period, and dividing the first sum by a first number of code segments for which execution time data was measured during the data collection time period;

computer readable program code means for causing the computer to compute an average responsiveness examiner code segment value by computing a second sum of the average call rate of all code segments for which call rate data was measured during the data collection time period, and dividing the second sum by a second number of code segments for which call rate data was measured during the data collection time period;

computer readable program code means for causing the computer to compute an average resource utilization examiner code segment value by multiplying the average throughput examiner value by the average responsiveness examiner value;

for a throughput examiner value vector, which comprises the average throughput examiner code segment value computed during the data collection time period, computer readable program code means for causing the computer to

update a first database used to create a current throughput examiner code segment value baseline by adding the average throughput examiner code segment value to the first database, and

compute an updated throughput examiner code segment value baseline from the updated first database;

for a responsiveness examiner value vector, which comprises the average responsiveness examiner code segment value computed during the data collection time period, computer readable program code means for causing the computer to

update a second database used to create a current responsiveness examiner code segment value baseline by adding the average responsiveness examiner code segment value to the second database, and

compute an updated responsiveness examiner code segment value baseline from the updated second database; and

for a resource utilization examiner value vector, which comprises the average resource utilization examiner code segment value computed during the data collection time period, computer readable program code means for causing the computer to

updating a third database used to create a current resource utilization examiner code segment value baseline by adding the average resource utilization examiner code segment value to the third database, and

computing an updated resource utilization examiner code segment value baseline from the updated third database.

41. The article of manufacture as recited in claim 40, further comprising computer readable program code means for causing the computer to maintain multiple sets of databases and baselines for each examiner value, each set corresponding to different time periods longer than the data collection time period.

42. The article of manufacture as recited in claim 40, the computer readable program code means for causing the computer to establish a state of health of the monitored target software based on comparing real-time measurements of the monitored target software to the baseline further comprising:

computer readable program code means for causing the computer to compute a normalized examiner value for each average examiner code segment value by computing (aiAvg−

Ea)/Ea, where Ea is the average examiner code segment value and aiAvg is the baseline for that average examiner code segment value.

43. The article of manufacture as recited in claim 40 wherein, during the data collection time period, the computer readable program code means for causing the computer to automatically derive a baseline from the subset of measurements further comprises:

computer readable program code means for causing the computer to create a deviant list of code segments having an execution time that exceeds a particular threshold, the code segments in the deviant list being excluded from the computation of the average responsiveness examiner value; and

computer readable program code means for causing the computer to include the deviant list as part of the examiner value vector.

44. The article of manufacture as recited in claim 42, further comprising computer readable program code means for causing the computer to display one examiner value gauge representing the normalized examiner value for each of the one or more average examiner values.

45. The article of manufacture as recited in claim 44, further comprising computer readable program code means for causing the computer to display examiner value gauges including multi-part indicators for displaying information from the central server.

46. The article of manufacture as recited in claim 44, further comprising:

computer readable program code means for causing the computer to compute a normalized instance examiner value by averaging the normalized examiner value for all of the one or more average examiner values; and

computer readable program code means for causing the computer to display one instance examiner value gauge representing the normalized instance examiner value.

47. The article of manufacture as recited in claim 42, further comprising computer readable program code means for causing the computer to generate an alert when the normalized examiner value exceeds a predetermined limit.

48. In a system for performing target software behavior pattern recognition and analysis on monitored target software being executed on a computer, the monitored target software including one or more monitored code segments, an article of manufacture comprising:

a computer usable medium having computer readable program code means embodied therein for automatically deriving a subset of measurements indicating performance and activity of the monitored target software for use in deriving a baseline from the subset of measurements and establishing a state of health of the monitored target software based on comparing real-time measurements of the monitored target software to the baseline, comprising;

computer readable program code means for causing a computer to determine resource utilization for each of the one or more monitored code segments;

computer readable program code means for causing the computer to adjust a number of measurements taken for each of the one or more monitored code segments based on its resource utilization; and

computer readable program code means for causing the computer to determine the resource utilization for each of the one or more monitored code segments by

collecting call rate and execution time data for the monitored code segment,

computing an average call rate and an average execution time for the monitored code segment, and

multiplying the average call rate by the average execution time.

49. The article of manufacture as recited in claim 48, further comprising computer readable program code means for causing the computer to adjust the number of measurements taken for each of the one or more monitored code segments by activating or deactivating the collection of call rate or execution time data for the monitored code segment.

50. The article of manufacture as recited in claim 48, further comprising computer readable program code means for causing the computer to adjust the number of measurements taken for each of the one or more monitored code segments based on a comparison of the average call rate or average execution time of the monitored code segment to specified thresholds.

51. The article of manufacture as recited in claim 48, further comprising computer readable program code means for causing the computer to adjust the number of measurements taken for each of the one or more monitored code segments by periodically sampling the call rate or execution time data for the monitored code segment.

52. The article of manufacture as recited in claim 48, further comprising computer readable program code means for causing the computer to adjust the number of measurements taken for each of the one or more monitored code segments such that an overhead penalty associated with the subset of measurements is less than a user selectable amount.

53. The article of manufacture as recited in claim 48, wherein for object-oriented monitored target software, the computer readable program code means for causing the computer to determine the resource utilization for each of the one or more monitored code segments further comprises:

computer readable program code means for causing the computer to include memory utilization data for a selected set of objects in the resource utilization for each of the one or more monitored code segments.

54. In a system for performing target software behavior pattern recognition and analysis on monitored target software being executed on a computer, the monitored target software including one or more monitored code segments capable of providing a subset of measurements during a data collection time period indicating performance and activity of the monitored target software, the subset of measurements determinable by collecting call rate and execution time data for each monitored code segment, computing an average call rate and an average execution time for each monitored code segment, multiplying the average call rate by the average execution time to generate a resource utilization for each monitored code segment, and adjusting a number of measurements taken for each monitored code segment based on its resource utilization, an article of manufacture comprising:

a computer usable medium having computer readable program code means embodied therein for automatically deriving a baseline from the subset of measurements during the data collection time period for use in establishing a state of health of the monitored target software by comparing real-time measurements of the monitored target software to the baseline, comprising;

computer readable program code means for causing a computer to compute an average throughput examiner code segment value by computing a first sum of the average execution times of all code segments for which execution time data was measured during the data collection time period, and dividing the first sum by a first number of code segments for which execution time data was measured during the data collection time period;

computer readable program code means for causing the computer to compute an average responsiveness examiner code segment value by computing a second sum of the average call rate of all code segments for which call rate data was measured during the data collection time period, and dividing the second sum by a second number of code segments for which call rate data was measured during the data collection time period;

computer readable program code means for causing the computer to compute an average resource utilization examiner code segment value by multiplying the average throughput examiner value by the average responsiveness examiner value;

for a throughput examiner value vector, which comprises the average throughput examiner code segment value computed during the data collection time period, computer readable program code means for causing the computer to

update a first database used to create a current throughput examiner code segment value baseline by adding the average throughput examiner code segment value to the first database, and

compute an updated throughput examiner code segment value baseline from the updated first database;

for a responsiveness examiner value vector, which comprises the average responsiveness examiner code segment value computed during the data collection time period, computer readable program code means for causing the computer to

update a second database used to create a current responsiveness examiner code segment value baseline by adding the average responsiveness examiner code segment value to the second database, and

compute an updated responsiveness examiner code segment value baseline from the updated second database; and

for a resource utilization examiner value vector, which comprises the average resource utilization examiner code segment value computed during the data collection time period, computer readable program code means for causing the computer to

updating a third database used to create a current resource utilization examiner code segment value baseline by adding the average resource utilization examiner code segment value to the third database, and

computing an updated resource utilization examiner code segment value baseline from the updated third database.

55. The article of manufacture as recited in claim 54, further comprising computer readable program code means for causing the computer to maintain multiple sets of databases and baselines for each examiner value, each set corresponding to different time periods longer than the data collection time period.

56. The article of manufacture as recited in claim 54, further comprising computer readable program code means for causing the computer to establish a state of health of the monitored target software by comparing real-time measurements of the monitored target software to the baseline.

57. The article of manufacture as recited in claim 56, the computer readable program code means for causing the computer to compare real-time measurements of the monitored target software to the baseline further comprising:

computer readable program code means for causing the computer to compute a normalized examiner value for each average examiner code segment value by computing (aiAvg−

Ea)/Ea, where Ea is the average examiner value and aiAvg is the baseline for that average examiner value.

58. The article of manufacture as recited in claim 54, further comprising:

computer readable program code means for causing the computer to create a deviant list of code segments having an execution time that exceeds a particular threshold, the code segments in the deviant list being excluded from the computation of the average responsiveness examiner value; and

computer readable program code means for causing the computer to include the deviant list as part of the examiner value vector.

59. The article of manufacture as recited in claim 58, further comprising computer readable program code means for causing the computer to display examiner value gauges representing the normalized examiner value for each of the one or more average examiner values.

60. The article of manufacture as recited in claim 59, further comprising computer readable program code means for causing the computer to display examiner value gauges including multi-part indicators for displaying information from the central server.

61. The article of manufacture as recited in claim 59, further comprising computer readable program code means for causing the computer to:

compute a normalized instance examiner value by averaging the normalized examiner value for all of the one or more average examiner values; and

display one instance examiner value gauge representing the normalized instance examiner value.

62. The article of manufacture as recited in claim 57, further comprising computer readable program code means for causing the computer to generate an alert when the normalized examiner value exceeds a predetermined limit.