Software behavior pattern recognition and analysis

• US 7,269,824 B2
• Filed: 02/13/2003
• Issued: 09/11/2007
• Est. Priority Date: 02/13/2003
• Status: Expired due to Fees

• Current Assignee: PATH Reliability, Inc.
• Original Assignee: PATH Reliability, Inc.
• Inventors: Noy, Oded, McBride, Jason
• Primary Examiner(s): Zhen; Wei
• Assistant Examiner(s): DENG, ANNA CHEN
• Application Number:
  US10/366,010
• Publication Number:
  US 20040163079A1
• Time in Patent Office: 1,671 Days
• Field of Search: 717/130, 717/131, 717/103, 717/108, 717/109, 717/120, 717/124, 717/127, 717/128, 702176-183, 702/188, 702/141, 705/10, 709/224, 714 38- 40, 714 47- 48
• US Class Current: 717/127
• CPC Class Codes: G06F 11/3409 for performance assessment
• CPC Class Codes: G06F 11/3419 by assessing time
• CPC Class Codes: G06F 11/3452 Performance evaluation by statistical analysis
• CPC Class Codes: G06F 11/3495 for systems
• CPC Class Codes: G06F 2201/81 Threshold
• CPC Class Codes: G06F 2201/865 Monitoring of software

Abstract

A target software behavior pattern recognition and analysis tool is disclosed that operates during the real-time execution of monitored target software on a computer, derives a subset of measurements indicating the target software performance and activity, automatically derives a baseline from the subset of measurements, and establishes a state of health of the target software based on comparing real-time measurements to the baseline. The monitored target software contains an agent for deriving a subset of measurements indicating the target software performance and activity. The agent includes sensors embedded into the target software to measure specific code segments, and an examiner which receives measurements from the sensors and transmits these examiner values to a central server. The central server automatically derives a baseline from the received examiner values, and automatically establishes a state of health of the target software based on comparing real-time measurements to the baseline.

Claims

1. A target software behavior pattern recognition and analysis method for monitored target software being executed on a computer, the method comprising:

automatically defining one or more monitored code segments in the monitored target software, during run time of the monitored target software, wherein the monitored code segments are defined by sensors that are selectively embedded during run time of the monitored target software, and/or selectively turned on or off during run time of the monitored target software;

automatically deriving a subset of measurements from the monitored code segments indicating performance and activity of the monitored target software;

automatically deriving in real time a baseline from the subset of measurements, by computing an examiner value indicative of an aggregate of degree of change; and

establishing in real time a state of health of the monitored target software based on comparing real-time measurements of the monitored target software to the baseline.

2. The method as recited in claim 1, the step of automatically deriving a subset of measurements further comprising:

determining resource utilization for each of the one or more monitored code segments; and

adjusting a number of measurements taken for each of the one or more monitored code segments based on its resource utilization.

3. The method as recited in claim 2, wherein the resource utilization for each of the one or more monitored code segments is determined by:

collecting call rate and execution time data for the monitored code segment;

computing an average call rate and an avenge execution time for the monitored code segment; and

multiplying the average call rate by the average execution time.

4. The method as recited in claim 3, further comprising adjusting the number of measurements taken for each of the one or more monitored code segments by activating or deactivating the collecting of call rate or execution time data for the monitored code segment.

5. The method as recited in claim 3, further comprising adjusting the number of measurements taken for each of the one or more monitored code segments based on a comparison of the average call rate or average execution time of the monitored code segment to specified thresholds.

6. The method as recited in claim 3, further comprising adjusting the number of measurements taken for each of the one or more monitored code segments by periodically sampling the call rate or execution time data for the monitored code segment.

7. The method as recited in claim 1, further comprising adjusting the number of measurements taken for each of the one or more monitored code segments such that an overhead penalty associated with the subset of measurements is less than a user selectable amount.

8. The method as recited, in claim 3, wherein for object-oriented monitored target software, the resource utilization for each of the one or more monitored code segments includes memory utilization data for a selected set of objects.

9. The method as recited in claim 3 wherein, during a data collection time period, tho stop of automatically deriving a baseline from the subset of measurements further comprises:

computing an average throughput examiner code segment value by computing a first sum of the average execution times of all code segments for which execution time data was measured during the data collection time period, and dividing the first sum by a first number of code segments fur which execution time data was measured during the data collection time period;

computing an average responsiveness examiner code segment value by computing a second sum of the average call rate of all code segments for which call rate data was measured during the data collection time period, and dividing the second sum by a second number of code segments for which call rate data was measured during the data collection time period;

computing an average resource utilization examiner code segment value by multiplying the average throughput examiner value by the average responsiveness examiner value;

for a throughput examiner value vector, which comprises the average throughput examiner code segment value computed during the data collection time period, updating a first database used to create a current throughput examiner code segment value baseline by adding the average throughput examiner code segment value to the first database, and computing an updated throughput examiner code segment value baseline from the updated first database;

for a responsiveness examiner value vector, which comprises the average responsiveness examiner code segment value computed during the data collection time period, updating a second database used to create a current responsiveness examiner code segment value baseline by adding the average responsiveness examiner code segment value to the second database, and computing an updated responsiveness examiner code segment value baseline from the updated second database; and

for a resource utilization examiner value vector, which comprises the average resource utilization examiner code segment value computed during the data collection time period, updating a third database used to create a current resource utilization examiner code segment value baseline by adding the average resource utilization examiner code segment value to the third database, and cumputing an updated resource utilization examiner code segment value baseline from the updated third database.

10. The method as recited in claim 9, further comprising maintaining multiple sets of databases and baselines for each examiner value, each set corresponding to different time periods longer than the data collection time period.

11. The method as recited in claim 9, the step of establishing a state of health of the monitored target software based on comparing real-time measurements of the monitored target software to the baseline further comprising computing a normalized examiner value for each average examiner code segment value by computing (aiAvg−

Ea)/Ea, where Ea is the average examiner code segment value and aiAvg is the baseline for that average examiner code segment value.

12. The method as recited in claim 9 wherein, during the data collection time period, the step of automatically deriving a baseline from the subset of measurements further comprises:

creating a deviant list of code segments having an execution time that exceeds a particular threshold, the code segments in the deviant list being excluded from the computation of the average responsiveness examiner value; and

including the deviant list as part or the examiner value vector.

13. The method as recited in claim 11, further comprising displaying one examiner value gauge representing die normalized examiner value for each of the one or more average examiner values.

14. The method as recited in claim 13, the examiner value gauges including multi-part indicators for displaying information from the central server.

15. The method as recited in claim 13, further comprising:

computing a normalized instance examiner value by averaging the normalized examiner value for all of the one or more average examiner values; and

displaying one instance examiner value gauge representing the normalized instance examiner value.

16. The method as recited in claim 11, further comprising generating an alert when the normalized examiner value exceeds a predetermined limit.

17. In a system for performing target software behavior pattern recognition and analysis on monitored target software being executed on a computer, the monitored target software including one or more monitored code segments defined during run time of the monitored target software, wherein the monitored code segments are defined by sensors that are selectively embedded during run time of the monitored target software, and/or selectively turned on or off during run time of the monitored target software, a method for automatically deriving a subset of measurements from the monitored code segments indicating performance and activity of the monitored target software for use in deriving in real time a baseline from the subset of measurements by computing an examiner value indicative of an aggregate of degree of change and establishing in real time a state of health of the monitored target software based on comparing real-time measurements of the monitored target software to the baseline, the method comprising:

determining resource utilization for each of the one or more monitored code segments; and

adjusting a number of measurements taken for each of the one or more monitored code segments based on its resource utilization;

wherein the resource utilization for each of the one or more monitored code segments is determined by collecting call rate and execution time data for the monitored code segment, computing an average call rate and an average execution time for the monitored code segment, and multiplying the average call rate by the average execution time.

18. The method as recited in claim 17, further comprising adjusting the number of measurements taken for each of the one or more monitored code segments by activating or deactivating the collection of call rate or execution time data for the monitored code segment.

19. The method as recited in claim 17, further comprising adjusting the number of measurements taken for each of the one or more monitored code segments based on a comparison of the average call rate or average execution time of the monitored code segment to specified thresholds.

20. The method as recited in claim 17, further comprising adjusting the number of measurements taken for each of the one or more monitored code segments by periodically sampling the call rate or execution time data for the monitored code segment.

21. The method as recited in claim 17, further comprising adjusting the number of measurements taken for each of the one or more monitored code segments such that an overhead penalty associated with the subset of measurements is less than a user selectable amount.

22. The method as recited in claim 17, wherein for object-oriented monitored target software, the resource utilization for each of the one or more monitored code segments includes memory utilization data for a selected set of objects.

23. In a system for performing target software behavior pattern recognition and analysis on monitored target software being executed on a computer, the monitored target software including one or more monitored code segments defined during run time of the monitored target software which are capable of providing a subset of measurements from the monitored code segments during a data collection time period indicating performance and activity of the monitored target software, the subset of measurements determinable by collecting call rate and execution time data for each monitored code segment, computing an average call rate and an average execution time for each monitored code segment, multiplying the average call rate by the average execution time to generate a resource utilization for each monitored code segment, and adjusting a number of measurements taken for each monitored code segment based on its resource utilization, wherein the monitored code segments are defined by sensors that are selectively embedded during run time of the monitored target software, and/or selectively turned on or off during run time of the monitored target software, a method comprising automatically deriving in real time a baseline from the subset of measurements by computing an examiner value indicative of an aggregate of degree of change during the data collection time period for use in establishing in real time a state of health of the monitored target software by comparing real-time measurements of the monitored target software to the baseline, the method further comprising:

determining an average throughput examiner code segment value by computing a first sum of the average execution times of all code segments for which execution time data was measured during the data collection time period, and dividing the first sum by a first number of code segments for which execution time data was measured during the data collection time period;

determining an average responsiveness examiner code segment value by computing a second sum of the average call rate of all code segments for which call rate data was measured during the data collection time period, and dividing the second sum by a second number of code segments for which call rate data was measured during the data collection time period;

determining an average resource utilization examiner code segment value by multiplying the average throughput examiner value by the average responsiveness examiner value;

for a throughput examiner value vector, which comprises the average throughput examiner code segment value computed during the data collection time period, updating a first database used to create a current throughput examiner code segment value baseline by adding the average throughput examiner code segment value to the first database, and computing an updated throughput examiner code segment value baseline from the updated first database;

for a responsiveness examiner value vector, which comprises the average responsiveness examiner code segment value computed during the data collection time period, updating a second database used to create a current responsiveness examiner code segment value baseline by adding the average responsiveness examiner code segment value to the second database, and computing an updated responsiveness examiner code segment value baseline from the updated second database; and

for a resource utilization examiner value vector, which comprises the average resource utilization examiner code segment value computed during the data collection time period, updating a third database used to create a current resource utilization examiner code segment value baseline by adding the average resource utilization examiner code segment value to the third database, and computing an updated resource utilization examiner code segment value baseline from the updated third database.

24. The method as recited in claim 23, further comprising maintaining multiple sets of databases and baselines for each examiner value, each set corresponding to different time periods longer than the data collection time period.

25. The method as recited in claim 23, further comprising establishing a state of health of the monitored target software by comparing real-time measurements of the monitored target software to the baseline.

26. The method as recited in claim 25, the step of comparing real-time measurements of the monitored target software to the baseline comprising computing a normalized examiner value fur each average examiner code segment value by computing (aiAvg−

Ea)/Ea, where Ea is the average examiner value and aiAvg is the baseline for that average examiner value.

27. The method as recited in claim 23, further comprising:

creating a deviant list of code segments having an execution time that exceeds a particular threshold, the code segments in the deviant list being excluded from the computation of the average responsiveness examiner value; and

including the deviant list as part of the examiner value vector.

28. The method as recited in claim 27, further comprising displaying one examiner value gauge representing the normalized examiner value for each of the one or more average examiner values.

29. The method as recited in claim 28, the examiner value gauges including multi-pan indicators for displaying information from the central server.

30. The method as recited in claim 28, further comprising:

computing a normalized instance examiner value by averaging the normalized examiner value for all of the one or more average examiner values; and

displaying one instance examiner value gauge representing the normalized instance examiner value.

31. The method as recited in claim 26, further comprising generating an alert when the normalized examiner value exceeds a predetermined limit.

32. An article of manufacture comprising:

a computer usable medium having computer readable program code means embodied therein for target software behavior pattern recognition and analysis of monitored target software being executed on a computer, the computer readable program code means in the article of manufacture comprising;

computer readable program code means for automatically defining one or more monitored code segments in the monitored target software during run time of the monitored target software, wherein the monitored code segments are defined by sensors that are selectively embedded during run time of the monitored target software, and/or selectively turned on or off during run time of the monitored target software;

computer readable program code means for causing a computer to automatically derive a subset of measurements from the monitored code segments indicating performance and activity of the monitored target software;

computer readable program code means for causing the computer to automatically derive in real time a baseline from the subset of measurements by computing an examiner value indicative of an aggregate of degree of change; and

computer readable program code means for causing the computer to establish in real time a state of health of the monitored target software based on comparing real-time measurements of the monitored target software to the baseline.

33. The article of manufacture as recited in claim 32, the computer readable program code means for causing a computer to automatically derive a subset of measurements further comprising:

computer readable program code means for causing the computer to determine resource utilization for each of the one or more monitored code segments; and

computer readable program code means for causing the computer to adjust a number of measurements taken for each of the one or more monitored code segments based on its resource utilization.

34. The article of manufacture as recited in claim 33, further comprising:

computer readable program code means for causing the computer to compute the resource utilization for each of the one or more monitored code segments by;

collecting call rate and execution time data for the monitored code segment;

computing an average call rate and an average execution time for the monitored code segment; and

multiplying the average call rate by the average execution time.

35. The article of manufacture as recited in claim 34, further comprising computer readable program code means for causing the computer to adjust the number of measurements taken for each of the one or more monitored code segments by activating or deactivating the collecting of call rate or execution time data forte monitored code segment.

36. The article of manufacture as recited in claim 34, further comprising computer readable program code means for causing the computer to, adjust the number of measurements taken for each of the one or more monitored code segments based on a comparison of the average call rate or average execution time of the monitored code segment to specified thresholds.

37. The article of manufacture as recited in claim 34, further comprising computer readable program code means for causing the computer to adjust the number of measurements taken for each of the one or more monitored code segments by periodically sampling the call rate or execution time data for the monitored code segment.

38. The article of manufacture as recited in claim 32, further comprising computer readable program code means for causing the computer to adjust the number of measurements taken fur each of the one or more monitored code segments such that an overhead penalty associated with the subset of measurements is less than a user selectable amount.

39. The article of manufacture as recited in claim 34, wherein for object-oriented monitored target software, the computer readable program code means for causing the computer to compute the resource utilization for each of the one or more monitored code segments further comprises:

computer readable program code means for causing the computer to include memory utilization data for a selected set of objects in the resource utilization for each of the one or more monitored code segments.

40. The article, of manufacture as recited in claim 34 wherein, during a data collection time period, the computer readable program code means for causing the computer to automatically derive a baseline from the subset of measurements further comprises:

computer readable program code means for causing the computer to compute an average throughput examiner code segment value by computing a first sum of the average execution times of all code segments for which execution time data was measured during the data collection time period, and dividing the first sum by a first number of code segments for which execution time data was measured during the data collection time period;

computer readable program code means for causing the computer to compute an average responsiveness examiner code segment value by computing a second sum of the average call rate of all code segments for which call rate data was measured during the data collection time period, and dividing the second sum by a second number of code segments for which call rate data was measured during the data collection time period;

computer readable program code means for causing the computer to compute an average resource utilization examiner code segment value by multiplying the average throughput examiner value by the average responsiveness examiner value;

for a throughput examiner value vector, which comprises the average throughput examiner code segment value computed during the data collection time period, computer readable program code means for causing the computer to update a first database used to create a current throughput examiner code segment value baseline by adding the average throughput examiner code segment value to the first database, and compute an updated throughput examiner code segment value baseline from the updated first database;

for a responsiveness examiner value vector, which comprises the average responsiveness examiner code segment value computed during die data collection time period, computer readable program code means for causing the computer to update a second database used to create a current responsiveness examiner code segment value baseline by adding the average responsiveness examiner code segment value to the second database, and compute an updated responsiveness examiner code segment value baseline from the updated second database; and

for a resource utilization examiner value vector, which comprises the average resource utilization examiner code segment value computed during the data collection time period, computer readable program code means for causing the computer to updating a third database used to create a current resource utilization examiner code segment value baseline by adding the average resource utilization examiner code segment value to the third database, and computing an updated resource utilization examiner code segment value baseline from the updated third database.

41. The article of manufacture as recited in claim 40, further comprising computer readable program code means for causing the computer to maintain multiple sets of databases and baselines for each examiner value, each set corresponding to different time periods longer than the data collection time period.

42. The article of manufacture as recited in claim 40, the computer readable program code means for causing the computer to establish a state of health of the monitored target software based on comparing real-time measurements of the monitored target software to the baseline further comprising:

computer readable program code means for causing the computer to compute a normalized examiner value for each average examiner code segment value by computing (aiAvg−

Ea)/Ea, where Ea is the average examiner code segment value and aiAvg is the baseline for that average examiner code segment value.

43. The article of manufacture as recited in claim 40 wherein, during the data collection time period, the computer readable program code means for causing the computer to automatically derive a baseline from the subset of measurements further comprises:

computer readable program code means for causing the computer to create a deviant list of code segments having an execution time that exceeds a particular threshold, the code segments in the deviant list being excluded from the computation of the average responsiveness examiner value; and

computer readable program code means for causing the computer to include the deviant list as part of the examiner value vector.

44. The article of manufacture as recited in claim 42, further comprising computer readable program code means for causing the computer to display one examiner value gauge representing the normalized examiner value for each of the one or more average examiner values.

45. The article of manufacture as recited in claim 44, further comprising computer readable program code means for causing the computer to display examiner value gauges including multi-pad indicators for displaying information from the central server.

46. The article of manufacture as recited in claim 44, further comprising:

computer readable program code means for causing the computer to compute a normalized instance examiner value by averaging the normalized examiner value for all of the one or more average examiner values; and

computer readable program code means for causing the computer to display one instance examiner value gauge representing the normalized instance examiner value.

47. The ankle of manufacture as recited in claim 42, further comprising computer readable program code means for causing the computer to generate an alert when the normalized examiner value exceeds a predetermined limit.

48. In a system for performing target software behavior pattern recognition and analysis on monitored target software being executed on a computer, an article of manufacture comprising:

a computer usable medium having computer readable program code means embodied therein for automatically deriving a subset of measurements from monitored code segments defined during run time of the monitored target software, indicating performance and activity of the monitored target software for use in deriving in real time a baseline from the subset of measurements by computing an examiner value indicative of an aggregate of degree of change and establishing in real time a state of health of the monitored target software based on comparing real-time measurements of the monitored target software to the baseline, wherein the monitored code segments are defined by sensors that are selectively embedded during run time of the monitored target software, and/or selectively turned on or off during run time of the monitored target software, comprising;

computer readable program code means for causing a computer to determine resource utilization for each of the one or more monitored code segments;

computer readable program code means for causing the computer to adjust a number of measurements taken for each of the one or more monitored code segments based on its resource utilization; and

computer readable program code means for causing the computer to determine the resource utilization for each of the one or more monitored code segments by collecting call rate and execution time data for the monitored code segment, computing an average call rate and an average execution time for the monitored code segment, and multiplying the average call rate by the average execution time.

49. The article of manufacture as recited in claim 48, further comprising computer readable program code means for causing the computer to adjust the number of measurements taken for each of the one or more monitored code segments by activating or deactivating the collection of call rate or execution time data for the monitored code segment.

50. The article of manufacture as recited in claim 48, further comprising computer readable program code means for causing the computer to adjust the number of measurements taken for each of the one or more monitored code segments bused on a comparison of the average call rate or average execution time of the monitored code segment to specified thresholds.

51. The article of manufacture as recited in claim 48, further comprising computer readable program code means for causing the computer to adjust the number of measurements taken for each of the one or more monitored code segments by periodically sampling the call rate or execution time data for the monitored code segment.

52. The article of manufacture as recited in claim 48, further comprising computer readable program code means for causing the computer to adjust the number of measurements taken for each of the one or more monitored code segments such that an overhead penalty associated with the subset of measurements is less than a user selectable amount.

53. The article of manufacture as recited in claim 48, wherein for object oriented monitored target software, the computer readable program code means for causing the computer to determine the resource utilization for each of the one or more monitored code segments further comprises:

computer readable program code means for causing the computer to include memory utilization data for a selected set of objects in the resource utilization for each of the one or more monitored code segments.

54. In a system for performing target software behavior pattern recognition and analysis on monitored target software being executed on a computer, the monitored target software including one or more monitored code segments capable of providing a subset of measurements from the monitored code segments during a data collection time period indicating performance and activity of the monitored target software, the subset of measurements determinable by collecting call rate and execution time data for each monitored code segment, computing an average call rate and an average execution time for each monitored code segment, multiplying the average call rate by the average execution time to generate a resource utilization for each monitored code segment, and adjusting a number of measurements taken for each monitored code segment based on its resource utilization, an article of manufacture comprising:

a computer usable medium comprising computer readable program code means embodied therein for automatically defining the monitored code segments during run time of the monitored target software, automatically deriving in real time a baseline from the subset of measurements by computing an examiner value indicative of an aggregate of degree of change during the data collection time period for use in establishing in real time a state of health of the monitored target software by comparing real-time measurements of the monitored target software to the baseline, wherein the monitored code segments are defined by sensors that are selectively embedded during run time of the monitored target software, and/or selectively turned on or off during run time of the monitored target software, said computer usable medium further comprising;

computer readable program code means for causing a computer to compute an average throughput examiner code segment value by computing a first sum of the average execution times of all code segments for which execution time data was measured during the data collection time period, and dividing the first sum by a first number of code segments for which execution time data was measured during the data collection time period;

computer readable program code means for causing the computer to compute an average responsiveness examiner code segment value by computing a second sum of the average call rate of all code segments for which call rate data was measured during the data collection time period, and dividing the second sum by a second number of code segments for which call rate data was measured during the data collection time period;

computer readable program code means for causing the computer to compute an average resource utilization examiner code segment value by multiplying the average throughput examiner value by the average responsiveness examiner value;

for a throughput examiner value vector, which comprises the average throughput examiner code segment value computed during the data collection time period, computer readable program code means for causing the computer to update a first database used to create a current throughput examiner code segment value baseline by adding the average throughput examiner code segment value to the first database, and compute an updated throughput examiner code segment value baseline from the updated first database;

for a responsiveness examiner value vector, which comprises the average responsiveness examiner code segment value computed during the data collection time period, computer readable program code means for causing the computer to update a second database used to create a current responsiveness examiner code segment value baseline by adding the average responsiveness examiner code segment value to the second database, and compute an updated responsiveness examiner code segment value baseline from the updated second database; and

for a resource utilization examiner value vector, which comprises the average resource utilization examiner code segment value computed during the data collection time period, computer readable program code means for causing the computer to updating a third database used to create a current resource utilization examiner code segment value baseline by adding the average resource utilization examiner code segment value to the third database, and computing an updated resource utilization examiner code segment value baseline from the updated third database.

55. The article of manufacture as recited in claim 54, further comprising computer readable program code means for causing the computer to maintain, multiple sets of databases and baselines for each examiner value, each set corresponding to different time periods longer than the data collection time period.

56. The article of manufacture as recited in claim 54, further comprising computer readable program code means for causing the computer to establish a state of health of the monitored target software by comparing real-time measurements of the monitored target software to the baseline.

57. The article of manufacture as recited in claim 56, the computer readable program code means for causing the computer to compare real-time measurements of the monitored target software to the baseline further comprising:

computer readable program code means for causing the computer to compute a normalized examiner value for each average examiner code segment value by computing (aiAvg−

Ea)/Ea, where Ea is the average examiner value and aiAvg is the baseline for that average examiner value.

58. The article of manufacture as recited in claim 54, further comprising:

computer readable program code means for causing the computer to create a deviant list of code segments having an execution time that exceeds a particular threshold, the code segments in the deviant list being excluded from the computation of the average responsiveness examiner value; and

computer readable program code means for causing the computer to include the deviant list as part of the examiner value vector.

59. The article of manufacture as recited in claim 58, further comprising computer readable program code means for causing the computer to display examiner value gauges representing the normalized examiner value for each of the one or more average examiner values.

60. The article of manufacture as recited in claim 59, further comprising computer readable program code means for causing the computer to display examiner value gauges including multi-part indicators for displaying information from the central server.

61. The article of manufacture as recited in claim 59, further comprising computer readable program code means for causing the computer to:

compute a normalized instance examiner value by averaging the normalized examiner value for all of the one or more average examiner values; and

display one instance examiner value gauge representing the normalized instance examiner value.

62. The article of manufacture as recited in claim 57, further comprising computer readable program code means for causing the computer to generate an alert when the normalized examiner value exceeds a predetermined limit.