Using metric to evaluate performance impact

• US 7,912,573 B2
• Filed: 06/17/2008
• Issued: 03/22/2011
• Est. Priority Date: 06/17/2008
• Status: Active Grant

• Current Assignee: Microsoft Technology Licensing LLC (Microsoft Corporation)
• Original Assignee: Microsoft Corporation
• Inventors: Yuan, Zhongsheng, Loboz, Charles Z., Lee, Seung H.
• Primary Examiner(s): Feliciano; Eliseo Ramos
• Assistant Examiner(s): Desta; Elias
• Application Number:
  US12/140,297
• Publication Number:
  US 20090312983A1
• Time in Patent Office: 1,008 Days
• Field of Search: 702/179, 702/182, 702/189, 709/200, 709/223, 710/36, 718/103, 718/104
• US Class Current: 700/182
• CPC Class Codes: G06F 11/3409 for performance assessment
• CPC Class Codes: G06F 11/3447 Performance evaluation by modeling
• CPC Class Codes: G06F 11/3466 Performance evaluation by tracing or monitoring
• CPC Class Codes: G06F 2201/88 Monitoring involving counting

Abstract

Analysis may be made of the amount that a load on a machine impacts the machine'"'"'s performance. Performance counters on the machine record raw statistical data, such as a given resource'"'"'s current utilization. The values of these counters may be captured. A n-bin histogram may be created that shows how many of the captured performance counter values occur within various ranges, such as 0-10% utilization, 10-20%, etc. A weight may be assigned to each bin. A weighted sum of the bins may be calculated by multiplying the number of occurrences in each bin by the bin'"'"'s weight, and adding the products together. The weights may be chosen to reflect the relative amounts that particular performance counter values impact the overall performance of a machine. Thus, a metric that represents performance impact may be calculated based on the weighted sum.

Claims

1. A method of analyzing performance of a plurality of machines, each of the machines maintaining a performance counter that can acquire values within a range, the method comprising:

determining amounts of performance impact associated with each of a plurality of values in the range;

assigning weights to each of the plurality of values based on a determined performance impact associated with each of the plurality of values;

for each machine among the plurality of machines;

receiving a set of performance counter values captured from the machine'"'"'s performance counter at instances during a period of time; and

calculating a metric for the machine by applying a formula to the set, the formula being based on the weights; and

presenting a report that is based on metrics calculated for the plurality of machines,

wherein said assigning of weights comprises;

applying, to said plurality of values, a function that grows faster than a linear function as a maximum value within said range is approached.

2. The method of claim 1, further comprising:

for each machine among the plurality of machines;

creating a histogram for the set of values captured from the machine, the histogram having a plurality of bins in which each bin represents a sub-range of the range, each of the bins storing a count of a number of occurrences of performance counter values in the set that fall within a given bin'"'"'s sub-range, each of the bins being associated with one of the weights, and wherein said calculating comprises;

multiplying a weight associated with each bin by the count stored in that bin.

3. The method of claim 2, wherein the bins represent uniformly-sized sub-ranges of the range.

4. The method of claim 1, wherein said assigning comprises:

applying, to each of the plurality of values, u, a function based on 1/(1−

u)ⁿ, where 0.5≦

n≦

4.0.

5. The method of claim 1, further comprising:

identifying which of the metrics exceeds a threshold, wherein the report comprises an identification of those machines whose metric exceeds said threshold.

6. The method of claim 1, further comprising:

identifying the n highest values among said metrics, wherein the report comprises an identification of machines whose metrics are among the n highest values.

7. One or more computer-readable storage media that store executable instructions to perform a method of analyzing performance of a machine, the method comprising:

receiving a plurality of performance counter values generated by said machine;

based on said plurality of performance counter values, creating a histogram that represents (a) a plurality of different ranges of said performance counter values and, (b) for each range, a count of a number of occurrences of each of the performance counter values that fall into the range;

calculating a metric that is based on said histogram, wherein said metric is based on a formula that is based on a weighted sum of counts in each of the ranges, wherein, in said weighted sum, each count is multiplied by a weight that is assigned by a non-linear function of representative values of the ranges; and

presenting a performance report that comprises or is based on said metric.

8. The one or more computer-readable storage media of claim 7, wherein said performance counter values represent a proportion of maximum utilization of a resource, said proportion being expressible as a value between zero and one, and wherein, for a representative value, u, of each of said ranges in said histogram, said function assigns to each of the ranges a weight that is proportional to 1/(1−

u)ⁿ, where 0.5≦

n≦

4.0.

9. The one or more computer-readable storage media of claim 7, wherein said formula is further based on a logarithm of said weighted sum, and wherein said metric is proportional to said logarithm.

10. The one or more computer-readable storage media of claim 7, further comprising:

assigning said machine to one of a plurality of performance categories based on said metric,

wherein content of said performance report is based on which of the categories said machine is assigned to.

11. The one or more computer-readable storage media of claim 7, wherein said machine is a member of a group of machines, and wherein said histogram is further based on performance counter values collected from all of the machines in said group.

12. The one or more computer-readable storage media of claim 7, wherein said performance counter values are bounded by a minimum and a maximum, and wherein said ranges are uniformly-sized numerical regions between said minimum and said maximum.

13. The one or more computer-readable storage media of claim 7, wherein said machine is one of a plurality of machines, and wherein the method further comprises:

calculating said metric for each of said plurality of machines; and

receiving a request to identify those ones of said plurality of machines whose metric exceeds a threshold;

and wherein said presenting comprises;

identifying, in said report, those ones of said plurality of machines whose metric exceeds said threshold.

14. The one or more computer-readable storage media of claim 7, wherein said machine is one of a plurality of machines, and wherein the method further comprises:

calculating said metric for each of said plurality of machines; and

receiving a request to identify n machines having the highest metrics;

and wherein said presenting comprises;

identifying, in said report, the n machines that have the highest metric among said plurality of machines.

15. A system comprising:

a plurality of machines, wherein each of the machines comprises a performance counter that measures an aspect of the machines'"'"' performance, said performance counter having a maximum acquirable value; and

an analyzer that is communicatively connected to a machine that is one of the plurality of machines, the analyzer comprising;

a histogram generator that receives a set of performance counter values captured at said machine, that creates a histogram having a plurality of bins corresponding to a plurality of ranges of performance counter values, each of the bins storing a count of occurrences of performance counter values in said set that fall into each of said ranges, each bin, i, having a representative value, r, that falls within each bin'"'"'s range; and

a metric calculator that calculates a metric for said machine based on a sum Σ

_iw_ic_i over all bins, i, in the histogram, where w_i is a weight associated with the i^th bin, where c_i is the count of the i^th bin, and where w_i is proportional to a function ƒ

(r) that grows faster than a linear function as said maximum acquirable value is approached; and

wherein the system further comprises;

a report generator that generates a report that is based on or comprises said metric.

16. The system of claim 15, wherein said function ƒ

(r) is 1/(1−

r)ⁿ, where 0.5≦

n≦

4.0.

17. The system of claim 15, wherein said metric calculator calculates metrics for said plurality of machines, and wherein said report generator selects those ones of said plurality of machines whose metrics exceed a threshold and indicates the selected machines in said report.

18. The system of claim 15, wherein said metric calculator calculates metrics for said plurality of machines, and wherein said report generator selects the n machines having the highest metrics and indicates the selected machines in said report.

19. The system of claim 15, wherein said report generator identifies a growth or a decline in said metric, and wherein said report is based on identification of said growth or said decline.