Method for ranking analysis tools

• US 8,839,048 B2
• Filed: 03/07/2012
• Issued: 09/16/2014
• Est. Priority Date: 03/07/2012
• Status: Active Grant

• Current Assignee: International Business Machines Corporation
• Original Assignee: International Business Machines Corporation
• Inventors: Cheaz, Nixon, Chalmers, Diane C, McVea, James Y Jr., Stecher, David M
• Primary Examiner(s): PATEL, KAMINI B
• Application Number:
  US13/413,993
• Publication Number:
  US 20130238940A1
• Time in Patent Office: 923 Days
• Field of Search: 714/47.3
• US Class Current: 714/47.3
• CPC Class Codes: G06F 11/079 Root cause analysis, i.e. error or fault diagnosis in a hardware test environment G06F11/22; in a software test environment G06F11/36
• CPC Class Codes: G06F 11/3428 Benchmarking
• CPC Class Codes: G06F 11/3688 for test execution, e.g. scheduling of test suites

Abstract

Analysis tools are used for resolving a service request for software performance problems. Ranking of the analysis tools includes measuring a plurality of times to resolution of a plurality of service requests for software performance problems after runnings of a plurality of analysis tools are initiated; capturing sets of errors in the plurality of service requests; storing identities of the plurality of analysis tools with the times to resolution of the service requests and the sets of errors; determining an average time to resolution of each of the plurality of analysis tools for each set of errors; organizing the plurality of analysis tools into one or more categories using the sets of errors; and ranking the analysis tools within each category using the average times to resolution of the analysis tools within the category.

Claims

1. A computer program product for ranking analysis tools, the computer program product comprising:

a computer readable memory device having computer readable program code embodied therewith, the computer readable program code configured to;

measure a time to resolution of a given software performance problem for each run of any of a plurality of analysis tools, further configured to;

receive a service request for the software performance problem, wherein the service request includes one or more files;

initiate a running of a given analysis tool of the plurality of analysis tools;

measure a time to resolution of the service request from a predetermined time after the given analysis tool is initiated;

capture a set of errors in the one or more files of the service request; and

store an identity of the given analysis tool with the time to resolution of the service request and the set of errors;

determine an average time to resolution of the given software performance problem for each of the plurality of analysis tools; and

rank the plurality of analysis tools using the average times to resolution of the given software performance problems for the plurality of analysis tools.

2. The computer program product of claim 1, wherein the computer readable program code configured to determine the average time to resolution of the given software performance problem for each of the plurality of analysis tools is further configured to:

aggregate the stored identity of the given analysis tool with the time to resolution of the service request and the set of errors across a plurality of service requests; and

determining an average time to resolution of each of the plurality of analysis tools for each set of errors.

3. The computer program product of claim 2, wherein the computer readable program code configured to rank the plurality of analysis tools using the average times to resolution of the given software performance problems for the plurality of analysis tools is further configured to:

organize the plurality of analysis tools into one or more categories using the sets of errors aggregated across the plurality of service requests; and

rank analysis tools within each category using the average times to resolution of the analysis tools within the category.

4. The computer program product of claim 3, wherein the computer readable program code is further configured to:

receive a second service request for a second software performance problem, wherein the second service request includes one or more files;

capture a second set of errors in the second service request files;

determine a given category of analysis tools using the second set of errors; and

recommend one or more analysis tools in the given category based on the rankings of the analysis tools within the given category.

5. A system, comprising:

a processor; and

a computer readable memory device, having computer readable program code embodied therewith, the computer readable program code configured to;

measure a time to resolution of a given software performance problem for each run of any of a plurality of analysis tools, further configured to;

receive a service request for the software performance problem, wherein the service request includes one or more files;

initiate a running of a given analysis tool of the plurality of analysis tools;

measure a time to resolution of the service request from a predetermined time after the given analysis tool is initiated;

capture a set of errors in the one or more files of the service request; and

store an identity of the given analysis tool with the time to resolution of the service request and the set of errors;

determine an average time to resolution of the given software performance problem for each of the plurality of analysis tools; and

rank the plurality of analysis tools using the average times to resolution of the given software performance problems for the plurality of analysis tools.

6. The system of claim 5, wherein the computer readable program code configured to determine the average time to resolution of the given software performance problem for each of the plurality of analysis tools is further configured to:

aggregate the stored identity of the given analysis tool with the time to resolution of the service request and the set of errors across a plurality of service requests; and

determining an average time to resolution of each of the plurality of analysis tools for each set of errors.

7. The system of claim 6, wherein the computer readable program code configured to rank the plurality of analysis tools using the average times to resolution of the given software performance problems for the plurality of analysis tools is further configured to:

organize the plurality of analysis tools into one or more categories using the sets of errors aggregated across the plurality of service requests; and

rank analysis tools within each category using the average times to resolution of the analysis tools within the category.

8. The system of claim 7, wherein the computer readable program code is further configured to:

receive a second service request for a second software performance problem, wherein the second service request includes one or more files;

capture a second set of errors in the second service request files;

determine a given category of analysis tools using the second set of errors; and

recommend one or more analysis tools in the given category based on the rankings of the analysis tools within the given category.

9. A method for ranking analysis tools by a computing system comprising a processor, comprising:

measuring by the processor a time to resolution of a given software performance problem for each run of any of a plurality of analysis tools, further comprising;

receiving a service request for the software performance problem, wherein the service request includes one or more files;

initiating a running of a given analysis tool of the plurality of analysis tools;

measuring a time to resolution of the service request from a predetermined time after the given analysis tool is initiated;

capturing a set of errors in the one or more files of the service request; and

storing an identity of the given analysis tool with the time to resolution of the service request and the set of errors;

determining, by the processor, an average time to resolution of the given software performance problem for each of the plurality of analysis tools; and

ranking, by the processor, the plurality of analysis tools using the average times to resolution of the given software performance problems for the plurality of analysis tools.

10. The method of claim 9, wherein the determining the average time to resolution of the given software performance problem for each of the plurality of analysis tools comprises:

aggregating the stored identity of the given analysis tool with the time to resolution of the service request and the set of errors across a plurality of service requests; and

determining an average time to resolution of each of the plurality of analysis tools for each set of errors.

11. The method of claim 10, wherein the ranking the plurality of analysis tools using the average times to resolution of the given software performance problems for the plurality of analysis tools comprises:

organizing the plurality of analysis tools into one or more categories using the sets of errors aggregated across the plurality of service requests; and

ranking analysis tools within each category using the average times to resolution of the analysis tools within the category.

12. The method of claim 11, further comprising:

receiving a second service request for a second software performance problem, wherein the second service request includes one or more files;

capturing a second set of errors in the second service request files;

determining a given category of analysis tools using the second set of errors; and

recommending one or more analysis tools in the given category based on the rankings of the analysis tools within the given category.

13. A method for ranking analysis tools by a computing system comprising a processor, comprising:

measuring, by the processor, a plurality of times to resolution of a plurality of service requests for software performance problems after runnings of a plurality of analysis tools are initiated;

capturing, by the processor, sets of errors in the plurality of service requests;

storing, by the processor, identities of the plurality of analysis tools with the times to resolution of the service requests and the sets of errors;

determining, by the processor, an average time to resolution of each of the plurality of analysis tools for each set of errors;

organizing, by the processor, the plurality of analysis tools into one or more categories using the sets of errors;

ranking, by the processor, the analysis tools within each category using the average times to resolution of the analysis tools within the category receiving a given service request for a given software performance problem, wherein the given service request includes one or more files;

capturing a given set of errors in the given service request files;

determining a given category of analysis tools using the given set of errors; and

recommending one or more analysis tools in the given category based on the rankings of the analysis tools within the given category.