Method and system for optimizing configuration classification of software

• US 8,326,783 B2
• Filed: 02/29/2008
• Issued: 12/04/2012
• Est. Priority Date: 04/05/2005
• Status: Active Grant

• Current Assignee: Rakuten Group, Inc.
• Original Assignee: International Business Machines Corporation
• Inventors: Ashcraft, Kenneth Michael, Bergamaschi, Flavio Alvarenga, Bertsis, Viktors, Burke, Erin Christine, Huynh, Duy, Rao, Santhosh, Vianney, Duc J., Wong, Peter Wai Yu
• Primary Examiner(s): Chaki, Kakali
• Assistant Examiner(s): Tran, Mai T
• Application Number:
  US12/040,188
• Publication Number:
  US 20080147589A1
• Time in Patent Office: 1,740 Days
• Field of Search: 706/45, 706/12, 706/21, 706/46, 706/53
• US Class Current: 706/45
• CPC Class Codes: G06F 11/3672 Test management
• CPC Class Codes: G06F 11/3696 Methods or tools to render software testable
• CPC Class Codes: G06F 8/71 Version control security arrangements therefor G06F21/57; Configuration management

Abstract

A method and system to minimize redundancy in testing a new software system by utilizing a knowledge base is provided. The knowledge base may be represented in a form of a decision tree including leaf nodes which store previously tested optimal configurations. When the knowledge base does not contain an optimal configuration for the new software, an optimal solution for the new software system may be determined by adding a further decision tree branch to the initial decision tree. A desirable tree branch may be selected from a table including a predetermined set of desirable tree branches for each category of software. As a result, the number of solutions to be tested on the input software may be limited and cost and effort to run redundant tests on a software system may be reduced.

Claims

1. A method, comprising steps of:

constructing a knowledge base having optimal configurations for a plurality of categorized software systems, comprising;

using a computer or processor to perform the steps of;

creating a plurality of vectors of a number of dimensions based on a sum of system parameters, profiling data and tuning variables;

generating a first set of clusters of the vectors based on an overall score of each of the vectors;

generating a second set of clusters of the vectors based on ordered system configuration parameters and tuning parameters;

deriving relative importance of system configuration parameters and the tuning parameters for different system types; and

prioritizing the system configuration parameters and the tuning parameters for each of said different system types and generating a decision tree for each of said different system types;

categorizing a software system;

searching the knowledge base having optimal configurations for said plurality of categorized software systems to determine a configuration parameter set for the software system based on the categorizing of said software system, said knowledge base having optimal configurations being stored in a storage device;

determining whether the configuration parameter set is an optimal configuration for said software system; and

if the configuration parameter set is not an optimal configuration for said software system, conducting further testing of said software system utilizing the knowledge base to determine an optimal configuration parameter set.

2. The method of claim 1, further comprising:

building a further knowledge base based on the received category of the software system, comprising;

selecting a desirable decision tree branch of said decision tree based on the received category of the software system; and

building a further decision tree branch connected to said decision tree.

3. The method of claim 1, wherein each of the configuration parameters comprises one of the ordered system configuration parameters, profiling data, the tuning parameters, or unordered system configuration parameters, the ordered system configuration parameters each have a real number value between an upper and a lower bound, and the unordered system configuration parameters each have a discrete value of a set of known possible discrete values with no direct comparability between discrete values.

4. The method of claim 1, wherein said generating a first set of clusters of the vectors based on an overall score of each of the vectors comprises:

generating the first set of clusters of the vectors based on an overall score of each of the vectors utilizing a convex hull algorithm.

5. The method of claim 4, wherein said generating the first set of clusters of the vectors based on an overall score of each of the vectors utilizing a convex hull algorithm comprises:

determining a minimum number of points which constitute a cluster; and

generating the first set of clusters of the vectors based around each of a plurality of possible values for at least one unordered top-level system parameter.

6. The method of claim 1, wherein said generating a second set of clusters of the vectors based on ordered system configuration parameters and tuning parameters comprises:

generating the second set of clusters of the vectors based on each of the tuning parameters and each of the ordered system configuration parameters from the first set of clusters of the vectors generated based on said overall score of each of the vectors.

7. The method of claim 6, wherein said generating the second set of clusters of the vectors based on each of the tuning parameters and each of the ordered system configuration parameters from the first set of clusters of the vectors generated based on said overall score of each of the vectors comprises:

generating the second set of clusters of the vectors based on each of the tuning parameters and each of the ordered system parameter from the clusters of the vectors with highest overall scores of the first set of clusters of the vectors generated based on an overall score of each of the vectors.

8. The method of claim 1, wherein said deriving relative importance of system configuration parameters and the tuning parameters for different system types comprises:

identifying the tuning parameters and the ordered system configuration parameters based on each of said different system types based on the tuning parameters and the ordered system configuration parameters of clusters with tight boundaries within a cluster with a highest overall score of the first set of clusters of the vectors generated based on an overall score of each of the vectors.

9. The method of claim 8, wherein said prioritizing the system configuration parameters and the tuning parameters for each of said different system types and generating a decision tree for each of said different system types comprises:

generating an ordered list for each of said different system types by prioritizing the tuning parameters and the ordered system configuration parameters in order of a distance between boundaries of the tuning parameters and the ordered system configuration parameters normalized to a range of said boundaries of the tuning parameters and the ordered system configuration parameters; and

generating said decision tree for each of said different system types utilizing said ordered list.

10. A non-transitory computer-readable medium embodying computer readable instructions for a processor of a computer to implement a method comprising:

constructing a knowledge base having optimal configurations for a plurality of categorized software systems, comprising;

creating a plurality of vectors of a number of dimensions based on a sum of system parameters, profiling data and tuning variables;

generating a first set of clusters of the vectors based on an overall score of each of the vectors;

generating a second set of clusters of the vectors based on ordered system configuration parameters and tuning parameters;

deriving relative importance of system configuration parameters and the tuning parameters for different system types; and

prioritizing the system configuration parameters and the tuning parameters for each of said different system types and generating a decision tree for each of said different system types;

categorizing a software system;

searching the knowledge base having optimal configurations for said plurality of categorized software systems to determine a configuration parameter set for the software system based on the categorizing of said software system;

determining whether the configuration parameter set is an optimal configuration for said software system; and

if the configuration parameter set is not an optimal configuration for said software system, conducting further testing of said software system utilizing the knowledge base to determine an optimal configuration parameter set.

11. The non-transitory computer-readable medium of claim 10, wherein the method further comprises:

building a further knowledge base based on the received category of the software system, comprising;

selecting a desirable decision tree branch of said decision tree based on the received category of the software system; and

building a further decision tree branch connected to said decision tree.

12. The non-transitory computer-readable medium of claim 10, wherein each of the configuration parameters comprises one of the ordered system configuration parameters, profiling data, the tuning parameters, or unordered system configuration parameters, the ordered system configuration parameters each have a real number value between an upper and a lower bound, and the unordered system configuration parameters each have a discrete value of a set of known possible discrete values with no direct comparability between discrete values.

13. The non-transitory computer-readable medium of claim 10, wherein said generating a first set of clusters of the vectors based on an overall score of each of the vectors comprises:

generating the first set of clusters of the vectors based on an overall score of each of the vectors utilizing a convex hull algorithm.

14. The non-transitory computer-readable medium of claim 13, wherein said generating the first set of clusters of the vectors based on an overall score of each of the vectors utilizing a convex hull algorithm comprises:

determining a minimum number of points which constitute a cluster; and

generating the first set of clusters of the vectors based around each of a plurality of possible values for at least one unordered top-level system parameter.

15. The non-transitory computer-readable medium of claim 10, wherein said generating a second set of clusters of the vectors based on ordered system configuration parameters and tuning parameters comprises:

generating the second set of clusters of the vectors based on each of the tuning parameters and each of the ordered system configuration parameters from the first set of clusters of the vectors generated based on said overall score of each of the vectors.

16. The non-transitory computer-readable medium of claim 15, wherein said generating the second set of clusters of the vectors based on each of the tuning parameters and each of the ordered system configuration parameters from the first set of clusters of the vectors generated based on said overall score of each of the vectors comprises:

generating the second set of clusters of the vectors based on each of the tuning parameters and each of the ordered system parameter from the clusters of the vectors with highest overall scores of the first set of clusters of the vectors generated based on an overall score of each of the vectors.

17. The non-transitory computer-readable medium of claim 10, wherein said deriving relative importance of system configuration parameters and the tuning parameters for different system types comprises:

identifying the tuning parameters and the ordered system configuration parameters based on each of said different system types based on the tuning parameters and the ordered system configuration parameters of clusters with tight boundaries within a cluster with a highest overall score of the first set of clusters of the vectors generated based on an overall score of each of the vectors.

18. The non-transitory computer-readable medium of claim 17, wherein said prioritizing the system configuration parameters and the tuning parameters for each of said different system types and generating a decision tree for each of said different system types comprises:

generating an ordered list for each of said different system types by prioritizing the tuning parameters and the ordered system configuration parameters in order of a distance between boundaries of the tuning parameters and the ordered system configuration parameters normalized to a range of said boundaries of the tuning parameters and the ordered system parameters; and

generating said decision tree for each of said different system types utilizing said ordered list.