Generating a test workload for a database

• US 8,924,402 B2
• Filed: 10/08/2012
• Issued: 12/30/2014
• Est. Priority Date: 12/20/2011
• Status: Expired due to Fees

• Current Assignee: International Business Machines Corporation
• Original Assignee: International Business Machines Corporation
• Inventors: Fuh, You-Chin, Liu, Heng, Schenker, Michael, Wei, Ke Wei, Yang, Xin Ying, Li, Shuo
• Primary Examiner(s): Spieler, William
• Application Number:
  US13/647,122
• Publication Number:
  US 20130159353A1
• Time in Patent Office: 813 Days
• Field of Search: 707/757
• US Class Current: 707/757
• CPC Class Codes: G06F 11/3414 Workload generation, e.g. scripts, playback
• CPC Class Codes: G06F 11/3672 Test management
• CPC Class Codes: G06F 21/6254 by anonymising data, e.g. decorrelating personal data from the owner's identification
• CPC Class Codes: G06F 2201/80 Database-specific techniques

Abstract

A method of transferring files in a data-processing network using a current node within the network includes reading an outbound content and outbound characteristics of an outbound file. An outbound message is created having outbound strings including a first set of the outbound strings representing the outbound characteristics and a second set of the outbound strings representing the outbound content. The outbound message is sent to a receiver node within the network. An inbound message is received from a sender node within the network. The inbound message has inbound strings including a first set of the inbound strings representing inbound characteristics and a second set of the inbound strings representing inbound content. An inbound file having the inbound content is stored, and the inbound characteristics are applied to the inbound file.

Claims

1. A system for generating a test workload for a database, comprising:

a computer hardware system having a processor, wherein the computer hardware system includes

a workload capturing module configured to capture a workload comprising database statements sent to a source database on a source system;

a meta information collecting module configured to collect the data schema and data distribution statistics in the source database;

a data generating module configured to, based on the data schema collected by the meta information collecting module, create data objects with the same data schema in a target database, and based on the data distribution statistics collected by the meta information collecting module, generate data with similar data distribution characteristics in the target database; and

a statement mutating module configured to mutate at least some original database statements in the workload captured by the workload capturing module, wherein

execution of the mutated database statements on the target database will have similar performance characteristics to those of execution of the original database statements on the source database to obtain a test workload.

2. The system as recited in claim 1, further comprising:

a test workload execution module configured to execute the test workload on the target database to perform testing on the target database.

3. The system as recited in claim 1, wherein

the meta information collecting module is configured to only collect the data schema and data distribution statistics in the source database involved by original database statements included in the workload.

4. The system as recited in claim 1, wherein

the similar performance characteristics refer to similar predicate filter factors.

5. The system as recited in claim 1, wherein

the data distribution statistics include the maximum, minimum, cardinality and size of the data in a column of a database table, or the respective occurrence frequencies of a number of values with the highest occurrence frequencies in the data of a column of a database table and the maximum, minimum, cardinality and size of each data section divided by the values.

6. The system as recited in claim 1, further comprising:

a constraint obtaining module configured to obtain constrains related to the selectivity of the data involved by original database statements on the source system, wherein

the data generating module is further configured to generate data in the target database that satisfies or substantially satisfies the constraints related to the selectivity and with the similar data distribution statistics.

7. The system as recited in claim 6, wherein

the constrains related to the selectivity of the data involved by original database statements comprise constraints related to the filter factors of a plurality of predicates in an original database statement involving a plurality of columns of a same database table;

and

the data generating module is further configured to;

first generate data satisfying or substantially satisfying the filter factors of predicates or combination thereof involving more columns, and then generate data satisfying or substantially satisfying the filter factors of predicates or combination thereof involving less columns.

8. The system as recited in claim 6, wherein

the constrains related to the selectivity of the data involved by original database statements comprise constrains related to the filter factors of the predicates in an original database statement involving a plurality of columns of a plurality of database tables having a join relationship therebetween;

and

the data generating module is further configured to;

make the data generated in the target database satisfy or substantially satisfy the filter factors of the predicates.

9. The system as recited in claim 6, further comprising:

a predicate classification module configured to classify predicates and combinations thereof in original database statements into different groups based on the involved columns of database tables; and

an occurrence frequency calculating module configured to calculate the occurrence frequencies of the predicates and combinations thereof in different groups in database statements, wherein

the constraint obtaining module is further configured to only obtain the filter factors of the predicates and combinations thereof in groups with higher frequencies.

10. The system as recited in claim 1, wherein the statement mutating module further comprises:

a statement mutating sub-module configured to, for an original database statement comprising a join relationship, delete the local predicates therein, and add the columns of database tables involved in the local predicates into the SELECT clause, to obtain a modified database statement;

a result set obtaining sub-module configured to execute the modified database statement on the target database to obtain a result set;

a column value finding sub-module configured to find in the result set column values the selectivity of which are similar to the individual or combined filter factors of the local predicates in the source database; and

a literal substituting sub-module configured to substitute the column values for the literals in the local predicates in the database statement comprising a join relationship.

11. A computer-implemented method for generating a test workload for a database, comprising:

capturing a workload comprising database statements sent to a source database on a source system;

collecting the data schema and data distribution statistics in the source database;

creating, based on the data schema collected, data objects with the same data schema in a target database;

generating, based on the data distribution statistics collected, data with similar data distribution characteristics in the target database; and

mutating at least some original database statements in the workload captured by a workload capturing module, wherein

execution of the mutated database statements on the target database will have similar performance characteristics to those of execution of the original database statements on the source database to obtain a test workload.

12. The method as recited in claim 11, further comprising:

executing the test workload on the target database to perform testing on the target database.

13. The method as recited in claim 11, wherein

the collecting the data schema and data distribution statistics in the source database comprises only collecting the data schema and data distribution statistics in the source database involved by original database statements included in the workload.

14. The method as recited in claim 11, wherein

the similar performance characteristics refer to similar predicate filter factors.

15. The method as recited in claim 11, wherein

the data distribution statistics include the maximum, minimum, cardinality and size of the data in a column of a database table, or the respective occurrence frequencies of a number of values with the highest occurrence frequencies in the data of a column of a database table and the maximum, minimum, cardinality and size of each data section divided by the values.

16. The method as recited in claim 11, further comprising:

obtaining constrains related to the selectivity of the data involved by original database statements on the source system, wherein

the generating data with similar data distribution characteristics in the target database further comprises generating data in the target database that satisfies or substantially satisfies the constraints related to the selectivity and with the similar data distribution statistics.

17. The method as recited in claim 16, wherein

the constrains related to the selectivity of the data involved by original database statements comprise constraints related to the filter factors of a plurality of predicates in an original database statement involving a plurality of columns of a same database table;

and

the generating data in the target database that satisfies or substantially satisfies the constraints related to the selectivity and with the similar data distribution statistics further comprises;

first generating data satisfying or substantially satisfying the filter factors of predicates or combination thereof involving more columns, and then generating data satisfying or substantially satisfying the filter factors of predicates or combination thereof involving less columns.

18. The method as recited in claim 16, wherein

the constrains related to the selectivity of the data involved by original database statements comprise constrains related to the filter factors of the predicates in an original database statement involving a plurality of columns of a plurality of database tables having a join relationship therebetween;

and

the generating data in the target database that satisfies or substantially satisfies the constraints related to the selectivity and with the similar data distribution statistics further comprises;

make the data generated in the target database satisfy or substantially satisfy the filter factors of the predicates.

19. The method as recited in claim 16, further comprising:

classifying predicates and combinations thereof in original database statements into different groups based on the involved columns of database tables; and

calculating the occurrence frequencies of the predicates and combinations thereof in different groups in database statements, wherein

the obtaining constrains related to the selectivity of the data involved by original database statements on the source system comprises only obtaining the filter factors of the predicates and combinations thereof in groups with higher frequencies.

20. The method as recited in claim 11, wherein the mutating at least some original database statements in the workload captured further comprises:

for an original database statement comprising a join relationship, deleting the local predicates therein, and adding the columns of database tables involved in the local predicates into the SELECT clause, to obtain a modified database statement;

executing the modified database statement on the target database to obtain a result set;

finding in the result set column values the selectivity of which are similar to the individual or combined filter factors of the local predicates in the source database; and

substituting the column values for the literals in the local predicates in the database statement comprising a join relationship.