STATISTICAL QUALITY MONITORING AND ENHANCEMENT

• US 20110145291A1
• Filed: 12/16/2009
• Published: 06/16/2011
• Est. Priority Date: 12/16/2009
• Status: Active Grant

• Current Assignee: ServiceNow Incorporated
• Original Assignee: International Business Machines Corporation
• Inventors: Aharoni, Ehud, Neuvirth-Telem, Hani, Rosset, Saharon
• Granted Patent: US 8,140,476 B2
• Time in Patent Office: Days
• US Class Current: 707/781
• CPC Class Codes: G06F 16/215 Improving data quality; Data cleansing, e.g. de-duplication, removing invalid entries or correcting typographical errors
• CPC Class Codes: G06F 16/2358 Change logging, detection, and notification replication G06F16/27

Abstract

A method and computer system for monitoring and controlling the quality of tests performed upon a database. A statistical quality layer is provided, which for each new test to be executed on the database, provides recommendations as to a confidence level to be applied to the test and to a number of additional data records to be added to the database in order to maintain the total expected number of type-I errors of tests performed upon the database. The method and computer system further provide access control of users to the database.

Claims

1. A computerized system comprising a database management component, the database management component comprising:

a statistical quality layer for controlling quality parameters of a test performed upon a database, the statistical quality layer comprising a compensation determination component for determining a number of data records to be added to the database so as to limit an expected number of errors associated with the database once the test is performed.

2. The computerized system of claim 1 further comprising a recommendation engine for issuing a recommendation related to the number of data records to be added to the database determined by said compensation determination component.

3. The computerized system of claim 1 further comprising a recommendation engine for issuing a recommendation related to a significance level of the test.

4. The computerized system of claim 1 further comprising a test execution component for executing the test.

5. The computerized system of claim 1 further comprising a report engine for generating a report related to the database or to the test performed upon the database.

6. The computerized system of claim 1 further comprising a manager user interface for receiving a command from a manager of the database.

7. The computerized system of claim 1 further comprising a researcher user interface for receiving a command from a user associated with a test to be performed upon the database.

8. The computerized system of claim 1 further comprising a control component for managing control and information flow within the statistical quality layer.

9. The computerized system of claim 1 further comprising an access control engine for controlling access of a user of the computerized system to the database.

10. The computerized system of claim 1, wherein the errors are type-I errors.

11. The computerized system of claim 1, wherein the recommendation relates to allowing the test, forbidding the test, or stipulating the test upon a condition.

12. A method for controlling a database stored on a storage device comprising:

receiving a request for a test to be executed upon the database;

determining a significance level for the test;

determining a number of data records to be added to the database so that the test is in compliance with the significance level; and

receiving data records as indicated by the number of data records and storing the data records in the database,

whereby the database is transformed to further include the data records.

13. The method of claim 12 further comprising executing the test upon the database.

14. The method of claim 12, wherein the number of additional data records is determined in accordance with an expected total number of type-I errors related to the database, and with the size of the database.

15. The method of claim 14, wherein the number of additional data records is determined such that L(n+δ

)<

aqⁿ−

aq^n+δ

, wherein L is a significance level associated with the test when executed upon n+δ

samples;

n is a number of records in the database;

δ

is the number of additional data records;

q is a number between 0 and 1; and

a is the predetermined quantity.

16. The method of claim 12 further comprising re-executing tests upon the database, in response to said receiving additional data records.

17. The method of claim 14, wherein the number of additional data records is determined such that current levels of all tests which have been previously executed, sums up to a number smaller than a predetermined quantity.

18. The method of claim 17, wherein the number of additional data records is determined such that L₁(n+δ

)+L₂(n+δ

)+ . . . +L_t(n+δ

)<

a, wherein;

L₁, L₂ . . . , L_t-1 are confidence level functions for previously submitted tests;

L_t is confidence level functions for the test;

n is a number of records in the database;

δ

is the number of additional data records; and

a is the predetermined quantity.

19. The method of claim 12 further comprising issuing a recommendation related to allowing the test, forbidding the test, or stipulating the test upon a condition.

20. The method of claim 12 further comprising controlling access of a user to the database.

21. The method of claim 12 wherein a manager of the database determines the significance level for the test and the number of additional data records to be added to the database, based upon at least one parameter supplied by a user of the test.

22. The method of claim 21 wherein the at least one parameter is selected from the group consisting of:

power;

effect size;

statistics of the test, and assumptions on data relevant for the test.

23. A computer program product comprising:

a computer readable media;

first program instruction for receiving a request for a test to be executed upon the database;

second program instruction for determining a significance level for the test;

third program instruction for determining a number of data records to be added to the database so that the test is in compliance with the significance level; and

fourth program instruction for receiving data records as indicated by the number of data records and storing the data records in the database,

whereby the database is transformed to further include the data records.