Data quality and validation within a relational database management system

• US 20070174234A1
• Filed: 01/24/2006
• Published: 07/26/2007
• Est. Priority Date: 01/24/2006
• Status: Abandoned Application

• Current Assignee: International Business Machines Corporation
• Original Assignee: International Business Machines Corporation
• Inventors: Ramsey, Mark
• Application Number:
  US11/338,541
• Publication Number:
  US 20070174234A1
• Time in Patent Office: Days
• US Class Current: 1/1
• CPC Class Codes: G06F 16/217 Database tuning G06F16/2282 takes precedence; database performance monitoring G06F11/3409

Abstract

A system and method for performing data quality and validation analysis within a relational database management system (RDBMS). A method is provided that includes generating a summarization table for a set of data values using an RDBMS function after a modification of the set of values takes place; calculating a deviation from the summarization table using an RDBMS function; and querying the set of data values against the deviation to identify any suspect values.

Claims

1. A method for validating data being inputted into a relational database management system (RDBMS), comprising:

generating a summarization table for a set of data values using an RDBMS function after a modification of the set of data values takes place;

calculating a deviation from the summarization table using an RDBMS function; and

querying the set of data values against the deviation to identify any suspect values.

2. The method of claim 1, wherein the modification includes an addition of new data.

3. The method of claim 1, wherein the generating step dynamically generates the summarization table whenever the set of data values changes.

4. The method of claim 1, wherein the generating and calculating steps are performed as a single process.

5. The method of claim 1, wherein the deviation is selected from the group consisting of:

a mean, a weighted average, a range, a standard deviation, a multiple of a standard deviation, and a statistical analysis.

6. The method of claim 1, comprising the further step of:

investigating any suspect values.

7. A method for validating data being inputted into a relational database management system (RDBMS), comprising:

generating a summarization table for a set of data values using an RDBMS function;

calculating a deviation from the summarization table using an RDBMS function;

proposing an addition of a new data value into the set of data values; and

comparing the new data value with the deviation to determine if the new data value is a suspect value.

8. The method of claim 7, wherein the addition of new data includes a modification of existing data in the set of data values.

9. The method of claim 7, wherein the generating step dynamically generates the summarization table whenever the set of data values changes.

10. The method of claim 7, wherein the generating and calculating steps are performed as a single process.

11. The method of claim 7, wherein the deviation is selected from the group consisting of:

a mean, a weighted average, a range, a standard deviation, a multiple of a standard deviation, and a statistical analysis.

12. The method of claim 7, comprising the further step of:

investigating the new data value if the new data value is a suspect value.

13. A relational database management system (RDBMS) that includes data validation capabilities, comprising:

a system for generating a summarization table for a set of data values using an RDBMS function after a modification of the set of data values takes place;

a system for calculating a deviation from the summarization table using an RDBMS function; and

a system for querying the set of data values against the deviation to identify any suspect values.

14. The RDBMS of claim 13, wherein the modification includes an addition of new data.

15. The RDBMS of claim 13, wherein the system for generating a summarization table generates the summarization table whenever the set of data values changes.

16. The RDBMS of claim 13, wherein the deviation is selected from the group consisting of:

a mean, a weighted average, a range, a standard deviation, a multiple of a standard deviation, and a statistical analysis.

17. A computer program product stored on a computer useable medium for validating data being entered into a database, comprising:

a relational database management system (RDBMS) having;

program code configured for generating a summarization table for a set of data values using an RDBMS function;

program code configured for calculating a deviation from the summarization table using an RDBMS function; and

a data preprocessor having program code configured for comparing a new data value being inputted into the RDBMS with the deviation to determine if the new data value is a suspect value.

18. The computer program product of claim 17, wherein the new data includes a modification of existing data in the set of data values.

19. The computer program product of claim 17, wherein the summarization table is generated dynamically whenever the set of data values changes.

20. The computer program product of claim 17, wherein the deviation is selected from the group consisting of:

a mean, a weighted average, a range, a standard deviation, a multiple of a standard deviation, and a statistical analysis.