Computer-implemented system and method for managing data integrity validation rules

• US 20040123101A1
• Filed: 12/18/2002
• Published: 06/24/2004
• Est. Priority Date: 12/18/2002
• Status: Active Grant

• Current Assignee: SAS Institute Incorporated
• Original Assignee: SAS Institute Incorporated
• Inventors: Rineer, Brian C., Jackson, Brent C., Fisher, Anthony F., Gidley, Scott A.
• Granted Patent: US 7,318,162 B2
• Time in Patent Office: Days
• US Class Current: 713/161
• CPC Class Codes: G06F 21/645 using a third party
• CPC Class Codes: H04L 63/123 received data contents, e.g. message integrity
• CPC Class Codes: Y10S 707/99942 Manipulating data structure, e.g. compression, compaction, compilation
• CPC Class Codes: Y10S 707/99953 Recoverability

Abstract

A computer-implemented system and method for customizing rules used in data integrity validation operations. A data integrity validation application uses the data integrity validation rules to perform the data integrity validation operations. The data integrity validation rules are stored in a knowledge base which is separate from and external to the data integrity validation application. The separately stored and externally located knowledge base allows customization of the data integrity validation rules without requiring recompilation of the data integrity validation application.

Claims

1. A computer-implemented system for customizing rules used in data integrity validation operations, comprising:

a data integrity validation application that accesses the data integrity validation rules to perform the data integrity validation operations,

wherein the data integrity validation application is stored in a low level compiled language;

a knowledge base containing the data integrity validation rules, wherein the knowledge base has a storage location that is separate from and external to the compiled data integrity validation application,

wherein the separate and external storage location of the knowledge base allows customization of the contained data integrity validation rules without requiring recompilation of the data integrity validation application,

wherein the data integrity validation application has a data access pathway to the separately stored and externally located knowledge base so that the data integrity validation application may perform the data integrity validation operations.

2. The system of claim 1 wherein the compiled data integrity validation application contains an engine that is capable of accessing the data integrity validation rules in the knowledge base and applying the rules to perform the data integrity validation operations.

3. The system of claim 1 wherein the low level compiled language is substantially unreadable to a human.

4. The system of claim 1 wherein the data integrity validation application comprises a plurality of software modules, wherein at least one of the software modules performs the data integrity validation operations.

5. The system of claim 1 wherein the separate and external storage location of the knowledge base relative to the compiled data integrity validation application allows the data integrity validation rules to be customized without requiring recompilation and redistribution of the document management application.

6. The system of claim 1 wherein the knowledge base comprises a database to store the data integrity validation rules.

7. The system of claim 1 wherein the knowledge base comprises a set of flat files to store the data integrity validation rules.

8. The system of claim 1 wherein the knowledge base is extensible in that existing data integrity validation rules may be added or updated.

9. The system of claim 1 wherein the data integrity validation rules are directed to cleansing number-based data.

10. The system of claim 1 wherein the data integrity validation rules are directed to cleansing text-based data.

11. The system of claim 1 wherein the data integrity validation rules comprise semantic cleansing rules.

12. The system of claim 1 wherein the data integrity validation rules are directed to analysis-related data cleansing operations.

13. The system of claim 1 wherein the data integrity validation rules are directed to standardization-related data cleansing operations.

14. The system of claim 1 wherein the data integrity validation rules are directed to augmentation-related data cleansing operations.

15. The system of claim 1 wherein the data integrity validation rules are directed to integration-related data cleansing operations.

16. The system of claim 1 wherein the data integrity validation rules in the knowledge base contain analysis reference materials and transformation specifications.

17. The system of claim 1 wherein a first computer server contains the data integrity validation application and a second computer server contains the knowledge base, wherein a network provides data access between the first and second computer servers.

18. The system of claim 17 wherein the network is an Internet network.

19. The system of claim 17 wherein a web browser located on a remote client computer accesses the data integrity validation rules stored in the knowledge base in order to customize the data integrity validation rules.

20. The system of claim 1 further comprising:

customization tools that access the data integrity validation rules in the knowledge base and contains at least one user interface to customize the rules.

21. The system of claim 20 wherein the customization tools contains at least one user interface related to testing the data integrity validation rules stored in the knowledge base.

22. The system of claim 20 wherein the customization tools contains at least one user interface related to debugging the data integrity validation rules stored in the knowledge base.

23. The system of claim 22 wherein the user interface is used to view intermediate results of the data integrity validation operations.

24. The system of claim 1 further comprising:

an import and export software facility for transferring data integrity validation rules from the knowledge base to another knowledge base for use in data integrity validation operations.

25. The system of claim 1 wherein the data integrity validation rules comprise data cleansing rules.

26. The system of claim 1 wherein the data integrity validation rules comprise data integration rules.

27. A computer-implemented method for managing rules used in data integrity validation operations, comprising the steps of:

retrieving from a knowledge base a data integrity validation rule;

modifying the retrieved data integrity validation rule;

storing the modified data integrity validation rule so that the modified data integrity rule may be used in a data integrity validation operation to be performed by a data integrity validation application, wherein the data integrity validation application is stored in a low level compiled language,

wherein the knowledge base has a storage location that is separate from and external to the compiled data integrity validation application,

wherein the separate and external storage location of the knowledge base allows modification of the stored data integrity validation rules without requiring recompilation of the data integrity validation application.

28. A computer-implemented apparatus for managing rules used in data integrity validation operations, comprising:

means for retrieving from a knowledge base a data integrity validation rule;

means for modifying the retrieved data integrity validation rule;

means for storing the modified data integrity validation rule so that the modified data integrity rule may be used in a data integrity validation operation to be performed by a data integrity validation application, wherein the data integrity validation application is stored in a low level compiled language,

wherein the knowledge base has a storage location that is separate from and external to the compiled data integrity validation application,

wherein the separate and external storage location of the knowledge base allows modification of the stored data integrity validation rules without requiring recompilation of the data integrity validation application.