Apparatus and method for dynamically auditing data migration to produce metadata

• US 7,725,728 B2
• Filed: 03/23/2005
• Issued: 05/25/2010
• Est. Priority Date: 03/23/2005
• Status: Active Grant

• Current Assignee: Business Objects Data Integration, Inc.
• Original Assignee: Business Objects Data Integration, Inc.
• Inventors: Todd, Mark, Chawla, Sachinder S., Ama, Ronaldo, Yee, Mon For, Gadhiraju, Balaji, Syed, Awez
• Primary Examiner(s): CERVETTI, DAVID GARCIA
• Application Number:
  US11/088,520
• Publication Number:
  US 20060218405A1
• Time in Patent Office: 1,889 Days
• Field of Search: 713/181, 726/20, 726/22, 726/23, 726/25, 726/34, 726/36
• US Class Current: 713/181
• CPC Class Codes: G06F 16/214 Database migration support

Abstract

A computer readable medium includes executable instructions to audit data migration. The executable instructions include instructions to define a data migration path from a source to a target. Audit points are specified within the data migration path. Audit statistics to be generated at the audit points are identified. The audit statistics including data corresponding to individual data values transmitted over the data migration path. Audit rules to be applied against the audit statistics are established.

Claims

1. A computer readable medium including executable instructions to audit data migration, comprising executable instructions to:

define a data migration path from a source to a target;

specify audit points within said data migration path;

identify audit statistics to be generated at said audit points, said audit statistics including data corresponding to individual data values transmitted over said data migration path;

generate audit statistics at said audit points by collecting aggregate information for values of transported data while said data is transported, said audit statistics forming an embedded data flow separate from said data migration path; and

establish audit rules to be applied against said audit statistics from said embedded data flow.

2. The computer readable medium of claim 1 wherein said executable instructions to specify audit points include executable instructions to specify audit points at data extraction, transform and load positions.

3. The computer readable medium of claim 1 wherein said executable instructions to specify audit points include executable instructions to allow a user to specify audit points.

4. The computer readable medium of claim 1 wherein said executable instructions to specify audit points include executable instructions to automatically specify audit points.

5. The computer readable medium of claim 1 wherein said executable instructions to identify audit statistics include executable instructions to present selectable audit statistics selected from COUNT, SUM, AVERAGE, CHECK SUM, and ERROR COUNT.

6. The computer readable medium of claim 1 wherein said executable instructions to identify audit statistics include executable instructions to automatically specify audit statistics.

7. The computer readable medium of claim 1 wherein said executable instructions to identify audit statistics include executable instructions to identify metadata audit statistics embedded with data from said source and delivered to said target.

8. The computer readable medium of claim 1 wherein said executable instructions to establish audit rules include executable instructions to specify an audit rule applied to data flow.

9. The computer readable medium of claim 1 wherein said executable instructions to establish audit rules include executable instructions to specify an audit rule applied to an audit point.

10. The computer readable medium of claim 1 wherein said executable instructions to establish audit rules include executable instructions to specify an email alert rule.

11. The computer readable medium of claim 1 wherein said executable instructions to establish audit rules include executable instructions to identify a process failure that raises an exception.

12. The computer readable medium of claim 1 wherein said executable instructions to establish audit rules include executable instructions to specify an executable script.

13. The computer readable medium of claim 1 further comprising executable instructions to initiate data migration on demand.

14. The computer readable medium of claim 1 further comprising executable instructions to initiate data migration in response to a scheduled event.

15. A computer readable medium including executable instructions to audit data migration, comprising executable instructions to:

identify audit points within a data migration path;

display the audit points to a user;

establish specified audit statistics for said audit points;

accept user input for audit rules applied to said audit statistics;

process data within said data migration path to produce an embedded data flow separate from said data migration path;

collect from said embedded data flow dynamic audit statistics corresponding to said specified audit statistics, wherein said dynamic audit statistics include aggregate information for values of transported data while said data is transported;

save the audit statistics as metadata;

optionally apply audit rules to said audit statistics; and

save the results of said audit rules as metadata.

16. The computer readable medium of claim 15 wherein said executable instructions include executable instructions to generate an email alert, to throw an exception in response to a process failure, and to execute a custom script.

17. The computer readable medium of claim 15 further comprising executable instructions to apply audit rules to a data flow and an audit point.

18. The computer readable medium of claim 15 further comprising executable instructions to generate audit statistics selected from COUNT, SUM, AVERAGE, CHECKSUM, and ERROR COUNT operations.