Referential Integrity, Consistency, and Completeness Loading of Databases

• US 20100114841A1
• Filed: 10/30/2009
• Published: 05/06/2010
• Est. Priority Date: 10/31/2008
• Status: Active Grant

• Current Assignee: Intel Corporation
• Original Assignee: Gravic Incorporated
• Inventors: Holenstein, Bruce D., Holenstein, Paul J., Highleyman, Wilbur H.
• Granted Patent: US 7,949,640 B2
• Time in Patent Office: Days
• US Class Current: 707/690
• CPC Class Codes: G06F 16/217 Database tuning G06F16/2282 takes precedence; database performance monitoring G06F11/3409

Abstract

A method is provided for loading data from a source database to a target database that includes at least one table. Prior to loading the data from the source database into the target database, at least one referential integrity constraint and/or at least one consistency requirement regarding the data is automatically identified. A subset of the data that satisfies the at least one referential integrity constraint and/or consistency requirement is then automatically identified. The identified subset of the data is then loaded into the target database as a unit of work.

Claims

1. A method for loading data from a source database to a target database that includes at least one table, comprising:

(a) prior to loading the data from the source database into the target database, automatically identifying at least one referential integrity constraint regarding the data;

(b) automatically identifying a first subset of the data that satisfies the at least one referential integrity constraint; and

(c) loading the identified first subset of the data into the target database as a unit of work.

2. The method of claim 1 wherein step (a) further comprises prior to loading the data from the source database into the target database, automatically identifying consistency requirements regarding the data, and step (b) further comprises automatically identifying a second subset of the data that satisfies the identified consistency requirements, wherein the second subset of the data includes the first subset of the data, and step (c) further comprises loading the identified second subset of the data into the target database as a unit of work.

3. The method of claim 2 wherein step (a) further comprises prior to loading the data from the source database into the target database, automatically identifying completeness requirements regarding the data, and step (b) further comprises automatically identifying a third subset of the data that satisfies the identified completeness requirements, wherein the third subset of the data includes the second subset of the data, and step (c) further comprises loading the identified third subset of the data into the target database as a unit of work.

4. The method of claim 3 wherein the unit of work is at least one of atomic, consistent, isolated and durable.

5. The method of claim 3 wherein the target database includes a plurality of tables, and the referential integrity constraint regarding the data is between multiple tables.

6. The method of claim 3 wherein the target database is usable during the data loading

7. The method of claim 3 wherein the target database is online during the data loading.

8. The method of claim 3 further comprising:

(d) repeating steps (b) and (c) for other third subsets of the data.

9. The method of claim 2 wherein step (a) further comprises prior to loading the data from the source database into the target database, automatically identifying relaxed completeness requirements regarding the data, and step (b) further comprises automatically identifying a third subset of the data that satisfies the identified relaxed completeness requirements, wherein the third subset of the data includes the second subset of the data, and step (c) further comprises loading the identified third subset of the data into the target database as a unit of work.

10. The method of claim 9 wherein the unit of work is at least one of atomic, consistent, isolated and durable.

11. The method of claim 9 wherein the target database includes a plurality of tables, and the referential integrity constraint regarding the data is between multiple tables.

12. The method of claim 9 wherein the target database is usable during the data loading

13. The method of claim 9 wherein the target database is online during the data loading.

14. The method of claim 9 further comprising:

(d) repeating steps (b) and (c) for other third subsets of the data.

15. The method of claim 2 further comprising:

(d) repeating steps (b) and (c) for other second subsets of the data.

16. The method of claim 2 wherein the unit of work is at least one of atomic, consistent, isolated and durable.

17. The method of claim 2 wherein the target database includes a plurality of tables, and the referential integrity constraint regarding the data is between multiple tables.

18. The method of claim 2 wherein the target database is usable during the data loading

19. The method of claim 2 wherein the target database is online during the data loading.

20. The method of claim 1 further comprising:

(d) repeating steps (b) and (c) for other first subsets of the data.

21. The method of claim 1 wherein the unit of work is a database transaction.

22. The method of claim 1 wherein the source database is online during the data loading.

23. The method of claim 1 wherein the data loading is performed by an online extract/transform/load (ETL) operation.

24. The method of claim 1 wherein the data loading is performed by online loading with concurrent change replication.

25. The method of claim 1 wherein the data loading is performed by online loading with integrated change replication.

26. The method of claim 1 wherein the unit of work is at least one of atomic, consistent, isolated and durable.

27. The method of claim 1 wherein the target database includes a plurality of tables, and the referential integrity constraint regarding the data is between multiple tables.

28. The method of claim 1 wherein the target database is usable during the data loading

29. The method of claim 1 wherein the target database is online during the data loading.

30. A method for loading data from a source database to a target database that includes at least one table, comprising:

(a) prior to loading the data from the source database into the target database, automatically identifying at least one consistency requirement regarding the data;

(b) automatically identifying a first subset of the data that satisfies the at least one consistency requirement; and

(c) loading the identified first subset of the data into the target database as a unit of work.

31. The method of claim 30 wherein step (a) further comprises prior to loading the data from the source database into the target database, automatically identifying completeness requirements regarding the data, and step (b) further comprises automatically identifying a second subset of the data that satisfies the identified completeness requirements, wherein the second subset of the data includes the first subset of the data, and step (c) further comprises loading the identified second subset of the data into the target database as a unit of work.

32. The method of claim 31 further comprising:

(d) repeating steps (b) and (c) for other second subsets of the data.

33. The method of claim 30 wherein step (a) further comprises prior to loading the data from the source database into the target database, automatically identifying relaxed completeness requirements regarding the data, and step (b) further comprises automatically identifying a second subset of the data that satisfies the identified relaxed completeness requirements, wherein the second subset of the data includes the first subset of the data, and step (c) further comprises loading the identified second subset of the data into the target database as a unit of work.

34. The method of claim 33 further comprising:

(d) repeating steps (b) and (c) for other second subsets of the data.

35. The method of claim 30 further comprising:

(d) repeating steps (b) and (c) for other first subsets of the data.

36. The method of claim 30 wherein the unit of work is a database transaction.

37. The method of claim 30 wherein the unit of work is at least one of atomic, consistent, isolated and durable.

38. The method of claim 30 wherein the target database includes a plurality of tables, and the at least one consistency requirement regarding the data is between multiple tables.

39. The method of claim 30 wherein the target database is usable during the data loading

40. The method of claim 30 wherein the target database is online during the data loading.

41. The method of claim 30 wherein the source database is online during the data loading.

42. The method of claim 30 wherein the data loading is performed by an online extract/transform/load (ETL) operation.

43. The method of claim 30 wherein the data loading is performed by online loading with concurrent change replication.

44. The method of claim 30 wherein the data loading is performed by online loading with integrated change replication.