Enterprise data movement system and method which performs data load and changed data propagation operations

• US 6,016,501 A
• Filed: 03/30/1998
• Issued: 01/18/2000
• Est. Priority Date: 03/18/1998
• Status: Expired due to Term

• Current Assignee: Informatica LLC (Informatica, Inc. (California))
• Original Assignee: BMC Software Incorporated (KKR & Co., Inc.)
• Inventors: Sirjani, Abolfazl, Martin, James L., Keeler, Lisa S., Seppi, Kevin D.
• Primary Examiner(s): Lintz, Paul R.
• Assistant Examiner(s): MIZRAHI, DIANE D
• Application Number:
  US09/050,609
• Time in Patent Office: 659 Days
• Field of Search: 707/6, 707/8, 707/9, 707/100, 707/101, 707/200, 707/201, 707/202, 707/203, 379/114, 379/28
• US Class Current: 1/1
• CPC Class Codes: G06F 16/275 Synchronous replication
• CPC Class Codes: Y10S 707/99936 Pattern matching access
• CPC Class Codes: Y10S 707/99938 Concurrency, e.g. lock management in shared database
• CPC Class Codes: Y10S 707/99939 Privileged access
• CPC Class Codes: Y10S 707/99942 Manipulating data structure, e.g. compression, compaction, compilation
• CPC Class Codes: Y10S 707/99952 Coherency, e.g. same view to multiple users
• CPC Class Codes: Y10S 707/99953 Recoverability
• CPC Class Codes: Y10S 707/99954 Version management

Abstract

An Enterprise Data Movement (EDM) system and method for performing bulk data move operations and changed data propagation operations in an enterprise computing system with improved efficiency. The present invention provides synergy of operations between a bulk data move operation and a changed data propagation, wherein the load process of a bulk data move operation and the apply process of a changed data propagation operation operate to communicate with each other to ensure proper operation and also to ensure that each of the bulk move and propagation operations do not interfere with each other. Thus, the present invention provides an improved and automated procedure for performing bulk move and changed data propagation operations which reduces the amount of time required by database administrators to monitor and/or perform tasks.

Claims

1. A method for performing data load and changed data propagation operations in an enterprise computing system, wherein the enterprise computing system comprises a plurality of computer systems and includes a source database and a target database, wherein the enterprise computing system also includes a changed data log for storing changed data of the source database, the method comprising:

initiating a load process to load data onto the target database;

determining the status of an apply process of a changed data propagation task;

providing information to the apply process regarding the load process;

the apply process suspending apply processing in response to said providing if the apply process is active;

the load process operating to load data into at least a portion of the target database;

updating control information of the apply process, wherein said updating includes providing information regarding a restart point in the changed data log;

restarting the changed data propagation task after said loading and said updating control information;

repositioning the changed data propagation task to a correct location within the changed data log in response to said updated control information.

2. The method of claim 1, wherein the apply process suspends apply processing prior to the load process operating to load said data into the at least a portion of the target database.

3. The method of claim 1, wherein said providing information to the apply process regarding the load process is performed if the apply process is active.

4. The method of claim 1, wherein said determining the status of an apply process of a changed data propagation task comprises:

identifying an apply process responsible for applying changes to the at least a portion of the target database being loaded by the load process.

5. The method of claim 1, wherein the apply process suspends apply processing if the apply process is active and suspension is necessary.

6. The method of claim 1, wherein said providing information to the apply process regarding the load process includes providing information regarding the at least a portion of the target database being loaded.

7. The method of claim 6, wherein the apply process suspends apply processing if the apply process is active and the apply process is operating on the at least a portion of the target database being loaded.

8. The method of claim 1, further comprising

the apply process notifying the load process that the load process can proceed with the load of the at least a portion of the target database prior to the load process operating.

9. The method of claim 8, further comprising

determining that the at least a portion of the target database is no longer subject to update by the apply process prior to the load process operating.

10. The method of claim 1, further comprising:

initializing the load process at the target computer system prior to the load process operating to load said data into the at least a portion of the target database.

11. The method of claim 1, further comprising:

reinitializing the target database to an empty status prior to the load process operating to load said data into the at least a portion of the target database.

12. The method of claim 1, further comprising:

notifying the apply process that the at least a portion of the target database has been loaded if the apply process was active;

wherein the changed data propagation task restarts in response to said notifying.

13. The method of claim 1, wherein the enterprise computing system includes a source computer system, wherein the source computer system includes the source database;

wherein said restarting the changed data propagation task comprises;

the apply process verifying that no other portion of the target database is currently suspended for load; and

the apply process performing an automatic restart with a partner log read processor task executing in the source computer system.

14. The method of claim 1, wherein the apply process was operating at a first location in the changed data log prior to the load operation operating;

wherein the changed data propagation task operates to reposition to said first location within the changed data log when the changed data propagation task restarts.

15. The method of claim 1, wherein the changed data propagation task operates to reposition to a new position in the changed data log to retrieve changes captured for the at least a portion of the target database.

16. The method of claim 1, further comprising:

notifying a utility that the target database has been reloaded if the apply process was not active;

wherein a customer-specified action is performed in response to said notifying.

17. The method of claim 1, further comprising:

initiating a bulk data move task from the source database to the target database;

performing the bulk data move task, wherein said performing the bulk data move task includes initiating said load process to load data onto the target database.

18. The method of claim 17, wherein said load process performs said:

determining the status of an apply process of a changed data propagation task;

providing information to the apply process regarding the load process;

performing the load process to load data into at least a portion of the target database; and

updating control information of the apply process, wherein said updating includes providing information regarding a restart point in a changed data log.

19. The method of claim 1, further comprising:

performing a recovery operation to recover data onto the target database, wherein said performing the recovery operation includes initiating said load process to load data onto the target database.

20. The method of claim 19, wherein said load process performs said:

determining the status of an apply process of a changed data propagation task;

providing information to the apply process regarding the load process;

performing the load process to load data into at least a portion of the target database; and

updating control information of the apply process, wherein said updating includes providing information regarding a restart point in a changed data log.

21. A method for performing data move and changed data propagation operations in an enterprise computing system, wherein the enterprise computing system comprises a plurality of computer systems and includes a source database and a target database, wherein the enterprise computing system also includes a changed data log for storing changed data of the source database, the method comprising:

initiating a bulk data move task from the source database to the target database;

performing the bulk data move task, wherein said performing the bulk data move task comprises;

determining the status of an apply process of a changed data propagation task;

providing information to the apply process regarding the bulk data move task;

wherein the apply process suspends apply processing in response to said providing if the apply process is active;

loading data into at least a portion of the target database;

updating control information of the apply process, wherein said updating includes providing information regarding a restart point in a changed data log;

restarting the changed data propagation task after said loading and said updating control information;

repositioning the changed data propagation task to a correct location within the changed data log in response to said updated control information.

22. The method of claim 21, wherein the apply process suspends apply processing prior to said loading said data into the at least a portion of the target database.

23. The method of claim 21, wherein said providing information to the apply process regarding the bulk data move task is performed if the apply process is active.

24. The method of claim 21, wherein said determining the status of an apply process of a changed data propagation task comprises:

identifying an apply process responsible for applying changes to the at least a portion of the target database being loaded by the bulk data move task.

25. The method of claim 21, wherein said providing information to the apply process regarding the bulk data move task includes providing information regarding the at least a portion of the target database being loaded;

wherein the apply process suspends apply processing if the apply process is active and the apply process is operating on the at least a portion of the target database being loaded.

26. The method of claim 21, further comprising

the apply process notifying the bulk data move task that the bulk data move task can proceed with the load of the at least a portion of the target database prior to said loading said data into the at least a portion of the target database.

27. The method of claim 26, further comprising

determining that the target database is no longer subject to update by the apply process prior to said loading said data into the at least a portion of the target database.

28. The method of claim 21, wherein said performing the bulk data move task further includes:

notifying the apply process that the at least a portion of the target database has been loaded if the apply process was active during the bulk data move task;

wherein the changed data propagation task restarts in response to said notifying.

29. The method of claim 21, wherein the enterprise computing system includes a source computer system, wherein the source computer system includes the source database;

wherein said restarting the changed data propagation task comprises;

the apply process verifying that no other portion of the target database is currently suspended for load; and

the apply process performing an automatic restart with a partner log read processor task executing in the source computer system.

30. The method of claim 21, wherein the apply process was operating at a first location in the changed data log prior to said loading;

wherein the changed data propagation task operates to reposition to said first location within the changed data log when the changed data propagation task restarts.

31. The method of claim 21, wherein the changed data propagation task operates to reposition to a new position in the changed data log to retrieve changes captured for the at least a portion of the target database.

32. The method of claim 21, wherein said performing the bulk data move task includes performing a transformation operation to transform data from a source data format to a target data format.

33. The method of claim 21, wherein said performing the bulk data move task includes moving the source data from the source database to the target database.

34. The method of claim 21, wherein said performing the bulk data move task further includes:

performing a transformation operation to transform data from a source data format to a target data format; and

moving the source data from the source database to the target database.

35. The method of claim 21, wherein the enterprise computing system includes a source computer system and a target computer system, wherein the source computer system includes the source database and wherein the target computer system includes the target database.

36. A method for performing data load and changed data propagation operations in an enterprise computing system, wherein the enterprise computing system comprises a plurality of computer systems and includes a source database and a target database, wherein the enterprise computing system also includes a changed data log for storing changed data of the source database, the method comprising:

initiating a load process to load data onto the target database;

performing the load process, wherein said performing the load process comprises;

determining the status of an apply process of a changed data propagation task;

providing information to the apply process regarding the load process;

wherein the apply process suspends apply processing in response to said providing if the apply process is active;

loading data into at least a portion of the target database;

updating control information of the apply process, wherein said updating includes providing information regarding a restart point in the changed data log;

restarting the changed data propagation task after said loading and said updating control information;

repositioning the changed data propagation task to a correct location within the changed data log in response to said updated control information.

37. A memory media which stores program instructions for performing data load and changed data propagation operations in an enterprise computing system, wherein the enterprise computing system comprises a plurality of computer systems and includes a source database and a target database, wherein the enterprise computing system also includes a changed data log for storing changed data of the source database, wherein, in response to a load process being initiated to load data onto the target database, the program instructions are executable to implement:

determining the status of an apply process of a changed data propagation task;

providing information to the apply process regarding the load process;

the apply process suspending apply processing in response to said providing if the apply process is active;

the load process operating to load data into at least a portion of the target database;

updating control information of the apply process, wherein said updating includes providing information regarding a restart point in the changed data log;

restarting the changed data propagation task after said loading and said updating control information;

repositioning the changed data propagation task to a correct location within the changed data log in response to said updated control information.

38. The memory media of claim 37, wherein the apply process suspends apply processing prior to the load process operating to load said data into the at least a portion of the target database.

39. The memory media of claim 37, wherein said determining the status of an apply process of a changed data propagation task comprises:

identifying an apply process responsible for applying changes to the at least a portion of the target database being loaded by the load process.

40. The memory media of claim 37, wherein said providing information to the apply process regarding the load process includes providing information regarding the at least a portion of the target database being loaded;

wherein the apply process suspends apply processing if the apply process is active and the apply process is operating on the at least a portion of the target database being loaded.

41. The memory media of claim 37, wherein said program instructions further implement:

the apply process notifying the load process that the load process can proceed with the load of the at least a portion of the target database prior to the load process operating; and

determining that the at least a portion of the target database is no longer subject to update by the apply process prior to the load process operating.

42. The memory media of claim 37, wherein said program instructions further implement:

notifying the apply process that the at least a portion of the target database has been loaded if the apply process was active;

wherein the changed data propagation task restarts in response to said notifying.

43. The memory media of claim 37, wherein the enterprise computing system includes a source computer system, wherein the source computer system includes the source database;

wherein said restarting the changed data propagation task comprises;

the apply process verifying that no other portion of the target database is currently suspended for load; and

the apply process performing an automatic restart with a partner log read processor task executing in the source computer system.

44. The memory media of claim 37, wherein the apply process was operating at a first location in the changed data log prior to the load operation operating;

wherein the changed data propagation task operates to reposition to said first location within the changed data log when the changed data propagation task restarts.

45. The memory media of claim 37, wherein the changed data propagation task operates to reposition to a new position in the changed data log to retrieve changes captured for the at least a portion of the target database.

46. The memory media of claim 37, wherein the program instructions further implement:

initiating a bulk data move task from the source database to the target database;

performing the bulk data move task, wherein said performing the bulk data move task includes initiating said load process to load data onto the target database.

47. The memory media of claim 37, wherein the program instructions further implement:

performing a recovery operation to recover data onto the target database, wherein said performing the recovery operation includes initiating said load process to load data onto the target database.

48. An enterprise computing system which performs data load and changed data propagation operations, wherein the enterprise computing system comprises:

a plurality of computer systems, wherein the plurality of computer systems includes a source computer system including a source database and a target computer system including a target database;

a changed data log for storing changed data of the source database;

a load process comprised in one or more of the source computer system or the target computer system which is operable to load data into at least a portion of the target database;

an apply process comprised in one or more of the source computer system or the target computer system, wherein the apply process is a part of a changed data propagation task;

a program for determining the status of the apply process, wherein the program is operable to provide information to the apply process regarding the load process;

wherein the apply process is operable to suspend apply processing in response to said information if the apply process is active;

wherein the program is operable to update control information of the apply process, including providing information regarding a restart point in the changed data log;

wherein the changed data propagation task restarts after the load process loads said data and the program updates said control information, wherein the changed data propagation task repositions to a correct location within the changed data log in response to said updated control information.