Enterprise data movement system and method including opportunistic performance of utilities and data move operations for improved efficiency

• US 6,035,307 A
• Filed: 03/30/1998
• Issued: 03/07/2000
• Est. Priority Date: 03/30/1998
• Status: Expired due to Term

• Current Assignee: Informatica LLC (Informatica, Inc. (California))
• Original Assignee: BMC Software Incorporated (KKR & Co., Inc.)
• Inventors: Sirjani, Abolfazl, Martin, James L., Seppi, Kevin D., Keeler, Lisa S.
• Primary Examiner(s): Black, Thomas G.
• Assistant Examiner(s): Mizrahi, Diane D.
• Application Number:
  US09/050,422
• Time in Patent Office: 708 Days
• Field of Search: 707/6, 707/8, 707/9, 707/101, 707/201, 707/202, 707/203
• US Class Current: 1/1
• CPC Class Codes: G06F 16/258 Data format conversion from or to a database
• CPC Class Codes: Y10S 707/99936 Pattern matching access
• CPC Class Codes: Y10S 707/99938 Concurrency, e.g. lock management in shared database
• CPC Class Codes: Y10S 707/99939 Privileged access
• CPC Class Codes: Y10S 707/99942 Manipulating data structure, e.g. compression, compaction, compilation
• CPC Class Codes: Y10S 707/99952 Coherency, e.g. same view to multiple users
• CPC Class Codes: Y10S 707/99953 Recoverability
• CPC Class Codes: Y10S 707/99954 Version management

Abstract

An Enterprise Data Movement (EDM) system and method for performing data move operations in an enterprise computing system with improved efficiency. Data move operations preferably use operations of an existing database utility, for example, one or more of an unload or load operation, wherein the utility is already required to be performed. In other words, according to the invention, data move operations preferably use the utilities in an opportunistic or parasitic fashion, when they are already being performed for other reasons, thereby providing a dual purpose operation and increased efficiency.

Claims

1. A method for performing data move operations in an enterprise computing system, wherein the enterprise computing system comprises a plurality of computer systems and includes a source database, wherein the data move operations are performed with improved efficiency, the method comprising:

initializing an extract utility to begin an extract operation to extract data from the source database, wherein the extract utility is initialized to perform a first database operation;

determining data move requirements within the enterprise computing system, wherein said determining determines that at least one data move task requires the data from the source database;

the extract utility performing the extract operation to extract the data;

providing the data to at least one data move process in response to performing the extract operation, wherein the at least one data move process corresponds to the at least one data move task, wherein said providing is performed in response to determining that at least one data move task requires the data from the source database;

the data move process moving the data received from said providing.

2. The method of claim 1, wherein the enterprise computing system includes a target database;

wherein the at least one data move task is required to move the data from the source database to the target database;

wherein said moving comprises the data move process moving the data to the target database.

3. The method of claim 1, wherein said determining data move requirements comprises:

providing information to an EDM process about the extract operation and the data which is the subject of the extract operation;

the EDM process determining if the data is desired for a data move operation using the information;

configuring the extract utility to provide the data to a specified destination in response to determining that the data is desired for a data move operation.

4. The method of claim 3, wherein the EDM process determining if the data is desired for a data move operation includes querying a repository for task profiles to determine data move requirements of the at least one data move task.

5. The method of claim 4,

wherein said determining data move requirements includes examining required transformation information in the repository regarding the at least one data move task;

the method further comprising;

creating a transform process in response to said required transformation information, wherein said creating is performed in response to determining that the at least one data move task requires the data from the source database and in response to examining the required transformation information in the repository regarding the at least one data move task.

6. The method of claim 1, further comprising:

initializing the at least one data move process in response to determining that said at least one data move process requires the data, wherein the at least one data move process is initialized prior to said providing the data, wherein the at least one data move process corresponds to the at least one data move task.

7. The method of claim 1, wherein said determining the data move requirements is performed in response to said initializing the extract utility.

8. The method of claim 1,

wherein the extract utility executes on a first computer system;

wherein the at least one data move process executes on the first computer system.

9. The method of claim 1, wherein the data move task requires movement of the data from the source database to a target database;

wherein the source database resides on a first computer system and wherein the target database resides on a second computer system;

wherein said moving the data includes;

establishing a connection to a receive process in the second computer system.

10. The method of claim 9, wherein said moving the data includes:

establishing a connection to a remote server process in the second computer system;

providing information to the remote server process about which tasks desire or require the data;

the remote server process starting the at least one data move process in the second computer system in response to said providing.

11. The method of claim 10, wherein said moving the data further includes:

the remote server process receiving information regarding the data move;

the method further comprising;

creating a transform process on the remote side in response to the remote server process receiving information regarding the data move.

12. The method of claim 1, wherein the data move task requires movement of the data from the source database to a target database;

wherein the source database and the target database both reside on a first computer system.

13. The method of claim 1, wherein said data move task is unrelated to said first database operation.

14. The method of claim 1, wherein said data move task is related to said first database operation.

15. The method of claim 1, further comprising:

configuring the data move task to take advantage of existing utility operations in an opportunistic fashion prior to said determining data move requirements;

wherein said determining data move requirements is performed in response to said configuring the data move task to take advantage of existing utility operations in an opportunistic fashion.

16. The method of claim 1, wherein the data move process takes advantage of the extract utility performing the extract operation in an opportunistic fashion.

17. The method of claim 1, wherein the extract utility comprises an unload utility.

18. The method of claim 1, wherein the extract utility comprises a copy utility.

19. A method for performing data move operations in an enterprise computing system, wherein the enterprise computing system comprises a plurality of distributed computer systems and includes a source database and a target database, wherein the data move operations are performed with improved efficiency, the method comprising:

initializing a load utility to begin a load operation to load data onto a source database, wherein the load utility is initialized to perform a first database operation;

determining data move requirements within the enterprise computing system, wherein said determining determines that at least one data move task requires the data being loaded onto the source database, wherein the at least one data move task is required to move the data from the source database to the target database;

the load utility performing the load operation to load the data onto the source database;

providing the data to at least one data move process in response to performing the load operation, wherein the at least one data move process corresponds to the at least one data move task, wherein said providing is performed in response to determining that at least one data move task requires the data being loaded onto the source database; and

the data move process moving the data received from said providing to the target database.

20. The method of claim 19, wherein said determining data move requirements comprises:

providing information to an EDM process about the load operation and the data which is the subject of the load operation;

the EDM process determining if the data is desired for a data move operation using the information;

configuring the load utility to provide the data to a specified destination in response to determining that the data is desired for a data move operation.

21. The method of claim 20, wherein the EDM process determining if the data is desired for a data move operation includes querying a repository for task profiles to determine data move requirements of the at least one data move task.

22. The method of claim 21,

wherein said determining data move requirements includes examining required transformation information in the repository regarding the at least one data move task;

the method further comprising;

creating a transform process in response to said required transformation information, wherein said creating is performed in response to determining that the at least one data move task requires the data from the source database and in response to examining the required transformation information in the repository regarding the at least one data move task.

23. The method of claim 19, further comprising:

initializing the at least one data move process in response to determining that said at least one data move process requires the data, wherein the at least one data move process is initialized prior to said providing the data, wherein the at least one data move process corresponds to the at least one data move task.

24. The method of claim 19, wherein said determining the data move requirements is performed in response to said initializing the load utility.

25. The method of claim 19,

wherein the load utility executes on a first computer system;

wherein the at least one data move process executes on the first computer system.

26. The method of claim 19, wherein the data move task requires movement of the data from the source database to a target database;

wherein the source database resides on a first computer system and wherein the target database resides on a second computer system;

wherein said moving the data includes;

establishing a connection to a receive process in the second computer system.

27. The method of claim 19, wherein said moving the data comprises:

establishing a connection to a remote server process in the second computer system;

providing information the remote server process regarding which tasks desire or require the data;

the remote server process starting the at least one data move process in the second computer system in response to said providing.

28. The method of claim 27,

the remote server process receiving information regarding the data move;

the method further comprising;

creating a transform process on the second computer system in response to the remote server process receiving information regarding the data move.

29. The method of claim 19, wherein said data move task is unrelated to said first database operation.

30. The method of claim 19, further comprising:

configuring the data move task to take advantage of existing utility operations in an opportunistic fashion prior to said determining data move requirements;

wherein said determining data move requirements is performed in response to said configuring the data move task to take advantage of existing utility operations in an opportunistic fashion.

31. The method of claim 19, wherein the data move process takes advantage of the load utility performing the load operation in an opportunistic fashion.

32. The method of claim 19, wherein the load utility comprises a reorganization utility.

33. A memory media which stores program instructions for performing data move operations in an enterprise computing system, wherein the enterprise computing system comprises a plurality of distributed computer systems and includes a source database, wherein the data move operations are performed with improved efficiency, wherein the program instructions implement:

initializing an extract utility to begin an extract operation to extract data from the source database, wherein the extract utility is initialized to perform a first database operation;

determining data move requirements within the enterprise computing system, wherein said determining determines that at least one data move task requires the data from the source database;

the extract utility performing the extract operation to extract the data;

providing the data to at least one data move process in response to performing the extract operation, wherein the at least one data move process corresponds to the at least one data move task, wherein said providing is performed in response to determining that at least one data move task requires the data from the source database;

the data move process moving the data received from said providing.

34. The memory media of claim 33, wherein the enterprise computing system includes a target database;

wherein the at least one data move task is required to move the data from the source database to the target database;

wherein said moving comprises the data move process moving the data to the target database.

35. The memory media of claim 33, wherein said determining data move requirements comprises:

providing information to an EDM process about the extract operation and the data which is the subject of the extract operation;

the EDM process determining if the data is desired for a data move operation using the information;

configuring the extract utility to provide the data to a specified destination in response to determining that the data is desired for a data move operation.

36. The memory media of claim 35, wherein the EDM process determining if the data is desired for a data move operation includes querying a repository for task profiles to determine data move requirements of the at least one data move task.

37. The memory media of claim 36,

wherein said determining data move requirements includes examining required transformation information in the repository regarding the at least one data move task;

wherein the program instructions further implement;

creating a transform process in response to said required transformation information, wherein said creating is performed in response to determining that the at least one data move task requires the data from the source database and in response to examining the required transformation information in the repository regarding the at least one data move task.

38. The memory media of claim 33, wherein the program instructions further implement:

initializing the at least one data move process in response to determining that said at least one data move process requires the data, wherein the at least one data move process is initialized prior to said providing the data, wherein the at least one data move process corresponds to the at least one data move task.

39. The memory media of claim 33, wherein said determining the data move requirements is performed in response to said initializing the extract utility.

40. The memory media of claim 33,

wherein the extract utility executes on a first computer system;

wherein the at least one data move process executes on the first computer system.

41. The memory media of claim 33, wherein the data move task requires movement of the data from the source database to a target database;

wherein the source database resides on a first computer system and wherein the target database resides on a second computer system;

wherein said moving the data includes;

establishing a connection to a receive process in the second computer system.

42. The memory media of claim 41, wherein said moving the data includes:

establishing a connection to a remote server process in the second computer system;

providing information to the remote server process about which tasks desire or require the data;

the remote server process starting the at least one data move process in the second computer system in response to said providing.

43. The memory media of claim 42, wherein said moving the data further includes:

the remote server process receiving information regarding the data move;

the program instructions further implementing;

creating a transform process on the remote side in response to the remote server process receiving information regarding the data move.

44. The memory media of claim 33, wherein the data move task requires movement of the data from the source database to a target database;

wherein the source database and the target database both reside on a first computer system.

45. The memory media of claim 33, wherein said data move task is unrelated to said first database operation.

46. The memory media of claim 33, wherein said data move task is related to said first database operation.

47. The memory media of claim 33, the program instructions further implementing:

configuring the data move task to take advantage of existing utility operations in an opportunistic fashion prior to said determining data move requirements;

wherein said determining data move requirements is performed in response to said configuring the data move task to take advantage of existing utility operations in an opportunistic fashion.

48. The memory media of claim 33, wherein the data move process takes advantage of the extract utility performing the extract operation in an opportunistic fashion.

49. The memory media of claim 33, wherein the extract utility comprises an unload utility.

50. The memory media of claim 33, wherein the extract utility comprises a copy utility.

51. A memory media which stores program instructions for performing data move operations in an enterprise computing system, wherein the enterprise computing system comprises a plurality of distributed computer systems and includes a source database and a target database, wherein the data move operations are performed with improved efficiency, wherein the program instructions implement:

initializing a load utility to begin a load operation to load data onto a source database, wherein the load utility is initialized to perform a first database operation;

determining data move requirements within the enterprise computing system, wherein said determining determines that at least one data move task requires the data being loaded onto the source database, wherein the at least one data move task is required to move the data from the source database to the target database;

the load utility performing the load operation to load the data onto the source database;

providing the data to at least one data move process in response to performing the load operation, wherein the at least one data move process corresponds to the at least one data move task, wherein said providing is performed in response to determining that at least one data move task requires the data being loaded onto the source database; and

the data move process moving the data received from said providing to the target database.

52. The memory media of claim 51, wherein said determining data move requirements wherein the program instructions implement:

providing information to an EDM process about the load operation and the data which is the subject of the load operation;

the EDM process determining if the data is desired for a data move operation using the information;

configuring the load utility to provide the data to a specified destination in response to determining that the data is desired for a data move operation.

53. The memory media of claim 52, wherein the EDM process determining if the data is desired for a data move operation includes querying a repository for task profiles to determine data move requirements of the at least one data move task.

54. The memory media of claim 53,

wherein said determining data move requirements includes examining required transformation information in the repository regarding the at least one data move task;

the program instructions further implementing;

creating a transform process in response to said required transformation information, wherein said creating is performed in response to determining that the at least one data move task requires the data from the source database and in response to examining the required transformation information in the repository regarding the at least one data move task.

55. The memory media of claim 51, wherein the program instructions further implement:

initializing the at least one data move process in response to determining that said at least one data move process requires the data, wherein the at least one data move process is initialized prior to said providing the data, wherein the at least one data move process corresponds to the at least one data move task.

56. The memory media of claim 51, wherein said determining the data move requirements is performed in response to said initializing the load utility.

57. The memory media of claim 51,

wherein the load utility executes on a first computer system;

wherein the at least one data move process executes on the first computer system.

58. The memory media of claim 51, wherein the data move task requires movement of the data from the source database to a target database;

wherein the source database resides on a first computer system and wherein the target database resides on a second computer system;

wherein said moving the data includes;

establishing a connection to a receive process in the second computer system.

59. The memory media of claim 51, wherein said moving the data comprises:

establishing a connection to a remote server process in the second computer system;

providing information the remote server process regarding which tasks desire or require the data;

the remote server process starting the at least one data move process in the second computer system in response to said providing.

60. The memory media of claim 59,

the remote server process receiving information regarding the data move;

the program instructions further implementing;

creating a transform process on the second computer system in response to the remote server process receiving information regarding the data move.

61. The memory media of claim 51, wherein said data move task is unrelated to said first database operation.

62. The memory media of claim 51, wherein the program instructions further implement:

configuring the data move task to take advantage of existing utility operations in an opportunistic fashion prior to said determining data move requirements;

wherein said determining data move requirements is performed in response to said configuring the data move task to take advantage of existing utility operations in an opportunistic fashion.

63. The memory media of claim 51, wherein the data move process takes advantage of the load utility performing the load operation in an opportunistic fashion.

64. The memory media of claim 51, wherein the load utility comprises a reorganization utility.

65. An enterprise computing system which performs data move operations with improved efficiency, comprising:

a plurality of computer systems, wherein said plurality of computer systems includes a source computer system which includes a source database;

an extract utility which executes in the source computer system which is initialized to begin an extract operation to extract data from the source database, wherein the extract utility is initialized to perform a first database operation, wherein the extract utility is operable to perform an extract operation to extract data;

a program which executes in the enterprise computing system which is operable to determine data move requirements within the enterprise computing system, wherein said program determines data move requirements in response to initialization of said extract utility, wherein the program determines that at least one data move task requires the data;

a data move process which corresponds to the at least one data move task;

wherein the program is operable to configure the extract utility to provide data to the data move process corresponding to the at least one data move task which is determined to require the data from the source database;

wherein the extract utility is operable to provide the data to the data move process during performance of the extract operation, wherein the extract utility is operable to provide the data to the data move process in response to the program determining that the at least one data move task requires the data from the source database;

wherein the data move process is operable to move the data received from the extract utility.

66. The enterprise computing system of claim 65,

wherein said plurality of computer systems includes a target computer system which includes a target database;

wherein the at least one data move task is required to move the data from the source database to the target database;

wherein the data move process is operable to move the data received from the extract utility to the target database.