Replicating data between heterogeneous data systems

• US 7,668,878 B2
• Filed: 07/31/2006
• Issued: 02/23/2010
• Est. Priority Date: 07/31/2006
• Status: Active Grant

• Current Assignee: Oracle International Corporation (Oracle Corporation)
• Original Assignee: Oracle International Corporation (Oracle Corporation)
• Inventors: Shadhan, Neeraj, Lakshminath, Anand, Arora, Nimar, Wong, Lik, Wang, Byron, Lu, Edwina, Hoang, Thuvan
• Primary Examiner(s): Robinson; Greta L
• Assistant Examiner(s): Wilcox; James J
• Application Number:
  US11/496,949
• Publication Number:
  US 20080027987A1
• Time in Patent Office: 1,303 Days
• Field of Search: None
• US Class Current: 707/610
• CPC Class Codes: G06F 16/184 implemented as replicated file system
• CPC Class Codes: G06F 16/258 Data format conversion from or to a database
• CPC Class Codes: G06F 16/27 Replication, distribution or synchronisation of data between databases or within a distributed database system; Distributed database system architectures therefor

Abstract

A method and apparatus for replicating data between heterogeneous databases is provided. Data is replicated between two heterogeneous databases with the use of a volatile storage queue, enabling the rapid replication of data across databases provided by different vendors or operating on different platforms. According to one embodiment, an in-memory queue is used to queue change operations to be performed on a target data repository. The change operations may be operations that were applied to a source data repository. An apply process retrieves the change operations from in-memory queue and commits the change operations to persistent storage. When the change operations have been committed, the apply process notifies the source platform that the particular change operation has been stored.

Claims

1. A computer-implemented method comprising:

retrieving, from a volatile storage queue, first data that identifies an operation that has been performed on a source data repository;

wherein the operation changed data within the source data repository; and

wherein the first data includes

(a) an operation sequence identifier, wherein the operation sequence identifier identifies a sequential order of each operation applied to the source data repository, and

(b) information necessary to replicate the change on a target data repository;

performing the operation on said target data repository;

after the operation on the target data repository has been performed, notifying a process that tracks changes on the source data repository that the operation has been performed on the target data repository, wherein notifying comprises sending to the process that tracks changes on the source repository, a message that includes the operation sequence identifier;

receiving, at the source data repository, the indication that the operation has been performed against the target data repository;

determining, at the source data repository, the operation identifier associated with the first data;

purging the first data from the source data repository; and

wherein the steps of the method are performed by one or more computing devices.

2. The method of claim 1, further comprising receiving the first data in response to a change occurring on a source data repository.

3. The method of claim 1, wherein the process that tracks changes on the source data repository associates the operation sequence identifier with the first data.

4. The method of claim 1, further comprising:

receiving the first data in response to a change occurring on the source data repository;

converting the first data into a non-native format;

wherein converting the first data comprises performing data translation to an intermediate data format understood by a target platform, wherein the target platform performs transactions against the target data repository; and

enqueuing the first data in the volatile storage queue.

5. The method of claim 4, further comprising:

in response to enqueuing the first data in the volatile storage queue, storing the operation sequence identifier of the first data in memory.

6. The method of claim 5, further comprising:

receiving second data, wherein the second data includes a second operation sequence identifier;

examining the operation sequence identifier of the first data in memory;

determining that the operation sequence identifier of the first data occurs after the operation sequence identifier for the second data; and

discarding the second data.

7. The method of claim 1, wherein the volatile storage queue is an in-memory queue.

8. The method of claim 1, wherein said source data repository is a source file system and wherein said target data repository is a target file system.

9. The method of claim 8, wherein said first data represents one or more changes performed on the source file system.

10. The method of claim 1, further comprising the additional step of:

in response to performing the operation against the target data repository, removing the first data from the volatile storage queue.

11. The method of claim 1, further comprising:

in response to performing the operation against the target data repository, storing the operation sequence identifier of the first data in persistent storage.

12. The method of claim 11, further comprising:

retrieving the operation sequence identifier of the first data from persistent storage;

determining that the operation represented by the first data has been performed against the target data repository; and

discarding any operations represented by data with an operation sequence identifier occurring before the operation sequence identifier of the first data.

13. The method of claim 1, wherein the volatile storage queue comprises a plurality of data representing operations to be performed against the target data repository.

14. The method of claim 6, wherein each data in the volatile storage queue contains an operation sequence identifier.

29. The method of claim 1, wherein the step of notifying is performed in response to performing the operation on the target repository.

30. The method of claim 1, wherein the step of notifying is performed in response to a query received by the target platform from the source platform.

15. A computer-readable storage medium storing one or more sequences of instructions which, when executed by one or more processors, causes the one or more processors to perform:

retrieving, from a volatile storage queue, first data that identifies an operation that has been performed on a source data repository;

wherein the operation changed data within the source data repository; and

wherein the first data includes

(a) an operation sequence identifier, wherein the operation sequence identifier identifies a sequential order of each operation applied to the source data repository, and

(b) information necessary to replicate the change on a target data repository;

performing the operation on said target data repository;

after the operation on the target data repository has been performed, notifying a process that tracks changes on the source data repository that the operation has been performed on the target data repository, wherein notifying comprises sending to the process that tracks changes on the source repository, a message that includes the operation sequence identifier;

receiving, at the source data repository, the message that indicates that the operation has been performed against the target data repository;

determining, at the source data repository, the operation identifier associated with the first data; and

purging the first data from the source data repository.

16. The computer-readable storage medium of claim 15, further comprising instructions for receiving the first data in response to a change occurring on a source data repository.

17. The computer-readable storage medium of claim 15,

wherein the process that tracks changes on the source data repository associates the operation sequence identifier with the first data.

18. The computer-readable storage medium of claim 15, further comprising instructions for performing:

receiving the first data in response to a change occurring on the source data repository;

converting the first data into a non-native format;

wherein converting the first data comprises performing data translation to an intermediate data format understood by a target platform, wherein the target platform performs transactions against the target data repository; and

enqueuing the first data in the volatile storage queue.

19. The computer-readable storage medium of claim 18, further comprising instructions for:

in response to enqueuing the first data in the volatile storage queue, storing the operation sequence identifier of the first data in memory.

20. The computer-readable storage medium of claim 19, further comprising instructions for:

receiving second data, wherein the second data includes a second operation sequence identifier;

examining the operation sequence identifier of the first data in memory;

determining that the operation sequence identifier of the first data occurs after the operation sequence identifier for the second data; and

discarding the second data.

21. The computer-readable storage medium of claim 15,

wherein the volatile storage queue is an in-memory queue.

22. The computer-readable storage medium of claim 15,

wherein said source data repository is a source file system and wherein said target data repository is a target file system.

23. The computer-readable storage medium of claim 22,

wherein said first data represents one or more changes performed on the source file system.

24. The computer-readable storage medium of claim 15, further comprising instructions for:

in response to performing the operation against the target data repository, removing the first data from the volatile storage queue.

25. The computer-readable storage medium of claim 15, further comprising instructions for:

in response to performing the operation against the target data repository, storing the operation sequence identifier of the first data in persistent storage.

26. The computer-readable storage medium of claim 25, further comprising instructions for:

retrieving the operation sequence identifier of the first data from persistent storage;

determining that the operation represented by the first data has been performed against the target data repository; and

discarding any operations represented by data with an operation sequence identifier occurring before the operation sequence identifier of the first data.

27. The computer-readable storage medium of claim 15, wherein the volatile storage queue comprises a plurality of data representing operations to be performed against the target data repository.

28. The computer-readable storage medium of claim 20,

wherein each data in the volatile storage queue contains an operation sequence identifier.

31. The computer readable storage medium of claim 15, wherein the step of notifying is performed in response to performing the operation on the target repository.

32. The computer readable storage medium of claim 15, wherein the step of notifying is performed in response to performing the operation on the target repository.