Parallel queue propagation

• US 5,870,761 A
• Filed: 12/19/1996
• Issued: 02/09/1999
• Est. Priority Date: 12/19/1996
• Status: Expired due to Term

• Current Assignee: Oracle International Corporation (Oracle Corporation)
• Original Assignee: Oracle Corporation
• Inventors: Oki, Brian, Jain, Sandeep, Stamos, James, Bamford, Roger J., Demers, Alan
• Primary Examiner(s): Black, Thomas G.
• Assistant Examiner(s): ALAM, SHAHID AL
• Application Number:
  US08/770,573
• Time in Patent Office: 782 Days
• Field of Search: 707/10, 707/103, 707/2, 707/7, 707/8, 707/101, 707/201, 707/203, 707/1
• US Class Current: 1/1
• CPC Class Codes: G06F 16/275 Synchronous replication
• CPC Class Codes: Y10S 707/99931 Database or file accessing
• CPC Class Codes: Y10S 707/99932 Access augmentation or optimizing
• CPC Class Codes: Y10S 707/99937 Sorting
• CPC Class Codes: Y10S 707/99938 Concurrency, e.g. lock management in shared database
• CPC Class Codes: Y10S 707/99942 Manipulating data structure, e.g. compression, compaction, compilation
• CPC Class Codes: Y10S 707/99952 Coherency, e.g. same view to multiple users
• CPC Class Codes: Y10S 707/99953 Recoverability

Abstract

A method and system are provided for duplicating at a destination site changes made to data at a source site. According to the method a plurality of streams are established between the source site and the destination site. The plurality of streams are used in parallel to propagate changes made at the source site to the destination site. A record of transactions that made changes that need to be propagated from the source site to the destination site is maintained at the source site. Before propagating changes made by a transaction to the destination site on a stream of the plurality of streams, the record of transactions is inspected to identify a set of transactions whose changes are not known to have been made permanent at the destination site. It is then determined whether the transaction could possibly depend on any transaction in the set of transactions. If the transaction could not possibly depend on any transaction in the set of transactions, then the changes made by the transaction are propagated to the destination site using one of the plurality of streams.

Claims

1. A method for duplicating at a destination site changes made to data at a source site, the method comprising the steps of:

A) establishing a plurality of streams between the source site and the destination site;

B) using said plurality of streams in parallel to propagate changes made at said source site to said destination site by performing the steps of;

(B1) maintaining a record of transactions whose changes must be propagated rom the source site to the destination site;

(B2) before propagating changes made by a transaction to the destination site on a stream of said plurality of streams, performing the steps of;

(a) identifying a set of transactions whose changes are not known to have been made permanent at the destination site by inspecting the record of transactions; and

(b) determining whether the transaction could possibly depend on any transaction in said set of transactions;

(B3) if said transaction could not possibly depend on any transaction in said set of transactions, then propagating the changes made by the transaction to the destination site using one of said plurality of streams.

2. The method of claim 1 wherein if said transaction could possibly depend on any transaction in said set of transactions, then performing the steps of:

determining whether all transactions in said set of transactions on which said transaction could possibly depend were propagated on the same stream of said plurality of streams; and

if all transactions in said set of transactions on which said transaction could possibly depend were propagated on a particular stream of said plurality of streams, then propagating the changes made by said transaction from said source site to said destination site using said particular stream.

3. The method of claim 2 further comprising the step of waiting to propagate the changes made by said transaction until either:

changes made by all transactions on which the transaction could possibly depend are known to have been made permanent at the destination site;

or

changes made by all transactions on which the transaction could possibly depend that are not known to have been made permanent at the destination site were sent over a single stream of said plurality of streams.

4. The method of claim 1 further comprising the step of waiting to propagate the changes made by said transaction until changes made by all transactions on which the transaction could possibly depend are known to have been made permanent at the destination site.

5. The method of claim 1 wherein the step of determining whether the transaction could possibly depend on any transaction in said set of transactions further comprises the steps of:

maintaining a depends-on time for said transaction, wherein the depends-on time indicates a latest commit time of all transactions on which said transaction could depend; and

comparing the depends-on time of the transaction with commit times of each transaction in said set of transactions.

6. The method of claim 1 wherein:

said record includes a plurality of entries, where each entry of said plurality of entries corresponds to a transaction to be propagated to said destination site;

the method further comprises the steps of

adding an entry for said transaction to said plurality of entries;

marking said entry to indicate that said transaction has not yet been sent to said destination site;

after propagating changes made by said transaction, marking said entry to indicate that said transaction has been sent.

7. A method for propagating changes from a source site to a destination site, the method comprising the steps of:

adding to a set of records a record for each transaction that has made changes that must be propagated from the source site to the destination site;

for each record in said set of records, performing the steps of;

determining whether the transaction associated with the record could possibly depend on any transaction associated with any record in the set of records;

if the transaction associated with the record could not possibly depend on any transaction associated with any record in the set of records, then

propagating the changes made by the transaction on one of a plurality of streams established between the source site and the destination site.

8. The method of claim 7 further comprising the step of removing records from the set of records when the source site is informed that the changes made by the transactions associated with the records have been made permanent at the destination site.

9. The method of claim 7 further including the step of marking the record with a value indicates that the transaction associated with the record has been sent to the destination site in response to propagating the changes made by the transaction.

10. The method of claim 7 wherein the step of determining whether the transaction associated with the record could possibly depend on any transaction associated with any record in said set of records comprises the steps of:

determining a depends-on time for the transaction associated with the record, where the depends-on time indicates a commit time of a latest transaction on which the transaction depends; and

comparing the depends-on time with commit times of transactions represented in said set of records.

11. The method of claim 7 wherein said step of adding to a set of records a record for each transaction that has made changes that must be propagated from the source site to the destination site includes the step of adding to a data structure stored in volatile memory an entry for each transaction that has made changes that must be propagated from the source site to the destination site.

12. A computer-readable medium having stored thereon sequences of instructions for propagating changes from a source site to a destination site, the sequences of instructions including sequences of instructions for performing the steps of:

adding to a set of records a record for each transaction that has made changes that must be propagated from the source site to the destination site;

for each record in said set of records, performing the steps of;

determining whether the transaction associated with the record could possibly depend on any transaction associated with any record in the set of records;

if the transaction associated with the record could not possibly depend on any transaction associated with any record in the set of records, then

propagating the changes made by the transaction on one of a plurality of streams established between the source site and the destination site.

13. The computer-readable medium of claim 12 further comprising sequences of instructions for performing the step of removing records from the set of records when the source site is informed that the changes made by the transactions associated with the records have been made permanent at the destination site.

14. The computer-readable medium of claim 12 further comprising sequences of instructions for performing the step of marking the record with a value indicates that the transaction associated with the record has been sent to the destination site in response to propagating the changes made by the transaction.

15. The computer-readable medium of claim 12 wherein the step of determining whether the transaction associated with the record could possibly depend on any transaction associated with any record in a second set of records comprises the steps of:

determining a depends-on time for the transaction associated with the record, where the depends-on time indicates a commit time of a latest transaction on which the transaction depends; and

comparing the depends-on time with commit times of transactions represented by said set of records.

16. The computer-readable medium of claim 12 wherein said step of adding to a set of records a record for each transaction that has made changes that must be propagated from the source site to the destination site includes the step of adding to a data structure stored in volatile memory an entry for each transaction that has made changes that must be propagated from the source site to the destination site.

17. A method for duplicating at a destination site changes made to data at a source site, the method comprising the steps of:

A) establishing a plurality of streams between the source site and the destination site;

B) using said plurality of streams in parallel to propagate changes made at said source site to said destination site by performing the steps of;

(B1) determining whether a transaction to be propagated to the destination site depends on any other transaction to be propagated to the destination site that is not known to have been made permanent at the destination site;

(B2) if said transaction could not possibly depend on any other transaction to be propagated to the destination site that is not known to have been made permanent at the destination site, then propagating the changes made by the transaction to the destination site using one of said plurality of streams.

18. The method of claim 17 wherein if said transaction could possibly depend on any other transaction to be propagated to the destination site that is not known to have been made permanent at the destination site, then performing the steps of:

determining whether all transactions to be propagated to the destination site on which said transaction could possibly depend were propagated on the same stream of said plurality of streams; and

if all transactions to be propagated to the destination site on which said transaction could possibly depend were propagated on a particular stream of said plurality of streams, then propagating the changes made by said transaction from said source site to said destination site using said particular stream.

19. The method of claim 17 further comprising the step of waiting to propagate the changes made by said transaction until changes made by all transactions on which the transaction could possibly depend are known to have been made permanent at the destination site.

20. The method of claim 17 wherein the step of determining whether a transaction to be propagated to the destination site depends on any other transaction to be propagated to the destination site that is not known to have been made permanent at the destination site further comprises the steps of:

maintaining a depends-on time for said transaction, wherein the depends-on time indicates a latest commit time of all transactions on which said transaction could depend; and

comparing the depends-on time of the transaction with commit times of transactions to be propagated to the destination site that are not known to have been made permanent at the destination site.

21. A computer-readable medium carrying one or more sequences of one or more instructions for duplicating at a destination site changes made to data at a source site, the one or more sequences of one or more instructions including instructions which, when executed by one or more processors, cause the one or more processors to perform the steps of:

A) establishing a plurality of streams between the source site and the destination site;

B) using said plurality of streams in parallel to propagate changes made at said source site to said destination site by performing the steps of;

(B1) maintaining a record of transactions whose changes must be propagated from the source site to the destination site;

(B2) before propagating changes made by a transaction to the destination site on a stream of said plurality of streams, performing the steps of;

(a) identifying a set of transactions whose changes are not known to have been made permanent at the destination site by inspecting the record of transactions; and

(b) determining whether the transaction could possibly depend on any transaction in said set of transactions;

(B3) if said transaction could not possibly depend on any transaction in said set of transactions, then propagating the changes made by the transaction to the destination site using one of said plurality of streams.

22. The computer-readable medium of claim 21, further comprising injections for performing the following steps if said transaction could possibly depend on any transaction in said set of transactions:

determining whether all transactions in said set of transactions on which said transaction could possibly depend were propagated on the same stream of said plurality of streams; and

if all transactions in said set of transactions on which said transaction could possibly depend were propagated on a particular stream of said plurality of streams, then propagating the changes made by said transaction from said source site to said destination site using said particular stream.

23. The computer-readable medium of claim 22 further comprising instructions for performing the step of waiting to propagate the changes made by said transaction until either:

changes made by all transactions on which the transaction could possibly depend are known to have been made permanent at the destination site;

or

changes made by all transactions on which the transaction could possibly depend that are not know to have been made permanent at the destination site were sent over a single stream of said plurality of streams.

24. The computer-readable medium of claim 21 further comprising instructions for performing the step of waiting to propagate the changes made by said transaction until changes made by all transactions on which the transaction could possibly depend are known to have been made permanent at the destination site.

25. The computer-readable medium of claim 21 wherein the step of determining whether the transaction could possibly depend on any transaction in said set of transactions further comprises the steps of:

maintaining a depends-on time for said transaction, wherein the depends-on time indicates a latest commit time of all transactions on which said transaction could depend; and

comparing the depends-on time of the transaction with commit times of each transaction in said set of transactions.

26. The computer-readable medium of claim 21 wherein:

said record includes a plurality of entries, where each entry of said plurality of entries corresponds to a transaction to be propagated to said destination site;

the computer-readable medium further comprising instructions for performing the steps of

adding an entry for said transaction to said plurality of entries;

marking said entry to indicate that said transaction has not yet been sent to said destination site;

after propagating changes made by said transaction, marking said entry to indicate that said transaction has been sent.

27. A computer-readable medium carrying one or more sequences of one or more instructions for duplicating at a destination site changes made to data at a source site, the one or more sequences of one or more instructions including instructions which, when executed by one or more processors, cause the one or more processors to perform the steps of:

A) establishing a plurality of streams between the source site and the destination site;

B) using said plurality of streams in parallel to propagate changes made at said source site to said destination site by performing the steps of;

(B1) determining whether a transaction to be propagated to the destination site depends on any other transaction to be propagated to the destination site that is not known to have been made permanent at the destination site;

(B2) if said transaction could not possibly depend on any other transaction to be propagated to the destination site that is not known to have been made permanent at the destination site, then propagating the changes made by the transaction to the destination site using one of said plurality of streams.

28. The computer-readable medium of claim 27, further comprising instructions for performing the following steps if said transaction could possibly depend on any other transaction to be propagated to the destination site that is not know to have been made permanent at the destination site:

determining whether all transactions to be propagated to the destination site on which said transaction could possibly depend were propagated on the same stream of said plurality of streams; and

if all transactions to be propagated to the destination site on which said transaction could possibly depend were propagated on a particular stream of said plurality of streams, then propagating the changes made by said transaction tom said source site to said destination site using said particular stream.

29. The computer-readable medium of claim 27 further comprising instructions for performing the step of waiting to propagate the changes made by said transaction until changes made by all transactions on which the transaction could possibly depend are known to have been made permanent at the destination site.

30. The computer-readable medium of claim 27 wherein the step of determining whether a transaction to be propagated to the destination site depends on any other transaction to be propagated to the destination site that is not known to have been made permanent at the destination site further comprises the steps of:

maintaining a depends-on time for said transaction, wherein the depends-on time indicates a latest commit time of all transactions on which said transaction could depend; and

comparing the depends-on time of the transaction with commit times of transactions to be propagated to the destination site that are not known to have been made permanent at the destination site.

31. A system for duplicating changes, the system comprising:

a source site;

a destination site;

a plurality of streams between the source site and the destination site; and

a propagation mechanism, wherein the propagation mechanism is configured to

propagate changes made at said source site to said destination site by performing the steps of;

A1) maintaining a record of transactions whose changes must be propagated from the source site to the destination site;

(A2) before propagating changes made by a transaction to the destination site on a stream of said plurality of streams, performing the steps of;

(a) identifying a set of transactions whose changes are not known to have been made permanent at the destination site by inspecting the record of transactions; and

(b) determining whether the transaction could possibly depend on any transaction in said set of transactions;

(A3) if said transaction could not possibly depend on any transaction in said set of transactions, then propagating the changes made by the transaction to the destination site using one of said plurality of streams.