COMBINING CAPTURE AND APPLY IN A DISTRIBUTED INFORMATION SHARING SYSTEM

• US 20090037422A1
• Filed: 07/31/2007
• Published: 02/05/2009
• Est. Priority Date: 07/31/2007
• Status: Active Grant

• Current Assignee: Oracle International Corporation (Oracle Corporation)
• Original Assignee: Oracle International Corporation (Oracle Corporation)
• Inventors: Arora, Nimar S., Lakshminath, Anand, Gao, Lei, Wong, Lik, Wu, Jingwei, Hoang, Thuvan
• Granted Patent: US 8,799,213 B2
• Time in Patent Office: Days
• US Class Current: 1/1
• CPC Class Codes: G06F 16/27 Replication, distribution or synchronisation of data between databases or within a distributed database system; Distributed database system architectures therefor

Abstract

In accordance with an embodiment of the present invention, an information sharing mechanism comprising a memory structure may be provided in a database system. In an embodiment, mined information transferred by a capture process to an associated apply process can be written into the memory structure without taking any latch. Similarly, the mined information can be read by the apply process from the memory structure without taking any latch. The capture and apply processes may work cooperatively to establish a safe point in log mining under various circumstances such as in an initial startup state, in a steady state, in a process restart scenario in the middle of information sharing. In some embodiments, the information sharing mechanism supports both checkpoint-free and checkpoint modes of log mining by the capture process. In addition, both the capture process and the apply process may employ an eager apply approach to increase processing capacity.

Claims

1. A method for consuming logical change records at a sink node by an apply process, comprising:

setting up a memory structure specifically for a capture process, wherein the capture process is remotely located at a source entity and is associated with the apply process at the sink entity;

receiving a message in a continuous stream of messages from the capture process, wherein the stream of messages comprises interesting logical change records continuously captured at the source entity by the capture process;

storing, without taking any latches on the memory structure, the message in the continuous stream of messages in the memory structure;

consuming, without taking any latches on the memory structure, the message among one or more messages that are stored in the memory structure, wherein the one or more messages are part of the continuous stream of messages; and

removing the message from the memory structure.

2. The method of claim 1, wherein the source entity is a local database system in a distributed database system and wherein the sink entity is another local database system in the distributed database system.

3. The method of claim 1, wherein the memory structure is a circular buffer that comprises multiple entries.

4. The method of claim 3, wherein the capture process is associated with the apply process through a capture proxy process located at the sink entity.

5. The method of claim 4, wherein the capture proxy process maintains a tail pointer to the circular buffer.

6. The method of claim 4, wherein the apply process maintains a head pointer to the circular buffer.

7. The method of claim 1, wherein the message contains a redo record that has a system change number.

8. The method of claim 1, further comprising:

identifying a transaction of which the message is a part;

determining an apply processor to process the transaction; and

forwarding the message to the apply processor.

9. The method of claim 1, further comprising:

determining whether the memory structure has memory space to store the message; and

in response to determining that the memory structure does not have memory space to store the message, writing the message in a secondary memory structure.

10. The method of claim 1, further comprising:

sending a request to the capture process to start mining logs at a particular logical time, wherein the particular logical time corresponds to a specific system change number that was assigned to a particular transaction record of a particular transaction, wherein the particular transaction record is the earliest record received by the apply process of the particular transaction, and wherein the specific system change number is indicated in the request; and

receiving one or more redo records that begins at the start transaction record of the particular transaction from the capture process.

11. The method of claim 1, wherein the capture process avoids taking checkpoints when capturing the interesting logical change records from one or more logs at the source entity.

12. The method of claim 1, further comprising:

reestablishing an association between the apply process and the capture process after the apply process restarts at the sink entity;

sending, by the apply process, a particular system change number to the capture process, wherein the apply process retrieves the particular system change number from a persistent data store and wherein the particular system change number indicates the lowest value for all subsequent system change numbers; and

receiving, from the capture process, one or more messages that contain one or more interesting logical change records captured at the source entity, wherein each of the one or more interesting logical change records carries a system change number that is no less than the particular system change number.

13. The method of claim 1, further comprising:

reestablishing an association between the apply process and the capture process after the capture process restarts at the source entity;

sending a particular system change number to the capture process, wherein the apply process stores the particular system change number as the lowest value for all subsequent system change numbers; and

receiving, from the capture process, one or more messages that contain one or more interesting logical change records captured at the source entity, wherein each of the one or more interesting logical change records carries a system change number that is no less than the particular system change number.

14. The method of claim 1, wherein the step of removing the message from the memory structure is performed before a transaction that is associated with the message is consumed at the sink entity.

15. A computer-readable medium carrying one or more sequences of instructions which, when executed by one or more processors, causes the one or more processors to perform the method recited in claim 1.

16. A computer-readable medium carrying one or more sequences of instructions which, when executed by one or more processors, causes the one or more processors to perform the method recited in claim 2.

17. A computer-readable medium carrying one or more sequences of instructions which, when executed by one or more processors, causes the one or more processors to perform the method recited in claim 3.

18. A computer-readable medium carrying one or more sequences of instructions which, when executed by one or more processors, causes the one or more processors to perform the method recited in claim 4.

19. A computer-readable medium carrying one or more sequences of instructions which, when executed by one or more processors, causes the one or more processors to perform the method recited in claim 5.

20. A computer-readable medium carrying one or more sequences of instructions which, when executed by one or more processors, causes the one or more processors to perform the method recited in claim 6.

21. A computer-readable medium carrying one or more sequences of instructions which, when executed by one or more processors, causes the one or more processors to perform the method recited in claim 7.

22. A computer-readable medium carrying one or more sequences of instructions which, when executed by one or more processors, causes the one or more processors to perform the method recited in claim 8.

23. A computer-readable medium carrying one or more sequences of instructions which, when executed by one or more processors, causes the one or more processors to perform the method recited in claim 9.

24. A computer-readable medium carrying one or more sequences of instructions which, when executed by one or more processors, causes the one or more processors to perform the method recited in claim 10.

25. A computer-readable medium carrying one or more sequences of instructions which, when executed by one or more processors, causes the one or more processors to perform the method recited in claim 11.

26. A computer-readable medium carrying one or more sequences of instructions which, when executed by one or more processors, causes the one or more processors to perform the method recited in claim 12.

27. A computer-readable medium carrying one or more sequences of instructions which, when executed by one or more processors, causes the one or more processors to perform the method recited in claim 13.

28. A computer-readable medium carrying one or more sequences of instructions which, when executed by one or more processors, causes the one or more processors to perform the method recited in claim 14.