Checkpoint-free in log mining for distributed information sharing

• US 7,801,852 B2
• Filed: 07/31/2007
• Issued: 09/21/2010
• Est. Priority Date: 07/31/2007
• Status: Active Grant

• Current Assignee: Oracle International Corporation (Oracle Corporation)
• Original Assignee: Oracle International Corporation (Oracle Corporation)
• Inventors: Arora, Nimar S., Schmidt, Cristina, Gao, Lei, Wong, Lik, Hoang, Thuvan
• Primary Examiner(s): Vo; Tim T.
• Assistant Examiner(s): Fan; Shiow-Jy
• Application Number:
  US11/831,447
• Publication Number:
  US 20090037494A1
• Time in Patent Office: 1,148 Days
• Field of Search: 707/101, 707/202, 707/E17.005, 707/999.202, 707/609, 707/610, 707/613, 707/615
• US Class Current: 707/615
• CPC Class Codes: G06F 16/2358 Change logging, detection, and notification replication G06F16/27

Abstract

A checkpoint free log mining mechanism comprising a capture process and an apply process that are logically associated with each other may be provided in a database system. In an embodiment, log mining information published by the capture process to the apply process comprises a complete set of log information for each interesting transaction. A system change number of a start transaction record of a transaction may be used to represent a start time of the entire transaction. The capture and apply processes may work cooperatively to establish a safe time point in the form of a particular system change number in logs to begin mining, under various circumstances such as in an initial startup state, in a steady state, in a process restart scenario in the middle of checkpoint free log mining, or in a transition between the checkpoint mode and the checkpoint free mode.

Claims

1. A method for selecting and sending one or more transactions, from a set of transactions, for replication between database systems, comprising:

receiving by a capturing process of a first database system, at a first time, a particular start transaction record;

adding, by the capturing process, the particular start transaction record in a list, wherein the list comprises zero or more start transaction records that were added to the list before the particular start transaction record is added to the list;

wherein each of the particular start transaction record and the zero or more start transaction records is a start transaction record for one of a subset of transactions in the first database system;

wherein, when a subset of transactions makes a data change in the first database system that should be replicated in a second database system, (a) all start transaction records in the list are sent to one or more apply processes of a second database system and (b) all the start transaction records in the list are removed from the list;

receiving by the capturing process, at a second time that is later than the first time, a particular end transaction record that signals an end of a transaction whose beginning is signaled by the particular start transaction record;

determining, by the capturing process, whether the particular start transaction record still exists in the list;

in response to determining that the particular start transaction record still exists in the list, performing;

determining that there is no data change that should be replicated in the second database system following the particular start transaction record; and

removing, by the capturing process, the particular start transaction record from the list;

and

in response to determining that the particular start transaction record does not exist in the list, performing;

determining that data changes following the particular start transaction record have been sent in the second database system; and

sending, by the capturing process, the particular end transaction record to the one or more apply processes;

wherein the method is performed by one or more computing devices comprising a processor.

2. The method of claim 1, further comprising:

receiving by the capturing process, at a third time, a particular interesting data change record, wherein the third time may be before or after the first time, and wherein the third time may be before or after the second time;

sending by the capturing process, all start transaction records that currently exist in the list to the one or more apply processes; and

emptying by the capturing process, the list by removing all the start transaction records that currently exist in the list.

3. A method for reducing communication costs by advancing system change numbers past certain transactions comprising:

determining by a capturing process, for a particular period, whether any start transaction record containing a system change number has been sent to an apply process; and

in response to determining by the capturing process, for the particular period, that no start transaction record containing a system change number has been sent to the apply process, performing by the capturing process;

establishing a plurality of active transactions that are active when the particular period ends;

identifying a particular active transaction in the plurality of active transactions, wherein each of the plurality of active transactions has a start transaction record associated with a system change number and wherein a particular system change number that is associated with the start transaction record of the particular active transaction has the lowest value among a set of system change numbers that comprises all system change numbers that are associated with the plurality of active transactions;

determining the particular system change number of the start transaction record of the particular active transaction; and

sending a ping record to the apply process, wherein the ping record contains the particular system change number, wherein the ping record informs the apply process that new data changes to the apply process will have system change numbers no lower than the particular system change number;

wherein the method is performed by one or more computing devices comprising a processor.

4. A computer-readable storage medium carrying one or more sequences of instructions which, when executed by one or more processors, causes the one or more processors to perform:

receiving by a capturing process of a first database system, at a first time, a particular start transaction record;

adding, by the capturing process, the particular start transaction record in a list, wherein the list comprises zero or more start transaction records that were added to the list before the particular start transaction record is added to the list;

wherein each of the particular start transaction record and the zero or more start transaction records is a start transaction record for one of a subset of transactions in the first database system;

wherein, when a subset of transactions makes a data change in the first database system that should be replicated in a second database system, (a) all start transaction records in the list are sent to one or more apply processes of a second database system and (b) all the start transaction records in the list are removed from the list;

receiving by the capturing process, at a second time that is later than the first time, a particular end transaction record that signals an end of a transaction whose beginning is signaled by the particular start transaction record;

determining, by the capturing process, whether the particular start transaction record still exists in the list;

in response to determining that the particular start transaction record still exists in the list, performing;

determining that there is no data change that should be replicated in the second database system following the particular start transaction record; and

removing, by the capturing process, the particular start transaction record from the list;

and

in response to determining that the particular start transaction record does not exist in the list, performing;

determining that data changes following the particular start transaction record have been sent in the second database system; and

sending, by the capturing process, the particular end transaction record to the one or more apply processes.

5. The medium of claim 4, further comprising:

receiving by the capturing process, at a third time, a particular interesting data change record, wherein the third time may be before or after the first time, and wherein the third time may be before or after the second time;

sending by the capturing process, all start transaction records that currently exist in the list to the one or more apply processes; and

emptying by the capturing process, the list by removing all the start transaction records that currently exist in the list.

6. A computer-readable storage medium carrying one or more sequences of instructions which, when executed by one or more processors, causes the one or more processors to perform:

determining by a capturing process, for a particular period, whether any start transaction record containing a system change number has been sent to an apply process; and

in response to determining by the capturing process, for the particular period, that no start transaction record containing a system change number has been sent to the apply process, performing by the capturing process;

establishing a plurality of active transactions that are active when the particular period ends;

identifying a particular active transaction in the plurality of active transactions, wherein each of the plurality of active transactions has a start transaction record associated with a system change number and wherein a particular system change number that is associated with the start transaction record of the particular active transaction has the lowest value among a set of system change numbers that comprises all system change numbers that are associated with the plurality of active transactions;

determining the particular system change number of the start transaction record of the particular active transaction; and

sending a ping record to the apply process, wherein the ping record contains the particular system change number, wherein the ping record informs the apply process that new data changes to the apply process will have system change numbers no lower than the particular system change number.