System and method for incremental replication
First Claim
Patent Images
1. A method comprising:
- receiving, at an exporting system, a command to execute an incremental replication job;
wherein the exporting system tracks changes to database data in a database by creating, in the database, change records for changes to the database data, the change records comprising logical clock values reflecting an order of the changes, the logical clock values obtained from a logical clock used to order changes to the database;
based on the command, the exporting system creating and storing an incremental replication plan for the incremental replication job;
wherein creating and storing the incremental replication plan comprises;
determining a number of incremental replication chunks to divide the incremental replication job into,determining a current logical clock value of the logical clock, andstoring the current logical clock value as part of the incremental replication plan;
wherein the stored incremental replication plan comprises a number of incremental replication chunk specifications, one for each of the number of incremental replication chunks that the incremental replication job is divided into;
wherein each of the number of incremental replication chunk specifications comprises information that identifies a set of a plurality of data objects that are to be exported as part of the incremental replication chunk;
after creating and storing the incremental replication plan, the exporting system executing the incremental replication job according to the incremental replication plan; and
wherein executing the incremental replication job comprises, for each of the incremental replication chunk specifications that the incremental replication job is divided into;
collecting, from the database, based on the information in the incremental replication chunk specification and based on the current logical clock value stored as part of the incremental replication plan, change data from one or more change records for the set of data objects to be exported as part of the incremental replication chunk, andproviding the collected change data to an importing system.
7 Assignments
0 Petitions
Accused Products
Abstract
A method of incrementally replicating investigative analysis data is disclosed along with a system for performing the same. The method and system provide the ability to break a data replication job into multiple “replication chunks” which can be exported and imported separately. By doing so, the method and system can efficiently replicate large numbers of data object changes over an unreliable data network.
-
Citations
20 Claims
-
1. A method comprising:
-
receiving, at an exporting system, a command to execute an incremental replication job;
wherein the exporting system tracks changes to database data in a database by creating, in the database, change records for changes to the database data, the change records comprising logical clock values reflecting an order of the changes, the logical clock values obtained from a logical clock used to order changes to the database;based on the command, the exporting system creating and storing an incremental replication plan for the incremental replication job; wherein creating and storing the incremental replication plan comprises; determining a number of incremental replication chunks to divide the incremental replication job into, determining a current logical clock value of the logical clock, and storing the current logical clock value as part of the incremental replication plan; wherein the stored incremental replication plan comprises a number of incremental replication chunk specifications, one for each of the number of incremental replication chunks that the incremental replication job is divided into; wherein each of the number of incremental replication chunk specifications comprises information that identifies a set of a plurality of data objects that are to be exported as part of the incremental replication chunk; after creating and storing the incremental replication plan, the exporting system executing the incremental replication job according to the incremental replication plan; and wherein executing the incremental replication job comprises, for each of the incremental replication chunk specifications that the incremental replication job is divided into; collecting, from the database, based on the information in the incremental replication chunk specification and based on the current logical clock value stored as part of the incremental replication plan, change data from one or more change records for the set of data objects to be exported as part of the incremental replication chunk, and providing the collected change data to an importing system. - View Dependent Claims (2, 3, 4, 5, 6, 7)
-
-
8. One or more non-transitory computer-readable media storing instructions which, when executed by one or more processors, causes performance of a method comprising the steps of:
-
receiving, at an exporting system, a command to execute an incremental replication job;
wherein the exporting system tracks changes to database data in a database by creating, in the database, change records for changes to the database data, the change records comprising logical clock values reflecting an order of the changes, the logical clock values obtained from a logical clock used to order changes to the database;based on the command, the exporting system creating and storing an incremental replication plan for the incremental replication job; wherein creating and storing the incremental replication plan comprises; determining a number of incremental replication chunks to divide the incremental replication job into, determining a current logical clock value of the logical clock, and storing the current logical clock value as part of the incremental replication plan; wherein the stored incremental replication plan comprises a number of incremental replication chunk specifications, one for each of the number of incremental replication chunks that the incremental replication job is divided into; wherein each of the number of incremental replication chunk specifications comprises information that identifies a set of a plurality of data objects that are to be exported as part of the incremental replication chunk; after creating and storing the incremental replication plan, the exporting system executing the incremental replication job according to the incremental replication plan; and wherein executing the incremental replication job comprises, for each of the incremental replication chunk specifications that the incremental replication job is divided into; collecting, from the database, based on the information in the incremental replication chunk specification and based on the current logical clock value stored as part of the incremental replication plan, change data from one or more change records for the set of data objects to be exported as part of the incremental replication chunk, and providing the collected change data to an importing system. - View Dependent Claims (9, 10, 11, 12, 13, 14)
-
-
15. A system comprising:
-
a database; an exporting system comprising one or more computer-readable media and one or more processors, the one or more computer-readable media storing instructions which, when executed by the one or more processors, causes performance of a method comprising the steps of; receiving, at the exporting system, a command to execute an incremental replication job; wherein the exporting system tracks changes to database data in the database by creating, in the database, change records for changes to the database data, the change records comprising logical clock values reflecting an order of the changes, the logical clock values obtained from a logical clock used to order changes to the database; based on the command, the exporting system creating and storing an incremental replication plan for the incremental replication job; wherein creating and storing the incremental replication plan comprises; determining a number of incremental replication chunks to divide the incremental replication job into, determining a current logical clock value of the logical clock, and storing the current logical clock value as part of the incremental replication plan; wherein the stored incremental replication plan comprises a number of incremental replication chunk specifications, one for each of the number of incremental replication chunks that the incremental replication job is divided into; wherein each of the number of incremental replication chunk specifications comprises information that identifies a set of a plurality of data objects that are to be exported as part of the incremental replication chunk; after creating and storing the incremental replication plan, the exporting system executing the incremental replication job according to the incremental replication plan; and wherein executing the incremental replication job comprises, for each of the incremental replication chunk specifications that the incremental replication job is divided into; collecting, from the database, based on the information in the incremental replication chunk specification and based on the current logical clock value stored as part of the incremental replication plan, change data from one or more change records for the set of data objects to be exported as part of the incremental replication chunk, and providing the collected change data to an importing system. - View Dependent Claims (16, 17, 18, 19, 20)
-
Specification