Method and A System for Synchronizing Data

• US 20130117226A1
• Filed: 04/27/2012
• Published: 05/09/2013
• Est. Priority Date: 04/28/2011
• Status: Active Grant

• Current Assignee: Opshub, Inc.
• Original Assignee: Prakash Chandra Tiwary, Sandeep Jain
• Inventors: Jain, Sandeep, Tiwary, Prakash Chandra
• Granted Patent: US 8,938,421 B2
• Time in Patent Office: Days
• US Class Current: 707/625
• CPC Class Codes: G06F 11/2082 Data synchronisation
• CPC Class Codes: G06F 13/22 using successive scanning, e.g. polling G06F13/24 takes precedence
• CPC Class Codes: G06F 16/2308 Concurrency control transaction processing G06F9/466
• CPC Class Codes: G06F 16/2322 using timestamps
• CPC Class Codes: G06F 16/27 Replication, distribution or synchronisation of data between databases or within a distributed database system; Distributed database system architectures therefor

Abstract

The embodiments herein provide a system and a method for synchronizing a plurality of changes from a source to a destination. The method comprises the steps of starting with an initial polling time, checking for an elapse of a polling interval from a last polled time, polling a source for fetching a data after an elapse of a polling interval from a last polled time, selecting a plurality of changes from the source made by a user, processing the data fetched at the last polling time in a single thread, writing the processed data at a destination and reporting a status of a synchronization process. The initial polling time is a time from which a synchronization of changes in a data is initiated. The method uses both an even t based trigger and a scheduler based trigger

Claims

1. A method for synchronizing a plurality of changes from a source to a destination comprises the steps of:

starting with an initial polling time, wherein the initial polling time is a time from which a synchronization of changes in a data is initiated;

checking for an elapse of a polling interval from a last polled time;

polling a source for fetching a data after an elapse of a polling interval from a last polled time;

selecting a plurality of changes from the source made by a user and wherein the user is not an integration user;

processing the data fetched at the last polling time in a single thread and wherein a step of processing the data fetched at the last polling time comprises a step of ordering the plurality of changes based on a data volume and a location;

writing the processed data at a destination; and

reporting a status of a synchronization process and wherein the status is any one of a successful synchronization and an unsuccessful synchronization;

wherein a step of polling the source for fetching a data is not initiated until the data fetched at the last polling time is not processed.

2. The method of claim 1, wherein the polling interval is set to a very small value and wherein the polling interval is set within a range of 1 min to 3 min.

3. The method of claim 1, wherein the integration user is a user authorized to login to the source or the destination.

4. The method of claim 1, wherein a change in the data volume is any one of an incremental change and a bulk change.

5. The method of claim 1 wherein the location is a single location of the source or a plurality of locations of the source.

6. The method of claim 1, wherein the step of ordering the plurality of changes based on the data volume and location comprises receiving the plurality of changes after the initial polling time, and storing the plurality of changes received based on a time, an entity id and a revision id, when the data volume is of an incremental change and the location is a single location from the source.

7. The method of claim 1, wherein the step of ordering the plurality of changes based on the data volume and location comprises:

receiving the plurality of changes and storing a state of a data change for the pluralities of changes received based on a last polled time, a last processed entity Id and a last change Id, when the data volume is of bulk changes and the location is a single location from the source;

wherein the last polled time is a change time of an entity and wherein the last processed entity Id is the change entity Id and wherein the last change Id is the change id of a change in the last processed entity and wherein the change entity Id includes a number sequence, and wherein the number sequence for a pluralities of change entities are assigned in an ascending order in a sequence; and

wherein the state of a data change is stored for the pluralities of changes received, when the change time is equal to the last polled time and the change entity Id is greater than or equal to the last processed entity Id.

8. The method of claim 1, wherein the step of ordering the plurality of changes based on the data volume and location, when the data volume is of bulk changes and the location is a plurality of locations from the source, comprises:

querying each location of the plurality of locations to get a last change time, and wherein the last change time is the time at which a plurality of changes are made at the source at a plurality of locations;

identifying a greater time from a plurality of last change times received from each location with respect to a query;

setting the identified greater time of the plurality of last change times as a maximum time;

waiting for a preset time before starting a polling process and wherein the preset time is 10 seconds; and

polling the plurality of changes done after an elapse of the preset waiting time and before a completion of the set maximum time.

9. The method of claim 1, wherein the step of reporting the status of the synchronization process comprises reporting the source of the successful synchronization after a completion of a successful synchronization process.

10. The method of claim 1, wherein the step of reporting the status of the synchronization process comprises passing a control to a failure manager after an unsuccessful synchronization process.

11. The method of claim 10, further comprises:

catching a one or more failed events and wherein a change in a data is not written to the destination in the one or more failed events;

creating a failure queue; and

retrying a synchronization process repeatedly for a preset number of attempts after an unsuccessful synchronization process.

12. The method of claim 1, further comprises employing a event based trigger and a scheduler based trigger to synchronize a data from a source to a destination.

13. A system for synchronizing a data from a source to a destination comprises:

a connector framework and wherein the connector framework comprises two connector modules, a polling module and an adopter module;

a processing manager and wherein the processing manager processes a poll event using a processing engine;

a mapping manager and wherein the mapping manager maps a plurality of fields of the source to a corresponding fields of a destination;

a recovery manager and wherein the recovery manager automatically recovers a data in case of an unexpected failure;

a conflict manager and wherein the conflict manager solves one or more conflicts occurring during a synchronization process and wherein a conflict is occurred, when a current value of any field of the destination does not match with a last value of a field of the source;

a failure manager and wherein the failure manager controls a failed event during a synchronization process and wherein the failed event is the one in which a change in a data is not written to the destination;

wherein the polling Module reads the plurality of changes from the source and the adapter module to write the plurality of changes at the destination.

14. The system of claim 13, wherein the processing engine is any one of a Java Process Definition Language (MOSS JPDL), Drools and an Apache Orchestration Director Engine (ODE) or any other processing engine.

15. The system of claim 13, wherein the processing manager reports a successful synchronization process back to the source after a completion of a successful synchronization process.

16. The system of claim 13, wherein the processing manager passes a control to a failure manager after an unsuccessful synchronization process.

17. The system of claim 13, wherein the recovery manager controls the adapter module and the polling module.

18. The system of claim 13, wherein the conflict manager detects a conflict to activate an integration model to resolve the conflict automatically.

19. The conflict manager of claim 13, wherein the integration model is at least one of a master/slave model, a partitioned ownership model, a dynamic ownership model, a peer to peer model and a custom configuration model, and wherein the entity is created at the master and wherein one or more read only copies of the entity are created in slave system in the master slave model, and wherein one or more fields of the entity are created at one or more systems in the partitioned ownership model, and wherein an ownership of the entity changes dynamically in the dynamic ownership model, and wherein a plurality of sites are able to make changes to the entities to carry out a conflict detection process and a conflict resolving process in the peer to peer model, and wherein a replication model is customized using the custom configuration model.

20. The system of claim 13 further comprises a event based trigger and a scheduler based trigger to synchronize a data from a source to a destination.