Method and a system for polling and processing data

• US 9,489,325 B2
• Filed: 04/27/2012
• Issued: 11/08/2016
• Est. Priority Date: 04/28/2011
• Status: Active Grant

• Current Assignee: Opshub, Inc.
• Original Assignee: Prakash Chandra Tiwary, Sandeep Jain
• Inventors: Jain, Sandeep, Tiwary, Prakash Chandra
• Primary Examiner(s): Duncan, Marc
• Application Number:
  US13/457,492
• Publication Number:
  US 20130117482A1
• Time in Patent Office: 1,656 Days
• Field of Search: 714/15, 707/610, 707/615, 707/616, 707/618, 707/641
• US Class Current: 1/1
• CPC Class Codes: G06F 11/2082 Data synchronisation
• CPC Class Codes: G06F 13/22 using successive scanning, e.g. polling G06F13/24 takes precedence
• CPC Class Codes: G06F 16/2308 Concurrency control transaction processing G06F9/466
• CPC Class Codes: G06F 16/2322 using timestamps
• CPC Class Codes: G06F 16/27 Replication, distribution or synchronisation of data between databases or within a distributed database system; Distributed database system architectures therefor

Abstract

The embodiments herein provide a method and system for polling and processing data. The method comprises computing a maximum time from a source after a last update time, waiting for a preset time to ensure that all transactions with respect to a change in a data is completed, querying for a plurality of changes after an elapse of the preset waiting time since the last update time and up to the maximum time, generating a time window, collecting a list of changes occurred within the generated time window, sending the collected list of changes for processing; and updating the processed data at the destination. The time window comprises a time interval between the last update time and the maximum time.

Claims

1. A computer implemented method comprising instructions stored on a non-transitory computer readable storage medium and run on a computing system comprising a hardware processor and memory for polling and processing a data from a source system to a destination system, wherein the method comprises:

processing a poll event with a processing manager using a processing engine;

mapping a plurality of fields of a source system to corresponding fields of a destination, system with a mapping manager module;

recovering a data automatically in ease of an unexpected failure with a recovery manager;

solving one or more conflicts occurring during a synchronization process with a conflict manager and wherein a conflict is deemed to have occurred when a current value of any field of the destination system does not match with a last value of a field of the source system;

controlling a failed event during the synchronization process with a failure manager and wherein the failed event is the one in which a change in a data is not written to the destination system;

reading a plurality of changes from the source system with a polling module;

writing the plurality of changes at the destination system with an adapter module;

computing a maximum time from the source system after a last update time with the processing manager;

wherein the polling module waits for a preset waiting time to ensure that all transactions with respect to a change in data are completed at the destination system, and wherein the polling module queries and fetches a plurality of changes after an elapse of the preset waiting time since the last update time and up to the computed maximum time and wherein the preset waiting period is a minimum period of time required between two successive polling processes and wherein the preset waiting time is selected such that all transactions which are in flight at a time of a first query are completed, and wherein the polling module executes a two steps polling algorithm for resolving a passing in night problem to ensure that all the transition happened within a time window are read and processed.

2. The method of claim 1, wherein the processing engine is anyone of a Java Process Definition Language (JBOSS JPDL), Drools and an Apache Orchestration Director Engine (ODE).

3. The method of claim 1, wherein the processing manager is configured to report a successful synchronization process back to the source system after a completion of a successful synchronization process.

4. The method of claim 1, wherein the processing manager is configured to pass a control to a failure manager after an unsuccessful synchronization process.

5. The method of claim 1, wherein the recovery manager is configured to control the adapter module and the polling module.

6. The method of claim 1, wherein the conflict manager is configured to detect a conflict to activate an integration model to resolve the conflict automatically.

7. The method of claim 6, wherein the integration model is at least one of a master/slave model, a partitioned ownership model, a dynamic ownership model, a peer to peer model and a custom configuration model and wherein an entity is created at the master system and wherein one or more read only copies of the entity are created in a slave system in the master/slave model and wherein one or more fields of the entity are created at one or more systems in the partitioned ownership model, and wherein an ownership of the entity changes dynamically in the dynamic ownership model, and wherein changes are made to the entity from a plurality of sites to carry out a conflict detection process and a conflict resolving process in the peer to peer model, and wherein a replication model is customized using the custom configuration model.

8. The method of claim 1, further comprises an event based trigger and a scheduler based trigger to synchronize a data from the source system to the destination system, and wherein the scheduler based trigger is skipped, when a poll is active at the time of a scheduler based trigger, and, wherein the event based trigger kicks off the polling process, when an event based trigger is received and no poll is active and the event based trigger sets a flag indicating a need to report at the end of a poll, when the poll is active.

9. The method of claim 8, wherein the polling module is configured to check for a repoll flag at the end of a poll, and wherein the polling module is configured to kick off another poll cycle, when the re poll tag is set.

10. The method of claim 1, wherein the processing manager is configured to generate a time window such that a time interval of the time window is set between the waiting time and the computed maximum time, wherein the polling module is configured to collect a list of changes occurred within the generated time window, and wherein the module is configured to send the collected list of changes for processing, and wherein the time window comprises a time interval between the last update time and the maximum time.

11. The method of claim 1, wherein the adapter module is configured to updates the processed data at the destination system.