Data gateway for jobs management based on a persistent job table and a server table

• US 8,065,680 B2
• Filed: 11/15/2005
• Issued: 11/22/2011
• Est. Priority Date: 11/15/2005
• Status: Active Grant

• Current Assignee: R2 Solutions LLC (Acacia Research Corporation)
• Original Assignee: Yahoo! Inc. (Apollo Global Management, Inc.)
• Inventors: Parvathaneni, Bhaskar A., Srinivasan, Venkatachary, Tendjoukian, Meher, Yang, Lie
• Primary Examiner(s): TO, JENNIFER N
• Application Number:
  US11/280,746
• Publication Number:
  US 20070109592A1
• Time in Patent Office: 2,198 Days
• Field of Search: 718/105, 718/102
• US Class Current: 718/102
• CPC Class Codes: G06F 9/5033 considering data affinity
• CPC Class Codes: H04L 51/00 User-to-user messaging in packet-switching networks, transmitted according to store-and-forward or real-time protocols, e.g. e-mail
• CPC Class Codes: H04L 67/1001 for accessing one among a plurality of replicated servers
• CPC Class Codes: H04L 67/1006 with static server selection, e.g. the same server being selected for a specific client
• CPC Class Codes: H04L 67/1029 using data related to the state of servers by a load balancer
• CPC Class Codes: H04L 67/1034 Reaction to server failures by a load balancer

Abstract

Method and apparatus for updating user data from multiple data sources in a data gateway are disclosed. The data gateway includes a persistent job table for centrally storing jobs to be processed by the plurality of servers, a server table for centrally storing statuses of activities of the plurality of servers, and a job engine for processing job requests from the multiple data sources using the persistent job table and the server table. The data gateway includes a server farm having a plurality of servers, and where both the persistent job table and the server table reside outside the plurality of servers. The data gateway further includes a virtual Internet Protocol load balancer for dynamically mapping of jobs to be processed to particular servers according to user identifiers.

Claims

1. A data gateway comprising:

a server farm comprising a plurality of servers;

a persistent job table storage unit for centrally storing a persistent job table of jobs to be processed by the plurality of servers, wherein a job comprises an update of user data from multiple data sources, and wherein the persistent job table storage unit resides outside the plurality of servers;

a server table storage unit for centrally storing a server table of statuses of activities of the plurality of servers, wherein the server table storage unit resides outside the plurality of servers; and

a job engine processor for processing job requests from the multiple data sources using the persistent job table and the server table, wherein the persistent job table further comprises;

a retry count for indicating number of times the data gateway has tried processing a job;

a due time for specifying a next time the job is due for processing; and

a reschedule flag indicating the data gateway has received another notification for the job that is currently being processed.

2. The data gateway of claim 1, wherein the persistent job table comprises:

user identifier information for identifying a user'"'"'s entries in the persistent job table;

data source type information for specifying types of data source accounts;

job type information for specifying gateway data synchronization jobs; and

server information for specifying a gateway server currently owning a particular job.

3. The data gateway of claim 2, wherein the data source type information comprises at least an element selected from the group consisting of mail, send mail, calendar, address book, task, and photo.

4. The data gateway of claim 1, wherein the server table comprises:

a list of servers and their corresponding schedule timestamps.

5. The data gateway of claim 1 further comprising:

a content router application interface for interfacing with a backend database; and

a data source application interface for interfacing with the multiple data sources, wherein the multiple data sources include IMAP, POP, WebDAV, and Exchange.

6. The data gateway of claim 5, wherein the content router application interface comprises:

a record exchange application interface for exchanging new mail and personal information manager information with a backend database;

a settings exchange application interface for exchanging folder list and meta-data information with the backend database; and

a command exchange application interface for exchanging file attachments with the backend database.

7. The data gateway of claim 5, wherein the data source application interface comprises:

an IMAP adaptor for communicating with at least an IMAP server;

an MSN adaptor for communicating with at least an MSN server;

a POP adaptor for communicating with at least a POP server; and

an Exchange adaptor for communicating with at least an Exchange server.

8. The data gateway of claim 1 further comprises:

a virtual Internet Protocol load balancer for dynamically mapping jobs to be processed to particular servers according to user identifiers.

9. A data gateway comprising:

a server farm comprising a plurality of servers;

a persistent job table storage unit for centrally storing a persistent job table of jobs to be processed by the plurality of servers, wherein a job comprises an update of user data from multiple data sources, and wherein the persistent job table storage unit resides outside the plurality of servers;

a server table storage unit for centrally storing a server table of statuses of activities of the plurality of servers, wherein the server table storage unit resides outside the plurality of servers; and

a job engine processor for processing job requests from the multiple data sources using the persistent job table and the server table, wherein the job engine processor comprises;

a job scheduler processor for scheduling job requests for updating user data;

a storage queue for storing jobs to be processed;

a job dispatcher for obtaining a lock for a job to be processed; and

a job handler processor for processing the job according to the lock and a data source type of the job.

10. The data gateway of claim 9, wherein the job engine processor further comprises:

a timer for periodically triggering job requests; and

a change notification handler for sending job requests from registered backend data sources and from user devices.

11. A method comprising:

receiving, via a data gateway, a job request, the data gateway comprising a server farm comprising a plurality of servers;

fetching, via the data gateway, jobs due for processing from a persistent job table, stored on a persistent jobs table storage unit, wherein the persistent job table stores jobs to be processed by the plurality of servers, and wherein the persistent job table storage unit resides outside the plurality of servers;

sending, via the data gateway, jobs to a storage queue;

rescheduling, via the data gateway, jobs in the persistent job table for processing at a future time

dispatching, via the data gateway, from the storage queue to a job handler processor a dispatched job to be processed;

processing, via the job handler processor, the dispatched job by the job handler processor according to a data source type of the dispatched job;

receiving, via the data gateway, a status from the job handler; and

rescheduling, via the data gateway, the job in the persistent job table in response to the job being a recurring job and the status indicating the job has been completed.

12. The method of claim 11 further comprising:

receiving, via the data gateway, a status from the job handler; and

removing, via the data gateway, the job from the persistent job table in response to the job being a one-time job and the status indicating the job has been completed.

13. The method of claim 11 further comprising:

receiving, via the data gateway, a status from the job handler; and

processing, via the data gateway, the job at a future time in response to the status indicating a busy-lock taken.

14. The method of claim 11, wherein dispatching a job further comprises:

obtaining, via the data gateway, a lock according to an account lock and a set of data type locks of the user'"'"'s account and data respectively.

15. The method of claim 11 further comprising:

updating, via the data gateway, a user account information by implementing a mutually exclusive write of the user account information.

16. The method of claim 15, wherein updating a user account information comprises:

provisioning, via the data gateway, the user account; and

updating, via the data gateway, the user account configurations.

17. The method of claim 15, wherein updating a user account information further comprises:

deleting, via the data gateway, the user account; and

deactivating, via the data gateway, the user account.

18. The method of claim 11 further comprising:

updating, via the data gateway, user data changes according to an account lock and a set of data type locks of the user'"'"'s data.

19. The method of claim 18, wherein updating user data changes comprises:

implementing, via the data gateway, a read lock on the account lock; and

implementing, via the data gateway, a write lock on a data type lock of the user data being updated.

20. A method comprising:

receiving, via a data gateway, a job request, the data gateway comprising a server farm comprising a plurality of servers;

fetching, via the data gateway, jobs due for processing from a persistent job table, stored on a persistent jobs table storage unit, wherein the persistent job table stores jobs to be processed by the plurality of servers, and wherein the persistent job table storage unit resides outside the plurality of servers;

sending, via the data gateway, jobs to a storage queue;

rescheduling, via the data gateway, jobs in the persistent job table for processing at a future time

dispatching, via the data gateway, from the storage queue to a job handler processor a dispatched job to be processed;

processing, via the job handler processor, the dispatched job by the job handler processor according to a data source type of the dispatched job;

receiving, via the data gateway, a status from the job handler;

rescheduling, via the data gateway, a job with a delay in the persistent job table in response to the status indicating a temporary error; and

incrementing a retry count in the persistent job table.

21. The method of claim 20 further comprising:

updating in a gateway server table a reschedule timestamp associated with a current server, wherein the gateway server table stores a list of servers and their corresponding schedule timestamps; and

rescheduling, via the data gateway, jobs in the persistent job table for processing at a future time.

22. The method of claim 21 further comprising:

retrieving, via the data gateway, a list of inactive servers that have not updated their corresponding reschedule timestamps within a timeout period; and

rescheduling, via the data gateway, jobs belonging to the inactive servers for processing.

23. The method of claim 22, wherein rescheduling comprises:

sending, via the data gateway, the jobs belonging to the inactive servers through a virtual Internet Protocol load balancer;

directing, via the data gateway, jobs to new servers for processing; and

directing, via the data gateway, subsequent jobs from the same users to the same servers for processing dynamically.

24. A non-transitory computer readable storage medium for tangibly storing thereon program code for execution by a processor, the program code comprising:

code for receiving a job request at a data gateway, the data gateway comprising a server farm comprising a plurality of servers;

code for fetching jobs due for processing from a persistent job table, stored in a persistent jobs table storage unit, wherein the persistent job table stores jobs to be processed by the plurality of servers, and wherein the persistent job table storage unit resides outside the plurality of servers;

code for sending jobs to a storage queue;

code for rescheduling jobs in the persistent job table for processing at a future time;

code for dispatching a job to be processed from the storage queue to a job handler processor;

code for processing the job by the job handler processor according to a data source type of the dispatched job;

code for receiving a status from the job handler; and

code for rescheduling the job in the persistent job table in response to the job being a recurring job and the status indicating the job has been completed.

25. The non-transitory computer readable storage medium of claim 24 further comprising:

code for receiving a status from the job handler; and

code for removing the job from the persistent job table in response to the job being a one-time job and the status indicating the job has been completed.

26. The non-transitory computer readable storage medium of claim 24 further comprising:

code for receiving a status from the job handler; and

code for processing the job at a future time in response to the status indicating a busy-lock taken.

27. The non-transitory computer readable storage medium of claim 24, wherein the code for dispatching a job further comprises:

code for obtaining a lock according to an account lock and a set of data type locks of the user'"'"'s account and data respectively.

28. The non-transitory computer readable storage medium of claim 24 further comprising:

code for updating user account information by implementing a mutually exclusive write of the user account information.

29. The non-transitory computer readable storage medium of claim 28, wherein the code for updating user account information comprises:

code for provisioning user account; and

code for updating user account configurations.

30. The non-transitory computer readable storage medium of claim 28, wherein the code for updating user account information further comprises:

code for deleting a user account; and

code for deactivating a user account.

31. The non-transitory computer readable storage medium of claim 24 further comprising:

code for updating changes to a user'"'"'s data according to an account lock and a set of data type locks of the user'"'"'s data.

32. The non-transitory computer readable storage medium of claim 31, wherein the code for updating user data changes comprises:

code for implementing a read lock on the account lock; and

code for implementing a write lock on a data type lock of the user'"'"'s data being updated.

33. A non-transitory computer readable storage medium for tangibly storing thereon program code for execution by a processor, the program code comprising:

code for receiving a job request at a data gateway, the data gateway comprising a server farm comprising a plurality of servers;

code for fetching jobs due for processing from a persistent job table, stored in a persistent jobs table storage unit, wherein the persistent job table stores jobs to be processed by the plurality of servers, and wherein the persistent job table storage unit resides outside the plurality of servers;

code for sending jobs to a storage queue;

code for rescheduling jobs in the persistent job table for processing at a future time;

code for dispatching a job to be processed from the storage queue to a job handler processor;

code for processing the job by the job handler processor according to a data source type of the dispatched job;

code for receiving a status from the job handler;

code for rescheduling the job with a delay in the persistent job table in response to the status indicating a temporary error; and

code for incrementing a retry count in the persistent job table.

34. The non-transitory computer readable storage medium of claim 33 further comprising:

code for updating a reschedule timestamp associated with a current server in a gateway server table, wherein the gateway server table stores a list of servers and their corresponding schedule timestamps; and

code for rescheduling a job in the persistent job table for processing at a future time.

35. The non-transitory computer readable storage medium of claim 34 further comprising:

code for retrieving a list of inactive servers that have not updated their corresponding reschedule timestamps within a timeout period; and

code for rescheduling jobs belonging to the inactive servers for processing.

36. The non-transitory computer readable storage medium of claim 35, wherein the code for rescheduling comprises:

code for sending the jobs belonging to the inactive servers through a virtual Internet Protocol load balancer; and

code for directing the jobs to new servers for processing; and

code for directing subsequent jobs from the same users to the same servers for processing dynamically.