Systems and methods for batch processing of data records in an on-demand system

• US 9,495,430 B2
• Filed: 08/09/2013
• Issued: 11/15/2016
• Est. Priority Date: 09/06/2012
• Status: Active Grant

• Current Assignee: Salesforce.com, Inc.
• Original Assignee: Salesforce.com, Inc.
• Inventors: Gupta, Varun, Rajendran, Raj, Ramineni, Navin K., Sethuraman, Priya, Gesin, Stephen
• Primary Examiner(s): Nguyen, Loan T
• Application Number:
  US13/963,541
• Publication Number:
  US 20140067760A1
• Time in Patent Office: 1,194 Days
• US Class Current: 1/1
• CPC Class Codes: G06F 16/20 of structured data, e.g. relational data
• CPC Class Codes: G06F 16/2372 Updates performed during offline database operations
• CPC Class Codes: G06F 16/27 Replication, distribution or synchronisation of data between databases or within a distributed database system; Distributed database system architectures therefor

Abstract

Systems and methods are provided for batch processing of data records in an on-demand system. A marker is stored that identifies a node in an on-demand system and identifies a most recent data record processed in a relational database management system by the node. The marker is used to query the database management system to identify a batch of records for the node to process. The batch of records is processed by the node to update a client of data changes reflected by the batch of records.

Claims

1. An apparatus for batch processing of data records in an on-demand system, the apparatus comprising:

a processor; and

a non-transitory computer readable medium storing a plurality of instructions which,

when executed by the processor, cause the processor to carry out the steps of;

storing a first marker that identifies a first node in an on-demand system and identifies a most recent data record processed in a relational database management system by the first node;

storing a second marker that identifies a second node in the on-demand system and identifies a most recent data record processed in the relational database management system by the second node;

using the first and second markers to query the database management system to identify a first batch of records for the first node to process and a second batch of records for the second node to process; and

processing the first batch of records by the first node to update a first client of data changes reflected by the first batch of records and the second batch of records by the second node to update a second client of data changes reflected by the second batch of records.

2. The apparatus of claim 1, wherein storing the first and second markers comprises storing the first and second markers in a transactions table associated with the relational database management system.

3. The apparatus of claim 1, wherein each of the first and second markers further identifies a data record type.

4. The apparatus of claim 1, wherein the first and second batches of records are each based on a maximum batch size.

5. A computer program product, comprising a non-transitory computer-readable medium carrying one or more sequences of instructions for batch processing of data records in an on-demand system, which instructions, when executed by one or more processors, cause the one or more processors to:

store a first marker that identifies a first node in an on-demand system and identifies a most recent data record processed in a relational database management system by the first node;

store a second marker that identifies a second node in the on-demand system and identifies a most recent data record processed in the relational database management system by the second node;

use the first and second markers to query the database management system to identify a first batch of records for the first node to process and a second batch of records for the second node to process; and

process the first batch of records by the first node to update a first client of data changes reflected by the first batch of records and the second batch of records by the second node to update a second client of data changes reflected by the second batch of records.

6. The computer program product of claim 5, wherein storing the first and second markers comprises storing the first and second markers in a transactions table associated with the relational database management system.

7. The computer program product of claim 5, wherein each of the first and second markers further identifies a data record type.

8. The computer program product of claim 5, wherein the first and second batches of records are each based on a maximum batch size.

9. A method for batch processing of data records in an on-demand system, the method comprising:

storing a first marker that identifies a first node in an on-demand system and identifies a most recent data record processed in a relational database management system by the first node;

storing a second marker that identifies a second node in the on-demand system and identifies a most recent data record processed in the relational database management system by the second node;

using the first and second markers to query the database management system to identify a first batch of records for the first node to process and a second batch of records for the second node to process; and

processing the first batch of records by the first node to update a first client of data changes reflected by the first batch of records and the second batch of records by the second node to update a second client of data changes reflected by the second batch of records.

10. The method of claim 9, wherein storing the first and second markers comprises storing the first and second markers in a transactions table associated with the relational database management system.

11. The method of claim 9, wherein each of the first and second markers further identifies a data record type.

12. The method of claim 9, wherein the first and second batches of records are each based on a maximum batch size.

13. A method for transmitting code for batch processing of data records in an on-demand system on a transmission medium, the method comprising:

transmitting code to store a first marker that identifies a first node in an on-demand system and identifies a most recent data record processed in a relational database management system by the first node;

transmitting code to store a second marker that identifies a second node in the on-demand system and identifies a most recent data record processed in the relational database management system by the second node;

transmitting code to use the first and second markers to query the database management system to identify a first batch of records for the first node to process and a second batch of records for the second node to process; and

transmitting code to process the first batch of records by the first node to update a first client of data changes reflected by the first batch of records and the second batch of records by the second node to update a second client of data changes reflected by the second batch of records.

14. The method for transmitting code of claim 13, wherein storing the first and second markers comprises storing the first and second markers in a transactions table associated with the relational database management system.

15. The method for transmitting code of claim 13, wherein each of the first and second markers further identifies a data record type.

16. The method for transmitting code of claim 13, wherein the first and second batches of records are each based on a maximum batch size.