Techniques for implementing batch processing in a database system

• US 10,007,576 B2
• Filed: 02/29/2016
• Issued: 06/26/2018
• Est. Priority Date: 08/25/2008
• Status: Active Grant

• Current Assignee: Salesforce.com, Inc.
• Original Assignee: Salesforce.com, Inc.
• Inventors: Yancey, Scott, Smith, Andrew, Waite, Andrew, Weissman, Craig
• Primary Examiner(s): Casanova, Jorge A
• Application Number:
  US15/056,764
• Publication Number:
  US 20160253243A1
• Time in Patent Office: 848 Days
• CPC Class Codes: G06F 11/1451 by selection of backup contents
• CPC Class Codes: G06F 16/2365 Ensuring data consistency and integrity
• CPC Class Codes: G06F 16/2372 Updates performed during offline database operations
• CPC Class Codes: G06F 16/273 Asynchronous replication or reconciliation

Abstract

In accordance with embodiments, there are provided techniques for implementing batch processing in a multi-tenant database system. These techniques for implementing batch processing in a multi-tenant database system may enable embodiments to provide great flexibility to a tenant of the architecture to perform desired functions on content of the database while allowing the owner of the architecture schedule the processing of the functions on the content.

Claims

1. A method for implementing asynchronous transaction processing in a database system having at least a processor and a memory therein to execute instructions for the asynchronous transaction processing, wherein the method comprises:

performing online transaction processing (OLTP) of database transactions for each of a plurality of users of the database system, wherein the OLTP processing provides an uninterrupted interaction for any database transaction submitted by users to the database system for OLTP processing;

executing a code interface, via the processor, wherein the code interface is to receive requests from the users of the database system for asynchronous processing, wherein the asynchronous processing requires at least manipulation of a plurality of database objects within the database system asynchronously from the OLTP processing of the database system;

receiving a request for asynchronous processing against the database system via the code interface from a user of the database system;

enqueuing the request for asynchronous processing in a request queue;

scheduling execution of the request for asynchronous processing in an execute queue;

executing the request for asynchronous processing against the database system pursuant to the scheduled execution; and

sending notification to the user of the database system indicating completion of the execution of the request for asynchronous processing against the database system.

2. The method of claim 1:

wherein scheduling execution of the request for asynchronous processing in the execute queue comprises;

calling a query locator object that points to a maximum number of objects for asynchronous processing in the request queue to be scheduled for uninterrupted processing against the database system to ensure that the processing does not consume in excess of a threshold amount of resources of the database system;

retrieving the maximum number of objects based on the called query locator object; and

scheduling execution of the maximum number of objects for asynchronous processing in the request queue by queuing the scheduled objects in the execute queue amongst a plurality of other objects in the execute queue.

3. The method of claim 1:

wherein executing the code interface asynchronously with operations of the database system to fulfill the received requests comprises invoking multiple methods providing an asynchronous processing facility to the tenant.

4. The method of claim 1, further comprising:

calling a query locator object that points to a maximum number of objects to be scheduled for uninterrupted processing in a single batch process against the database system to ensure that the single batch process does not consume in excess of a threshold amount of resources of the database system; and

wherein scheduling execution of the request for asynchronous processing in the execute queue amongst a plurality of other objects in the execute queue comprises retrieving the maximum number of objects based on the called query locator object.

5. The method of claim 4, further comprising:

invoking multiple methods in fulfillment of executing the request for asynchronous processing against the database system;

wherein one of the multiple methods calls the query locator object to identify multiple subgroups of the other objects in the execute queue to process in fulfillment of executing the request for asynchronous processing against the database system pursuant to the scheduled execution amongst the plurality of other objects in the execute queue.

6. The method of claim 5, wherein executing the multiple methods comprises sequentially processing the identified multiple subgroups of the other objects in the execute queue.

7. The method of claim 1:

wherein scheduling execution of the request for asynchronous processing in the execute queue amongst the plurality of other objects in the execute queue comprises scheduling execution of the request for asynchronous processing, wherein the request is to be executed via uninterrupted processing during a single batch process by the database system.

8. The method of claim 1:

wherein the database system comprises a multi-tenant on-demand database system having elements of hardware and software shared multiple user tenants of the database system;

wherein receiving the request for asynchronous processing against the database system via the code interface from the user of the database system comprises receiving the request from a user tenant of the multi-tenant on-demand database system; and

wherein the request from the tenant of the multi-tenant on-demand database system is enqueued in the execute queue amongst a plurality of other requests associated with one or more other user tenants of the multi-tenant on-demand database system.

9. The method of claim 1, further comprising:

wherein executing the request for asynchronous processing against the database system pursuant to the scheduled execution comprises executing multiple processes via multiple threads of execution.

10. The method of claim 1, further comprising:

calling a query locator object that points to a maximum number of objects to be scheduled for uninterrupted processing in a single batch process against the database system;

wherein a START method identifies the maximum number of objects to be scheduled for uninterrupted processing as a full set of objects in fulfillment of executing the request for asynchronous processing against the database system amongst the plurality of other objects in the execute queue;

wherein scheduling execution of the batch request to be executed comprises identifying objects to be subjected to an EXECUTE method; and

wherein identifying the objects comprises iteratively calling a QUERYMORE method to determine whether additional sub-groups of objects are to be included in a set of objects to be subjected to the EXECUTE method until it is determined that no additional sub-groups of objects are to be subjected to the EXECUTE method.

11. The method of claim 1:

wherein a QueryLocator object points to a maximum number of objects to be scheduled for uninterrupted processing via a server side database cursor persisted on architecture implementing the database system, wherein the maximum number of objects to be scheduled includes at least a portion of objects to be processed from the execute queue in fulfillment of the request for asynchronous processing;

wherein the maximum number of objects to be scheduled for uninterrupted processing comprises a quantity defined via the architecture based at least in part on processing capacity of the architecture and a priori information concerning consumption history of the resources of the architecture; and

wherein the maximum number of objects to be scheduled for uninterrupted processing is determined at least in part by dynamically monitoring other processes executing via the architecture and including the a priori information about particular times during which consumption of resources of the architecture has historically been reduced.

12. Non-transitory computer readable storage media having instructions stored thereupon for asynchronous transaction processing that, when executed by a processor of an application hosting service, the processor causes the application hosting service to perform operations comprising:

performing online transaction processing (OLTP) of database transactions for each of a plurality of users of the database system, wherein the OLTP processing provides an uninterrupted interaction for any database transaction submitted by users to the database system for OLTP processing;

executing a code interface, via the processor, wherein the code interface is to receive requests from the users of the database system for asynchronous processing, wherein the asynchronous processing requires at least manipulation of a plurality of database objects within the database system asynchronously from the OLTP processing of the database system;

receiving a request for asynchronous processing against the database system via the code interface from a user of the database system;

enqueuing the request for asynchronous processing in a request queue;

scheduling execution of the request for asynchronous processing in an execute queue;

executing the request for asynchronous processing against the database system pursuant to the scheduled execution; and

sending notification to the user of the database system indicating completion of the execution of the request for asynchronous processing against the database system.

13. The non-transitory computer readable storage media of claim 12:

wherein scheduling execution of the request for asynchronous processing in the execute queue comprises;

calling a query locator object that points to a maximum number of objects for asynchronous processing in the request queue to be scheduled for uninterrupted processing against the database system to ensure that the processing does not consume in excess of a threshold amount of resources of the database system;

retrieving the maximum number of objects based on the called query locator object; and

scheduling execution of the maximum number of objects for asynchronous processing in the request queue by queuing the scheduled objects in the execute queue amongst a plurality of other objects in the execute queue.

14. The non-transitory computer readable storage media of claim 12:

wherein executing the code interface asynchronously with operations of the database system to fulfill the received requests comprises invoking multiple methods providing an asynchronous processing facility to the tenant.

15. The non-transitory computer readable storage media of claim 12, wherein the instructions cause the application hosting service to perform operations further comprising:

calling a query locator object that points to a maximum number of objects to be scheduled for uninterrupted processing in a single batch process against the database system to ensure that the single batch process does not consume in excess of a threshold amount of resources of the database system, wherein scheduling execution of the request for asynchronous processing in the execute queue amongst a plurality of other objects in the execute queue comprises retrieving the maximum number of objects based on the called query locator object; and

invoking multiple methods in fulfillment of executing the request for asynchronous processing against the database system, wherein one of the multiple methods calls the query locator object to identify multiple subgroups of the other objects in the execute queue to process in fulfillment of executing the request for asynchronous processing against the database system pursuant to the scheduled execution amongst the plurality of other objects in the execute queue.

16. The non-transitory computer readable storage media of claim 12:

wherein the database system comprises a multi-tenant on-demand database system having elements of hardware and software shared multiple user tenants of the database system;

wherein receiving the request for asynchronous processing against the database system via the code interface from the user of the database system comprises receiving the request from a user tenant of the multi-tenant on-demand database system; and

wherein the request from the tenant of the multi-tenant on-demand database system is enqueued in the execute queue amongst a plurality of other requests associated with one or more other user tenants of the multi-tenant on-demand database system.

17. A system comprising:

a processor; and

a memory;

a database system;

wherein the database system is to perform online transaction processing (OLTP) of database transactions for each of a plurality of users of the database system, wherein the OLTP processing provides an uninterrupted interaction for any database transaction submitted by users to the database system for OLTP processing;

a code interface to receive requests from the users of the database system for asynchronous processing, wherein the asynchronous processing requires at least manipulation of a plurality of database objects within the database system asynchronously from the OLTP processing of the database system;

the code interface to execute asynchronously with operations of the database system to fulfill the received requests;

a receive interface to receive a request for asynchronous processing against the database system via the code interface from a user of the database system;

a request queue to enqueue the received request for asynchronous processing;

an execute queue to schedule execution of the request for asynchronous processing;

an EXECUTE method to execute the request for asynchronous processing against the database system pursuant to the scheduled execution; and

a FINISH method to send notification to the user of the database system indicating completion of the execution of the request for asynchronous processing against the database system.

18. The system of claim 17:

wherein the code interface is to invoke multiple threads to provide an asynchronous processing facility to the user.

19. The system of claim 17, further comprising:

a query locator object to point to a maximum number of objects for asynchronous processing in the request queue to be scheduled for uninterrupted processing against the database system to ensure that the processing does not consume in excess of a threshold amount of resources of the database system;

wherein the code interface is to retrieve the maximum number of objects based on the called query locator object schedule execution of the maximum number of objects for asynchronous processing in the request queue by queuing the scheduled objects in the execute queue amongst a plurality of other objects in the execute queue.

20. The system of claim 17:

wherein the database system comprises a multi-tenant on-demand database system having elements of hardware and software shared multiple user tenants of the database system;

wherein the receive request is to receive the request from a user tenant of the multi-tenant on-demand database system; and

wherein the request from the tenant of the multi-tenant on-demand database system is enqueued in the execute queue amongst a plurality of other requests associated with one or more other user tenants of the multi-tenant on-demand database system.