Methods and systems for batch processing in an on-demand service environment
First Claim
1. A method comprising:
- receiving a processing request for a multi-tenant database, wherein the processing request specifies processing logic and a processing target group within the multi-tenant database;
dividing the processing target group into a plurality of processing target sub-groups;
queuing the processing request with a batch processing queue for the multi-tenant database among a plurality of previously queued processing requests;
releasing each of the plurality of processing target sub-groups for processing in the multi-tenant database via the processing logic at one or more times specified by the batch processing queue, the releasing being subject to one or more release limits enforced by a batch processing governor;
wherein violation of any of the one or more release limits causes the batch processing governor to throttle releases of batch processing jobs to the multi-tenant database; and
wherein the one or more release limits are selected from the group comprising;
(i) a maximum amount of memory permissibly allocated to each release, (ii) a maximum number of database statements permissibly executable with each release, (iii) a maximum number of permissible relational database row transactions for each release, (iv) a maximum number of permissible non-relational database object transactions for each release, (v) a maximum amount of database heap permissibly allocated or consumed by each release, (vi) a maximum total number of aggregate transactions by any one customer organization within a specified time period, (vii) a maximum total number of aggregate releases for batch processing jobs by any one customer organization within a specified time period, (viii) a maximum number of concurrent batch processing jobs releasable to the multi-tenant database, and (ix) a maximum current workload of the multi-tenant database.
1 Assignment
0 Petitions
Accused Products
Abstract
In accordance with embodiments disclosed herein, there are provided mechanisms and methods for batch processing in an on-demand service environment. For example, in one embodiment, mechanisms include receiving a processing request for a multi-tenant database, in which the processing request specifies processing logic and a processing target group within the multi-tenant database. Such an embodiment further includes dividing or chunking the processing target group into a plurality of processing target sub-groups, queuing the processing request with a batch processing queue for the multi-tenant database among a plurality of previously queued processing requests, and releasing each of the plurality of processing target sub-groups for processing in the multi-tenant database via the processing logic at one or more times specified by the batch processing queue.
126 Citations
17 Claims
-
1. A method comprising:
-
receiving a processing request for a multi-tenant database, wherein the processing request specifies processing logic and a processing target group within the multi-tenant database; dividing the processing target group into a plurality of processing target sub-groups; queuing the processing request with a batch processing queue for the multi-tenant database among a plurality of previously queued processing requests; releasing each of the plurality of processing target sub-groups for processing in the multi-tenant database via the processing logic at one or more times specified by the batch processing queue, the releasing being subject to one or more release limits enforced by a batch processing governor; wherein violation of any of the one or more release limits causes the batch processing governor to throttle releases of batch processing jobs to the multi-tenant database; and wherein the one or more release limits are selected from the group comprising;
(i) a maximum amount of memory permissibly allocated to each release, (ii) a maximum number of database statements permissibly executable with each release, (iii) a maximum number of permissible relational database row transactions for each release, (iv) a maximum number of permissible non-relational database object transactions for each release, (v) a maximum amount of database heap permissibly allocated or consumed by each release, (vi) a maximum total number of aggregate transactions by any one customer organization within a specified time period, (vii) a maximum total number of aggregate releases for batch processing jobs by any one customer organization within a specified time period, (viii) a maximum number of concurrent batch processing jobs releasable to the multi-tenant database, and (ix) a maximum current workload of the multi-tenant database. - View Dependent Claims (2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12)
-
-
13. A method comprising:
-
receiving a processing request for a multi-tenant database, wherein the processing request specifies processing logic and a processing target group within the multi-tenant database; dividing the processing target group into a plurality of processing target sub-groups; queuing the processing request with a batch processing queue for the multi-tenant database among a plurality of previously queued processing requests; releasing each of the plurality of processing target sub-groups for processing in the multi-tenant database via the processing logic at one or more times specified by the batch processing queue; sending a completion notification to the originator of the processing request, wherein the completion notification comprises a completion report specifying;
(i) each of the plurality of processing target sub-groups released for processing in the multi-tenant database, and (ii) a completion status for each of the plurality of processing target sub-groups indicating a successful or unsuccessful completion for each processing target sub-group; andwherein each completion status comprises a status selected from the group comprising;
(i) complete and committed without error, (ii) complete and committed after re-try due to excessive use of resources, (iii) complete and committed after re-try due to excessive multi-tenant database workload, and (iv) abort due to one or more errors for the respective processing target sub-group, wherein any database transaction error for the respective processing target sub-group prevents committing transaction associated with the respective processing target sub-group to the multi-tenant database.
-
-
14. A method comprising:
-
receiving a processing request for a multi-tenant database, wherein the processing request specifies processing logic and a processing target group within the multi-tenant database; dividing the processing target group into a plurality of processing target sub-groups, wherein the dividing comprises (i) determining a total size of the processing target group, and (ii) dividing the processing target group into the one or more processing target sub-groups based on a size threshold, wherein the one or more processing target sub-groups do not overlap, and further wherein the one or more processing target sub-groups correspond in size to the determined total size; queuing the processing request with a batch processing queue for the multi-tenant database among a plurality of previously queued processing requests; releasing each of the plurality of processing target sub-groups for processing in the multi-tenant database via the processing logic at one or more times specified by the batch processing queue; and wherein the total size of the processing target group corresponds to a criterion selected from the group comprising;
(i) a total number of records in the multi-tenant database to be processed in accordance with the processing request, (ii) a total number of rows in the multi-tenant database to be processed in accordance with the processing request, (iii) a total number of objects in the multi-tenant database to be processed in accordance with the processing request, and (iv) a total number of database statements to be transacted against the multi-tenant database in accordance with the processing request.
-
-
15. Non-transitory computer readable storage media having instructions stored thereon that, when executed by a system in a host organization having a processor and memory therein, the instructions cause the system to perform operations comprising:
-
receiving a processing request for a multi-tenant database, wherein the processing request specifies processing logic and a processing target group within the multi-tenant database; dividing the processing target group into a plurality of processing target sub-groups; queuing the processing request with a batch processing queue for the multi-tenant database among a plurality of previously queued processing requests; releasing each of the plurality of processing target sub-groups for processing in the multi-tenant database via the processing logic at one or more times specified by the batch processing queue, the releasing being subject to one or more release limits enforced by a batch processing governor; wherein violation of any of the one or more release limits causes the batch processing governor to throttle releases of batch processing jobs to the multi-tenant database; and wherein the one or more release limits are selected from the group comprising;
(i) a maximum amount of memory permissibly allocated to each release, (ii) a maximum number of database statements permissibly executable with each release, (iii) a maximum number of permissible relational database row transactions for each release, (iv) a maximum number of permissible non-relational database object transactions for each release, (v) a maximum amount of database heap permissibly allocated or consumed by each release, (vi) a maximum total number of aggregate transactions by any one customer organization within a specified time period, (vii) a maximum total number of aggregate releases for batch processing jobs by any one customer organization within a specified time period, (viii) a maximum number of concurrent batch processing jobs releasable to the multi-tenant database, and (ix) a maximum current workload of the multi-tenant database. - View Dependent Claims (16)
-
-
17. A system comprising:
-
a processor and memory to execute instructions; a request interface to receive a processing request for a multi-tenant database communicably interfaced with the system, wherein the processing request to specify processing logic and a processing target group within the multi-tenant database; a request chunker to chunk the processing target group into a plurality of processing target sub-groups; a batch processing queue to queue the processing request for release to the multi-tenant database; and a release agent to release each of the plurality of processing target sub-groups for processing in the multi-tenant database via the processing logic; a batch processing governor to enforce one or more release limits for releases by the release agent, wherein violation of any of the one or more release limits causes the batch processing governor to throttle releases of batch processing jobs to the multi-tenant database; and wherein the one or more release limits are selected from the group comprising;
(i) a maximum amount of memory permissibly allocated to each release, (ii) a maximum number of database statements permissibly executable with each release, (iii) a maximum number of permissible relational database row transactions for each release, (iv) a maximum number of permissible non-relational database object transactions for each release, (v) a maximum amount of database heap permissibly allocated or consumed by each release, (vi) a maximum total number of aggregate transactions by any one customer organization within a specified time period, (vii) a maximum total number of aggregate releases for batch processing jobs by any one customer organization within a specified time period, (viii) a maximum number of concurrent batch processing jobs releasable to the multi-tenant database, and (ix) a maximum current workload of the multi-tenant database.
-
Specification