Systems and methods for supporting transactional message handling

• US 9,519,547 B2
• Filed: 09/16/2014
• Issued: 12/13/2016
• Est. Priority Date: 06/24/2011
• Status: Active Grant

• Current Assignee: Salesforce.com, Inc.
• Original Assignee: Salesforce.com, Inc.
• Inventors: Devadhar, Vijayanth
• Primary Examiner(s): LIAO, JASON G
• Application Number:
  US14/488,159
• Publication Number:
  US 20150081645A1
• Time in Patent Office: 819 Days
• Field of Search: None
• US Class Current: 1/1
• CPC Class Codes: G06F 11/1446 Point-in-time backing up or restoration of persistent data
• CPC Class Codes: G06F 16/2343 Locking methods, e.g. distributed locking or locking implementation details
• CPC Class Codes: G06F 9/466 Transaction processing

Abstract

In accordance with embodiments disclosed herein, there are provided methods, systems, and apparatuses including, for supporting transactional message handling in an on-demand service environment including, for example: enqueuing a message specifying a transaction to be processed via a host organization; inserting a row into a database of the host organization associating the message with a status of pending, wherein the row is autocommitted to the database upon insertion; updating the status for the row to ready if a commit operation for the transaction is initiated; requesting a lock on the row; and performing final processing for the transaction based on the status for the message and based further on whether the lock is obtained for the row. Final processing may include, for example, a transaction roll back, a transaction commit, a transaction requeue, a termination of transaction processing, or an orphaned transaction clean up.

Claims

1. A method performed by a system having at least a processor and a memory therein, the method comprising:

enqueuing a message into a message queue framework, the message specifying a transaction requiring at least a row to be inserted into a database system;

inserting the row into the database system, wherein a status for the row inserted into the database system remains pending in the message queue framework absent initiation of a commit operation for the row;

initiating the commit operation for the row inserted into the database system upon insertion of the row into the database system regardless of whether any further processing occurs;

requesting a lock on the row inserted into the database system;

determining the transaction failed resulting in the enqueued message becoming an orphaned transaction within the message queue framework which will not be consumed, wherein determining the transaction failed is based on whether the lock requested on the row is obtained or denied and based further on whether the status for the row is pending or ready in the message queue framework; and

performing an orphaned transaction clean up operation including at least discarding the enqueued message by removing the row from the database system and committing the deletion of the row from the database system; and

wherein the database system embodies a multi-tenant database system, the multi-tenant database system having elements of hardware and software that are shared by a plurality of separate and distinct customer organizations, each of the separate and distinct customer organizations being remotely located from the multi-tenant database system.

2. The method of claim 1, wherein determining the transaction failed or processed successfully based on whether the status for the row is pending or ready in the message queue framework includes determining whether the enqueued message has been orphaned and will not be consumed based on whether the lock on the row is available, whether the status for the row is pending, and whether the enqueued message has exceeded a time threshold.

3. The method of claim 1, wherein the orphaned message remains enqueued within the message queue framework which will not be consumed;

and

wherein performing the orphaned transaction clean up operation includes removing the orphaned message from the message queue framework.

4. The method of claim 3, wherein performing the orphaned transaction clean up operation includes initiating a clean up operation for the orphaned message remaining enqueued within the message queue framework specifying the transaction, the clean up operation requiring at least the row to be inserted into the database system based on the status for the row remaining pending in the message queue framework.

5. The method of claim 1:

wherein initiating, via the message queue framework, the commit operation for the row inserted into the database upon insertion of the row into the database regardless of whether any further processing occurs comprises the message queue framework implementing an out-of-band commit operation; and

wherein determining the transaction failed or processed successfully comprises concluding the transaction failed based on the row insert not successfully committing to the database system responsive to the out-of-band commit operation for the row and retaining a status of pending for the message in the message queue framework.

6. The method of claim 1, further comprising:

performing a first determination that the lock on the row is obtained;

performing a second determination that the status for the row is pending and not yet committed; and

rolling back the transaction and discarding the message by removing the row from the database system based on the first and second determinations.

7. The method of claim 6, wherein the first and second determinations collectively indicate the message does not need to be processed by the message queue framework because processing of the transaction has failed.

8. The method of claim 1, further comprising for a second message enqueued in the message queue framework specifying a second transaction requiring at least a second row to be inserted into a database system:

performing a first determination that the lock on the second row is obtained;

performing a second determination that the status for the second row is ready and committed; and

processing the second message by committing a removal of the second row from the database system.

9. The method of claim 8:

wherein the first and second determinations collectively indicate the processing of the second transaction has completed successfully, the second row is committed to the database system, and the second message may be processed; and

wherein processing the second message by committing a removal of the second row from the database system comprises;

(a) marking the second row for deletion, and (b) committing the deletion of the second row to maintain transactional atomicity.

10. The method of claim 1, further comprising for a second message enqueued in the message queue framework specifying a second transaction requiring at least a second row to be inserted into a database system:

performing a first determination that the lock on the second row is denied;

performing a second determination that the status for the second row is pending;

requeuing the second message with a time delay, wherein requeuing the second message schedules the system to reinitiate processing for the second transaction after the time delay; and

wherein the first and second determinations collectively indicate that the processing of the second transaction is active but not complete due to the second row being uncommitted and further that the second message is not yet ready to be processed.

11. The method of claim 10:

wherein requeuing the second message with a time delay enables an application server processing the second transaction to;

(a) maintain exclusive authority for processing the second transaction;

(b) complete processing of the second transaction; and

(c) initiate the commit operation, wherein the commit operation comprises the application server;

(i) marking the second row for deletion and (ii) simultaneously committing the second row inserted and committing the deletion of the second row to maintain transactional atomicity;

wherein the system reinitiates processing for the second transaction after the time delay subsequent to requeuing the second message into the message queue framework; and

wherein the reinitiated processing for the second transaction after the time delay is terminated contingent on a determination that the second row has been deleted from the database system.

12. The method of claim 1, wherein the method further comprises for a second message enqueued in the message queue framework specifying a second transaction requiring at least a second row to be inserted into a database system:

determining that the status of the second row inserted is non-existent and that the lock cannot be obtained because the second row does not exist in the database system;

dequeuing the second message from the message queue framework to prevent further processing associated with the second transaction specified by the second message;

wherein determining that the status is non-existent and that the lock cannot be obtained because the second row does not exist in the database system collectively indicates that the processing of the second transaction has previously completed successfully resulting in the second transaction being committed successfully and the second row being deleted successfully; and

determining the enqueued second message specifying the second transaction to be processed is an erroneous message.

13. The method of claim 1, further comprising:

performing a first determination that the lock on the row is available;

performing a second determination that the status for the row is pending;

performing a third determination that the enqueued message has exceeded a time threshold; and

wherein performing the orphaned transaction clean up operation comprises initiating a clean up operation for the enqueued message based on the first, second, and third determinations;

wherein the first, second, and third determinations collectively indicate that the enqueued message has been orphaned and that it will not be consumed; and

wherein the clean up operation comprises discarding the message by removing the row from the database system and committing the deletion of the row to the database system.

14. The method of claim 1, further comprising for a second message enqueued in the message queue framework specifying a second transaction requiring at least a second row to be inserted into a database system:

performing a first determination that the lock on the second row is not available;

performing a second determination that the status for the second row is pending;

performing a third determination that the enqueued second message has exceeded a time threshold; and

skipping a clean up operation for the enqueued second message based on the first, second, and third determinations; and

wherein the first, second, and third determinations collectively indicate that the second transaction associated with the enqueued second message is actively processing and requires a longer than expected processing period in excess of the time threshold.

15. The method of claim 1, further comprising:

initiating a clean up operation on a repeating periodic basis, wherein the clean up operation comprises;

identifying a set of one or more enqueued orphan messages based on each of the one or more enqueued orphan messages meeting the following criteria;

determining that a row corresponding to the enqueued orphan message is not locked;

determining the row corresponding to the enqueued orphan message has a status of pending; and

determining the enqueued orphan message has been enqueued for a time period in excess of a threshold.

16. The method of claim 1, further comprising:

receiving the message at the message queue framework of the system from a message broker;

wherein inserting the row into a database system of the system comprises the message queue framework inserting the row into the database system and causing the database system to autocommit the row upon insertion;

wherein the message broker lacks native transactional atomicity;

wherein receiving the message at the message queue framework comprises intercepting a message enqueued at an enqueue Application Programming Interface (enqueue API) of the message broker and triggering the row insertion responsive to intercepting the message;

dispatching the message to a worker thread to process the transaction specified by the message;

wherein requesting the lock on the row comprises the worker thread requesting the lock; and

wherein the worker thread performs at least the following operations;

(i) transacting the commit operation for the row inserted with the database system and updating the status for the row from pending to ready responsive to successfully committing the row, (ii) marking the row for deletion, and (iii) committing deletion of the row from the database system;

wherein dispatching the message to a worker comprises communicating the message to the worker via a message transport to perform its operations as an execution thread within one of a plurality of application servers within the system.

17. The method of claim 1:

wherein enqueuing the message specifying a transaction to be processed via the system comprises enqueuing the message at the system as one of a plurality of requests received from a plurality of customer organizations, wherein each customer organization is an entity selected from the group consisting of;

a separate and distinct remote organization having network based access to the multi-tenant database system, an organizational group having network based access to the multi-tenant database system, a business partner having network based access to the multi-tenant database system, or a customer organization that subscribes to cloud computing services provided the multi-tenant database system via a public Internet.

18. A non-transitory computer readable storage medium having instructions stored thereon that, when executed by a system having a processor and memory therein, the instructions cause the system to perform operations comprising:

enqueuing a message into a message queue framework, the message specifying a transaction requiring at least a row to be inserted into a database system;

inserting the row into the database system, wherein a status for the row inserted into the database system remains pending in the message queue framework absent initiation of a commit operation for the row;

initiating the commit operation for the row inserted into the database system upon insertion of the row into the database system regardless of whether any further processing occurs;

requesting a lock on the row inserted into the database system;

determining the transaction failed resulting in the enqueued message becoming an orphaned transaction within the message queue framework which will not be consumed, wherein determining the transaction failed is based on whether the lock requested on the row is obtained or denied and based further on whether the status for the row is pending or ready in the message queue framework; and

performing an orphaned transaction clean up operation including at least discarding the enqueued message by removing the row from the database system and committing the deletion of the row from the database system; and

wherein the database system embodies a multi-tenant database system, the multi-tenant database system having elements of hardware and software that are shared by a plurality of separate and distinct customer organizations, each of the separate and distinct customer organizations being remotely located from the multi-tenant database system.

19. The non-transitory computer readable storage medium of claim 18, wherein the orphaned message remains enqueued within the message queue framework which will not be consumed;

and

wherein performing the orphaned transaction clean up operation includes removing the orphaned message from the message queue framework.

20. A system comprising:

a processor to execute instructions;

a request interface to receive a request from a client device;

a message broker to enqueue a message into a message queue framework, the message specifying a transaction requiring at least a row to be inserted into a database system;

the message queue framework to insert the row into the database system, wherein a status for the row inserted into the database system remains pending in the message queue framework absent initiation of a commit operation for the row;

the message queue framework to initiate the commit operation for the row inserted into the database system upon insertion of the row into the database system regardless of whether any further processing occurs;

a worker thread to request a lock on the row inserted into the database system;

the worker thread to determine the transaction failed resulting in the enqueued message becoming an orphaned transaction within the message queue framework which will not be consumed, wherein determining the transaction failed is based on whether the lock requested on the row is obtained or denied and based further on whether the status for the row is pending or ready in the message queue framework; and

the message queue framework to perform an orphaned transaction clean up operation including at least discarding the enqueued message by removing the row from the database system and committing the deletion of the row from the database system; and

wherein the database system embodies a multi-tenant database system, the multi-tenant database system having elements of hardware and software that are shared by a plurality of separate and distinct customer organizations, each of the separate and distinct customer organizations being remotely located from the multi-tenant database system.