Systems and methods for supporting transactional message handling

• US 8,838,526 B2
• Filed: 11/01/2011
• Issued: 09/16/2014
• Est. Priority Date: 06/24/2011
• Status: Active Grant

• Current Assignee: Salesforce.com, Inc.
• Original Assignee: Salesforce.com, Inc.
• Inventors: Devadhar, Vijayanth
• Primary Examiner(s): LIAO, JASON G
• Application Number:
  US13/286,726
• Publication Number:
  US 20120330913A1
• Time in Patent Office: 1,050 Days
• Field of Search: None
• US Class Current: 707/607
• CPC Class Codes: G06F 11/1446 Point-in-time backing up or restoration of persistent data
• CPC Class Codes: G06F 16/2343 Locking methods, e.g. distributed locking or locking implementation details
• CPC Class Codes: G06F 9/466 Transaction processing

Abstract

In accordance with embodiments disclosed herein, there are provided methods, systems, and apparatuses including, for supporting transactional message handling in an on-demand service environment including, for example: enqueuing a message specifying a transaction to be processed via a host organization; inserting a row into a database of the host organization associating the message with a status of pending, wherein the row is autocommitted to the database upon insertion; updating the status for the row to ready if a commit operation for the transaction is initiated; requesting a lock on the row; and performing final processing for the transaction based on the status for the message and based further on whether the lock is obtained for the row. Final processing may include, for example, a transaction roll back, a transaction commit, a transaction requeue, a termination of transaction processing, or an orphaned transaction clean up.

Claims

1. A method in a host organization having at least a processor and a memory therein, the method comprising:

enqueuing a message into a message queue framework of the host organization, the message specifying a transaction to be processed via the host organization, the transaction comprising at least a row to be inserted into a database of the host organization;

inserting the row into the database of the host organization via the message queue framework, wherein a status for the row inserted into the database remains pending absent initiation of a commit operation for the row;

initiating, via the message queue framework, the commit operation for the row inserted into the database upon insertion of the row into the database regardless of whether any further processing occurs;

requesting a lock on the row inserted into the database;

concluding the transaction failed or processed successfully based on whether the lock requested on the row is obtained or denied and based further on whether the status for the row is pending or ready by determining whether the enqueued message has been orphaned and will not be consumed based on whether the lock on the row is available, whether the status for the row is pending, and whether the enqueued message has exceeded a time threshold; and

initiating a clean up operation for the enqueued message based on the determining whether the enqueued message has been orphaned and will not be consumed, wherein the clean up operation comprises discarding the message by removing the row from the database and committing the deletion of the row to the database.

2. The method of claim 1:

wherein initiating, via the message queue framework, the commit operation for the row inserted into the database upon insertion of the row into the database regardless of whether any further processing occurs comprises the message queue framework implementing an out-of-band commit operation; and

wherein concluding the transaction failed comprises the row not successfully committing to the database responsive to the out-of-band commit operation and retaining a status of pending.

3. The method of claim 1, further comprising:

performing a first determination that the lock on the row is obtained;

performing a second determination that the status for the row is pending and not yet committed; and

rolling back the transaction and discarding the message by removing the row from the database based on the first and second determinations.

4. The method of claim 3, wherein the first and second determinations collectively indicate the message does not need to be processed because processing of the transaction has failed.

5. The method of claim 1, further comprising:

performing a first determination that the lock on the row is obtained;

performing a second determination that the status for the row is ready and committed; and

processing the message by committing a removal of the row from the database.

6. The method of claim 5:

wherein the first and second determinations collectively indicate the processing of the transaction has completed successfully, the row is committed to the database, and the message may be processed; and

wherein processing the message by committing a removal of the row from the database comprises;

(a) marking the row for deletion, and

(b) committing the deletion of the row to maintain transactional atomicity.

7. The method of claim 1, further comprising:

performing a first determination that the lock on the row is denied;

performing a second determination that the status for the row is pending; and

requeuing the message with a time delay, wherein requeuing the message schedules the host organization to reinitiate processing for the transaction after the time delay.

8. The method of claim 7:

wherein requeuing the message with a time delay enables an application server processing the transaction to;

(a) maintain exclusive authority for processing the transaction;

(b) complete processing of the transaction; and

(c) initiate the commit operation, wherein the commit operation comprises the application server;

(1) marking the row for deletion and (2) simultaneously committing the row inserted and committing the deletion of the row to maintain transactional atomicity.

9. The method of claim 8:

wherein the host organization reinitiates processing for the transaction after the time delay subsequent to requeuing the message; and

wherein the reinitiated processing for the transaction after the time delay is terminated contingent on a determination that the row has been deleted from the database.

10. The method of claim 1, wherein the method further comprises:

determining that the status of the row inserted is non-existent and that the lock cannot be obtained because the row does not exist in the database; and

dequeuing the message from the message queue framework to prevent further processing associated with the transaction specified by the message.

11. The method of claim 10:

wherein determining that the status is non-existent and that the lock cannot be obtained because the row does not exist in the database collectively indicates that the processing of the transaction has previously completed successfully resulting in the transaction being committed successfully and the row being deleted successfully; and

determining the enqueued message specifying the transaction to be processed is an erroneous message.

12. The method of claim 1, further comprising:

performing a first determination that the lock on the row is not available;

performing a second determination that the status for the row is pending;

performing a third determination that the enqueued message has exceeded a time threshold; and

skipping a clean up operation for the enqueued message based on the first, second, and third determinations.

13. The method of claim 12, wherein the first, second, and third determinations collectively indicate that the transaction associated with the enqueued message is actively processing and requires a longer than expected processing period in excess of the time threshold.

14. The method of claim 1, further comprising:

initiating a clean up operation on a repeating periodic basis, wherein the clean up operation comprises;

identifying a set of one or more enqueued orphan messages based on each of the one or more enqueued orphan messages meeting the following criteria;

determining that a row corresponding to the enqueued orphan message is not locked;

determining the row corresponding to the enqueued orphan message has a status of pending; and

determining the enqueued orphan message has been enqueued for a time period in excess of a threshold.

15. The method of claim 1, further comprising:

receiving the message at the message queue framework of the host organization from a message broker; and

wherein inserting the row into a database of the host organization comprises the message queue framework inserting the row into the database and causing the database to autocommit the row upon insertion.

16. The method of claim 15, wherein the message broker lacks native transactional atomicity.

17. The method of claim 15, wherein receiving the message at the message queue framework comprises:

intercepting a message enqueued at an enqueue Application Programming Interface (enqueue API) of the message broker; and

triggering the row insertion responsive to intercepting the message.

18. The method of claim 15, further comprising:

dispatching the message to a worker thread to process the transaction specified by the message;

wherein requesting the lock on the row comprises the worker thread requesting the lock; and

wherein the worker thread performs at least the following operations;

the worker thread transacting the commit operation for the row inserted with the database and updating the status for the row from pending to ready responsive to successfully committing the row;

marking the row for deletion; and

committing deletion of the row from the database.

19. The method of claim 18, wherein dispatching the message to a worker comprises:

communicating, via a message transport, the message to the worker to perform its operations as an execution thread within one of a plurality of application servers within the host organization.

20. The method of claim 1, wherein the row inserted into the database comprises:

a first field for an organization identifier (OrgID) associated with the message;

a second field for a message tracker identifier to uniquely identify the message among a plurality of messages; and

a third field for the status.

21. The method of claim 1:

wherein the database of the host organization comprises a multi-tenant database system of the host organization; and

wherein enqueuing the message specifying a transaction to be processed via the host organization comprises enqueuing the message at the host organization as one of a plurality of requests received from a plurality of customer organizations, wherein each customer organization is an entity selected from the group consisting of;

a separate and distinct remote organization, an organizational group within the host organization, a business partner of the host organization, or a customer organization that subscribes to cloud computing services provided by the host organization.

22. The method of claim 21, wherein the multi-tenant database system comprises elements of hardware and software that are shared by a plurality of separate and distinct customer organizations, each of the separate and distinct customer organizations being remotely located from the host organization having the multi-tenant database system operating therein.

23. A non-transitory computer readable storage medium having instructions stored thereon that, when executed by a system in a host organization having a processor and memory therein, the instructions cause the system to perform operations comprising:

enqueuing a message into a message queue framework of the host organization, the message specifying a transaction to be processed via the host organization, the transaction comprising at least a row to be inserted into a database of the host organization;

inserting the row into the database of the host organization via the message queue framework, wherein a status for the row inserted into the database remains pending absent initiation of a commit operation for the row;

initiating, via the message queue framework, the commit operation for the row inserted into the database upon insertion of the row into the database regardless of whether any further processing occurs;

requesting a lock on the row inserted into the database;

concluding the transaction failed or processed successfully based on whether the lock requested on the row is obtained or denied and based further on whether the status for the row is pending or ready by determining whether the enqueued message has been orphaned and will not be consumed based on whether the lock on the row is available, whether the status for the row is pending, and whether the enqueued message has exceeded a time threshold; and

initiating a clean up operation for the enqueued message based on the determining whether the enqueued message has been orphaned and will not be consumed, wherein the clean up operation comprises discarding the message by removing the row from the database and committing the deletion of the row to the database.

24. The non-transitory computer readable storage medium of claim 23, wherein the instructions cause the system to perform further operations comprising:

dispatching the message to a worker thread to process the transaction specified by the message;

wherein requesting the lock on the row comprises the worker thread requesting the lock; and

wherein the worker thread performs at least the following operations;

the worker thread transacting the commit operation for the row inserted with the database and updating the status for the row from pending to ready responsive to successfully committing the row;

marking the row for deletion; and

committing deletion of the row from the database.

25. A system comprising:

a client device operating at a customer organization, the client device having a processor and a memory therein;

a host organization having a plurality of application servers therein, each having at least a processor to execute instructions;

a network communicably interfacing the client device with the host organization;

a request interface at the host organization to receive a request from the client device;

a message broker at the host organization to enqueue a message into a message queue framework of the host organization, the message specifying a transaction to be processed via the host organization, the transaction comprising at least a row to be inserted into a database of the host organization;

a message queue framework at the host organization to insert the row into the database of the host organization via the message queue framework, wherein a status for the row inserted into the database remains pending absent initiation of a commit operation for the row;

the message queue framework at the host organization further to initiate the commit operation for the row inserted into the database upon insertion of the row into the database regardless of whether any further processing is to occur;

a worker thread within one of the plurality of application servers of the host organization to request a lock on the row inserted into the database, and

the worker thread to further determine the transaction failed or processed successfully based on whether the lock requested on the row is obtained or denied and based further on whether the status for the row is pending or ready by determining whether the enqueued message has been orphaned and will not be consumed based on whether the lock on the row is available, whether the status for the row is pending, and whether the enqueued message has exceeded a time threshold; and

the worker thread to further initiate a clean up operation for the enqueued message based on the determining whether the enqueued message has been orphaned and will not be consumed, wherein the clean up operation comprises discarding the message by removing the row from the database and committing the deletion of the row to the database.

26. The system of claim 25, wherein the worker thread is to perform further processing for the transaction based on the status for the message and based further on whether the lock is obtained for the row, wherein the worker thread is to perform one of:

(a) rollback the transaction and discard the message when the lock on the row is obtained and the status for the row is pending indicating the row is not yet committed to the database;

(b) confirm the commit of the row to the database and process the message by committing removal of the row from the database when the lock on the row is obtained and the status for the row is ready;

(c) requeue the message with a time delay and schedule the system to reinitiate processing for the transaction after the time delay when the lock on the row is denied and the status for the row is pending;

(d) de-queue the message to prevent further processing associated with the transaction specified by the message when the status is non-existent and the lock cannot be obtained because the row does not exist in the database; and

(e) initiate a clean up operation for the enqueued message when the lock on the row is available and the status for the row is pending and the enqueued message has exceeded a time threshold.