Duplicate message elimination during recovery when multiple threads are delivering messages from a message store to a destination queue

• US 20060200829A1
• Filed: 02/18/2005
• Published: 09/07/2006
• Est. Priority Date: 02/18/2005
• Status: Active Grant

• Current Assignee: International Business Machines Corporation
• Original Assignee: International Business Machines Corporation
• Inventors: Astl, Kenneth L., Sundaramurthy, Arul, Gooty, Subramanyam Agraharam, Zhang, Kai Mike
• Granted Patent: US 7,480,918 B2
• Time in Patent Office: Days
• US Class Current: 719/314
• CPC Class Codes: G06F 11/0715 in a system implementing multitasking multitasking per se G06F9/46
• CPC Class Codes: G06F 11/0793 Remedial or corrective actions recovery from an exception in an instruction pipeline G06F9/3861; by retry G06F11/1402; for recovering from a failure of a protocol instance or entity H04L69/40
• CPC Class Codes: G06F 2209/547 Messaging middleware
• CPC Class Codes: G06F 9/546 Message passing systems or structures, e.g. queues

Abstract

Provided are a method, system, and program for duplicate message elimination during recovery when multiple threads are delivering messages from a message store to a destination queue. A plurality of message threads process operations to deliver messages from a message store to a destination queue, wherein one message thread processes one message. An in-doubt list is generated identifying messages that are in-progress of being delivered form the message store to the destination queue by the message threads. One message thread processing one message adds an entry including the message identifier and the thread identifier to a monitor queue. The message thread further adds the message to the destination queue. A recovery thread is generated in response to detecting a failure in the processing by the threads to deliver the messages from the message store to the destination queue. The recovery thread processes the messages indicated in the in-doubt list and compares with message identifiers in the monitor queue to prevent duplicate delivery of messages to the destination queue.

Claims

1. A method, comprising:

processing, by a plurality of message threads, operations to deliver messages from a message store to a destination queue, wherein one message thread processes one message;

generating an in-doubt list identifying messages that are in-progress of being delivered form the message store to the destination queue by the message threads;

adding, by one message thread processing one message, an entry including the message identifier and the thread identifier to a monitor queue;

adding, by the message thread, the message to the destination queue;

generating a recovery thread in response to detecting a failure in the processing by the message threads to deliver the messages from the message store to the destination queue; and

processing, by the recovery thread, the messages indicated in the in-doubt list and comparing with message identifiers in the monitor queue to prevent duplicate delivery of messages to the destination queue.

2. The method of claim 1, wherein the message added to the destination queue comprises a current message, wherein an entry in the monitor queue indicates a previous message identifier of a previous message processed by the message thread, further comprising:

replacing, by the message thread, the entry in the monitor queue for the previous message with the added entry including the message identifier of the current message in response to delivering the current message to the destination queue; and

removing the current message from the destination queue and returning the entry to the monitor queue including the previous message identifier in response to the failure occurring before the current message is committed to the destination queue.

3. The method of claim 1, wherein processing, by the recovery thread, the messages indicated in the in-doubt list and comparing with message identifiers in the monitor queue to prevent duplicate delivery of messages to the destination queue further comprises the recovery thread performing for each message in the in-doubt list:

receiving the message identifier of a message in the in-doubt list;

determining whether the received message identifier is included in one entry in the monitor queue; and

acknowledging to the message store that the message having the received message identifier was delivered to the destination queue in response to determining that the received message identifier is included in one entry in the monitor queue, wherein the message store removes the message acknowledged to have been delivered to the destination queue from the message store and the in-doubt list.

4. The method of claim 1, wherein processing, by the recovery thread, the messages indicated in the in-doubt list and comparing with message identifiers in the monitor queue to prevent duplicate delivery of messages to the destination queue further comprises the recovery thread performing for each message in the in doubt list:

receiving the message identifier of a message in the in-doubt list;

determining whether the received message identifier is included in one entry in the monitor queue; and

adding one entry to the monitor queue including the recovery thread identifier and the received message identifier in response to determining that the received message identifier is not included in one entry in the monitor queue.

5. The method of claim 4, wherein the recovery thread further performs for each message indicated in the in-doubt list:

delivering to the destination queue the message identified by the received message identifier in response to determining that that the received message identifier is not included in one entry in the monitor queue.

6. The method of claim 4, wherein the entry added to the monitor queue including the recovery thread identifier replaces any preexisting entry in the monitor queue indicating the recovery thread and a message identifier different from the received message identifier.

7. The method of claim 4, wherein the recovery thread further performs:

acknowledging to the message store that the message having the received message identifier was delivered to the destination queue in response to determining that the received message identifier is included in one entry in the monitor queue, wherein the message store removes the message acknowledged to have been delivered to the destination queue from the message store and the in-doubt list.

8. The method of claim 4, wherein the recovery thread further performs:

reinitiating the processing of the messages in the in-doubt list and comparing with message identifiers in the monitor queue to prevent duplicate delivery of messages to the destination queue in response to recovering from a failure that occurred while the recovery thread was processing messages indicated in the in-doubt list.

9. The method of claim 1, further comprising:

returning processing to the message threads in response to the recovery thread processing all messages indicated in the in-doubt list and delivering the messages indicated in the in-doubt list to the destination queue.

10. The method of claim 1, wherein the monitor and destination queues are in a message broker, and where an agent spawns message threads and the recover threads to deliver messages from the message store to the message broker.

11. A system, comprising:

a message store;

a destination broker;

a monitor queue;

a destination queue;

an in-doubt list identifying messages that are in-progress of being delivered form the message store to the destination queue;

a processor;

a plurality of message threads executed by the processor processing operations to deliver messages from the message store to the destination queue, by performing;

(i) adding, by one message thread processing one message, an entry including the message identifier and the thread identifier to a monitor queue;

(ii) adding the message to the destination queue;

a recovery thread generated by the processor in response to detecting a failure in the processing by the message threads to deliver the messages from the message store to the destination queue, wherein the recovery thread executed by the processor processes the messages indicated in the in-doubt list and compares with message identifiers in the monitor queue to prevent duplicate delivery of messages to the destination queue.

12. The system of claim 11, wherein the message added to the destination queue comprises a current message, wherein an entry in the monitor queue indicates a previous message identifier of a previous message processed by the message thread, further comprising:

wherein the method thread executed by the processor further performs replacing the entry in the monitor queue for the previous message with the added entry including the message identifier of the current message in response to delivering the current message to the destination queue; and

a rollback operation executed by the processor to remove the current message from the destination queue and returning the entry to the monitor queue including the previous message identifier in response to the failure occurring before the current message is committed to the destination queue.

13. The system of claim 11, wherein processing, by the recovery thread, the messages indicated in the in-doubt list and comparing with message identifiers in the monitor queue to prevent duplicate delivery of messages to the destination queue further comprises the recovery thread performing for each message in the in-doubt list:

receiving the message identifier of a message in the in-doubt list;

determining whether the received message identifier is included in one entry in the monitor queue; and

acknowledging to the message store that the message having the received message identifier was delivered to the destination queue in response to determining that the received message identifier is included in one entry in the monitor queue, wherein the message store removes the message acknowledged to have been delivered to the destination queue from the message store and the in-doubt list.

14. The system of claim 11, wherein processing, by the recovery thread, the messages indicated in the in-doubt list and comparing with message identifiers in the monitor queue to prevent duplicate delivery of messages to the destination queue further comprises the recovery thread performing for each message in the in doubt list:

receiving the message identifier of a message in the in-doubt list;

determining whether the received message identifier is included in one entry in the monitor queue; and

adding one entry to the monitor queue including the recovery thread identifier and the received message identifier in response to determining that the received message identifier is not included in one entry in the monitor queue.

15. The system of claim 14, wherein the recovery thread further performs for each message indicated in the in-doubt list:

delivering to the destination queue the message identified by the received message identifier in response to determining that that the received message identifier is not included in one entry in the monitor queue.

16. The system of claim 14, wherein the entry added to the monitor queue including the recovery thread identifier replaces any preexisting entry in the monitor queue indicating the recovery thread and a message identifier different from the received message identifier.

17. The system of claim 14, wherein the recovery thread further performs:

acknowledging to the message store that the message having the received message identifier was delivered to the destination queue in response to determining that the received message identifier is included in one entry in the monitor queue, wherein the message store removes the message acknowledged to have been delivered to the destination queue from the message store and the in-doubt list.

18. The system of claim 14, wherein the recovery thread further performs:

reinitiating the processing of the messages in the in-doubt list and comparing with message identifiers in the monitor queue to prevent duplicate delivery of messages to the destination queue in response to recovering from a failure that occurred while the recovery thread was processing messages indicated in the in-doubt list.

19. The system of claim 11, wherein the recovery thread further performs:

returning processing to the message threads in response to the recovery thread processing all messages indicated in the in-doubt list and delivering the messages indicated in the in-doubt list to the destination queue.

20. The system of claim 11, further comprising:

a message broker including the monitor and destination queues; and

an agent, executed by the processor, that spawns the message threads and the recovery threads to deliver messages from the message store to the message broker.

21. An article of manufacture comprising code executed to generate message threads and a recovery thread to perform operations with respect to a message store, a monitor queue, and a destination queue, wherein the operations comprise:

processing, by the message threads, operations to deliver messages from the message store to the destination queue, wherein one message thread processes one message;

generating an in-doubt list identifying messages that are in-progress of being delivered form the message store to the destination queue by the message threads;

adding, by one message thread processing one message, an entry including the message identifier and the thread identifier to a monitor queue;

adding, by the message thread, the message to the destination queue;

generating a recovery thread in response to detecting a failure in the processing by the message threads to deliver the messages from the message store to the destination queue; and

processing, by the recovery thread, the messages indicated in the in-doubt list and comparing with message identifiers in the monitor queue to prevent duplicate delivery of messages to the destination queue.

22. The article of manufacture of claim 21, wherein the message added to the destination queue comprises a current message, wherein an entry in the monitor queue indicates a previous message identifier of a previous message processed by the message thread, wherein the operations further comprise:

replacing, by the message thread, the entry in the monitor queue for the previous message with the added entry including the message identifier of the current message in response to delivering the current message to the destination queue; and

removing the current message from the destination queue and returning the entry to the monitor queue including the previous message identifier in response to the failure occurring before the current message is committed to the destination queue.

23. The article of manufacture of claim 21, wherein processing, by the recovery thread, the messages indicated in the in-doubt list and comparing with message identifiers in the monitor queue to prevent duplicate delivery of messages to the destination queue further comprises the recovery thread performing for each message in the in-doubt list:

receiving the message identifier of a message in the in-doubt list;

determining whether the received message identifier is included in one entry in the monitor queue; and

acknowledging to the message store that the message having the received message identifier was delivered to the destination queue in response to determining that the received message identifier is included in one entry in the monitor queue, wherein the message store removes the message acknowledged to have been delivered to the destination queue from the message store and the in-doubt list.

24. The article of manufacture of claim 21, wherein processing, by the recovery thread, the messages indicated in the in-doubt list and comparing with message identifiers in the monitor queue to prevent duplicate delivery of messages to the destination queue further comprises the recovery thread performing for each message in the in doubt list:

receiving the message identifier of a message in the in-doubt list;

determining whether the received message identifier is included in one entry in the monitor queue; and

adding one entry to the monitor queue including the recovery thread identifier and the received message identifier in response to determining that the received message identifier is not included in one entry in the monitor queue.

25. The article of manufacture of claim 24, wherein the recovery thread further performs for each message indicated in the in-doubt list:

delivering to the destination queue the message identified by the received message identifier in response to determining that that the received message identifier is not included in one entry in the monitor queue.

26. The article of manufacture of claim 24, wherein the entry added to the monitor queue indicating the recovery thread identifier replaces any preexisting entry in the monitor queue indicating the recovery thread and a message identifier different from the received message identifier.

27. The article of manufacture of claim 24, wherein the recovery thread further performs:

acknowledging to the message store that the message having the received message identifier was delivered to the destination queue in response to determining that the received message identifier is included in one entry in the monitor queue, wherein the message store removes the message acknowledged to have been delivered to the destination queue from the message store and the in-doubt list.

28. The article of manufacture of claim 24, wherein the recovery thread further performs:

reinitiating the processing of the messages in the in-doubt list and comparing with message identifiers in the monitor queue to prevent duplicate delivery of messages to the destination queue in response to recovering from a failure that occurred while the recovery thread was processing messages indicated in the in-doubt list.

29. The article of manufacture of claim 21, wherein the operations further comprise:

returning processing to the message threads in response to the recovery thread processing all messages indicated in the in-doubt list and delivering the messages indicated in the in-doubt list to the destination queue.

30. The article of manufacture of claim 21, wherein the monitor and destination queues are in a message broker, and where the code implements an agent that spawns message threads and the recover threads to deliver messages from the message store to the message broker.