Publisher flow control and bounded guaranteed delivery for message queues

• US 20060149787A1
• Filed: 07/29/2005
• Published: 07/06/2006
• Est. Priority Date: 12/30/2004
• Status: Active Grant

• Current Assignee: Oracle International Corporation (Oracle Corporation)
• Original Assignee: Oracle International Corporation (Oracle Corporation)
• Inventors: Mishra, Shailendra, Surlaker, Kapil, Bhatt, Neerja
• Granted Patent: US 7,779,418 B2
• Time in Patent Office: Days
• US Class Current: 1/1
• CPC Class Codes: G06F 9/546 Message passing systems or structures, e.g. queues

Abstract

Techniques for managing messages in computer systems are provided. In one embodiment, in response to a publisher attempting to enqueue a message in a queue, a determination is made whether a condition is satisfied. The condition is based on the current usage of the queue by the publisher. Based on whether the condition is satisfied, a decision is made whether to enqueue the message in the queue. The decision whether to enqueue the message may comprise restricting the publisher from enqueueing any more messages in the queue until the same or a different condition is satisfied.

Claims

1. A method for managing messages in computer systems, the method comprising:

in response to a publisher attempting to enqueue a message in a queue, determining whether a condition is satisfied,

wherein the condition is based on the current usage of the queue by the publisher; and

based on whether the condition is satisfied, deciding whether to enqueue the message in the queue.

2. The method of claim 1, wherein:

the publisher is associated with one or more consumers, wherein the one or more consumers consume the messages that are enqueued by the publisher in the queue; and

the queue is stored in volatile memory.

3. The method of claim 2, wherein the step of determining whether the condition is satisfied comprises determining whether the number of messages that are not consumed by any consumer of the one or more consumers exceeds a predetermined threshold number of messages that is specified for the publisher.

4. The method of claim 3, wherein:

the current usage of the queue by the publisher is associated with;

a first parameter indicating the number of messages in the queue that were enqueued by the publisher, and

a second parameter indicating the number of messages that have been consumed by at least one consumer of the one or more consumers; and

the number of messages that are not consumed by any consumer of the one or more consumers is the difference between the value of the first parameter and the value of the second parameter.

5. The method of claim 1, wherein the publisher is a particular publisher of a plurality of publishers, wherein each publisher of the plurality of publishers enqueues messages in the queue.

6. The method of claim 5, further comprising the step of determining, for each publisher of the plurality of publishers, whether that publisher is permitted to enqueue messages in the queue based on the current usage of the queue by that publisher.

7. The method of claim 1, wherein:

the current usage of the queue by the publisher is associated with at least one of;

a first parameter indicating the number of messages in the queue that were enqueued by the publisher;

a second parameter indicating the number of messages enqueued by the publisher that have been consumed by at least one consumer of one or more consumers that are associated with the publisher;

a third parameter indicating the size of any message that is enqueued by the publisher in the queue; and

a fourth parameter indicating the amount of volatile memory allocated for the queue; and

the condition is based on at least one of the first parameter, the second parameter, the third parameter, and the fourth parameter.

8. The method of claim 1, wherein the step of deciding whether to enqueue the message in the queue further comprises:

if the condition is satisfied, then declining to publish the message in the queue.

9. The method of claim 8, wherein the method further comprises notifying the publisher that the message has not been enqueued in the queue.

10. The method of claim 1, wherein the step of deciding whether to enqueue the message in the queue further comprises:

if the condition is not satisfied, then publishing the message in the queue.

11. The method of claim 1, wherein:

the condition is a first condition; and

the method further comprises, if the first condition is not satisfied, then restricting the publisher from enqueueing any more messages in the queue until a second condition is satisfied.

12. The method of claim 11, wherein the second condition is different than the first condition.

13. The method of claim 11, wherein the second condition is the same as the first condition.

14. The method of claim 1, wherein:

the publisher is a particular publisher of a plurality of publishers that share the queue, wherein the queue is stored in volatile memory; and

one or more messages previously enqueued in the queue by the particular publisher have been spilled over to permanent store.

15. The method of claim 14, wherein the condition is based on the current usage by the particular publisher of the permanent store.

16. The method of claim 15, wherein:

the particular publisher is associated with one or more consumers that consume messages enqueued by the particular publisher in the queue;

the current usage of the queue by the particular publisher is associated with a first parameter indicating the number of messages enqueued by the particular publisher that have been consumed by at least one consumer of the one or more consumers;

the current usage of the permanent store by the particular publisher is associated with a second parameter indicating the number of messages enqueued by the particular publisher that have been spilled over to the permanent store; and

the step of determining whether the condition is satisfied comprises determining whether the difference between the value of the second parameter and the value of the first parameter exceeds a threshold number of messages.

17. The method of claim 16, further comprising the step of:

defining a separate threshold number of messages for each publisher of the plurality of publishers.

18. The method of claim 14, wherein:

the particular publisher is associated with one or more consumers that consume messages enqueued by the particular publisher in the queue; and

the method further comprises determining whether to spill over the one or more messages to the permanent store based on at least one of;

the amount of recovery time configured for the particular publisher, wherein the recovery time is the maximum period of time allotted upon restart of the particular publisher for re-creating any messages that were enqueued in the queue before the restart and that have not been consumed by all consumers of the one or more consumers; and

the amount of time that any message of the one or more messages has spent in the queue.

19. The method of claim 14, wherein deciding whether to enqueue the message in the queue further comprises restricting the publisher from enqueueing the message in the queue.

29. A computer-readable medium carrying one or more sequences of instructions which, when executed by one or more processors, causes the one or more processors to perform the method recited in claim 1.

30. A computer-readable medium carrying one or more sequences of instructions which, when executed by one or more processors, causes the one or more processors to perform the method recited in claim 2.

31. A computer-readable medium carrying one or more sequences of instructions which, when executed by one or more processors, causes the one or more processors to perform the method recited in claim 3.

32. A computer-readable medium carrying one or more sequences of instructions which, when executed by one or more processors, causes the one or more processors to perform the method recited in claim 4.

33. A computer-readable medium carrying one or more sequences of instructions which, when executed by one or more processors, causes the one or more processors to perform the method recited in claim 5.

34. A computer-readable medium carrying one or more sequences of instructions which, when executed by one or more processors, causes the one or more processors to perform the method recited in claim 6.

35. A computer-readable medium carrying one or more sequences of instructions which, when executed by one or more processors, causes the one or more processors to perform the method recited in claim 7.

36. A computer-readable medium carrying one or more sequences of instructions which, when executed by one or more processors, causes the one or more processors to perform the method recited in claim 8.

37. A computer-readable medium carrying one or more sequences of instructions which, when executed by one or more processors, causes the one or more processors to perform the method recited in claim 9.

38. A computer-readable medium carrying one or more sequences of instructions which, when executed by one or more processors, causes the one or more processors to perform the method recited in claim 10.

39. A computer-readable medium carrying one or more sequences of instructions which, when executed by one or more processors, causes the one or more processors to perform the method recited in claim 11.

40. A computer-readable medium carrying one or more sequences of instructions which, when executed by one or more processors, causes the one or more processors to perform the method recited in claim 12.

41. A computer-readable medium carrying one or more sequences of instructions which, when executed by one or more processors, causes the one or more processors to perform the method recited in claim 13.

42. A computer-readable medium carrying one or more sequences of instructions which, when executed by one or more processors, causes the one or more processors to perform the method recited in claim 14.

43. A computer-readable medium carrying one or more sequences of instructions which, when executed by one or more processors, causes the one or more processors to perform the method recited in claim 15.

44. A computer-readable medium carrying one or more sequences of instructions which, when executed by one or more processors, causes the one or more processors to perform the method recited in claim 16.

45. A computer-readable medium carrying one or more sequences of instructions which, when executed by one or more processors, causes the one or more processors to perform the method recited in claim 17.

46. A computer-readable medium carrying one or more sequences of instructions which, when executed by one or more processors, causes the one or more processors to perform the method recited in claim 18.

47. A computer-readable medium carrying one or more sequences of instructions which, when executed by one or more processors, causes the one or more processors to perform the method recited in claim 19.

20. A method for managing messages in computer systems, the method comprising:

if one or more conditions are satisfied, then spilling over a first one or more messages from a queue in volatile memory to a permanent store, wherein the first one or more messages were enqueued in the queue by a publisher; and

in response to spilling over the one or more messages, updating a parameter associated with the publisher, wherein the parameter indicates a particular message enqueued by the publisher in the queue.

21. The method of claim 20, further comprising the step of:

in response to the occurrence of an event that requires a replay operation, inspecting the parameter to determine that no message enqueued prior to the particular message should be re-enqueued during the replay operation; and

re-enqueueing in the queue, by the publisher, second one or more messages, that were previously enqueued by the publisher after the particular message, wherein the second one or more messages do not include any message of the first one or more messages.

22. The method of claim 21, wherein the second one or more messages include all messages that have been enqueued in the queue by the publisher after the particular message.

23. The method of claim 21, wherein the first one or more messages and the second one or more messages have not been consumed by all consumers of one or more consumers that are associated with the publisher.

24. The method of claim 20, wherein the particular message is not among the first one or more messages.

25. The method of claim 20, wherein the one or more conditions are associated with the cost incurred by the publisher in producing the first one or more messages.

26. The method of claim 25, wherein a particular condition of the one or more conditions is associated with the amount of time it takes the publisher to re-create the first one or more messages after a re-start performed by the publisher.

27. The method of claim 25, wherein a particular condition of the one or more conditions is associated with the amount of computing resources it takes the publisher to re-create the first one or more messages.

28. The method of claim 25, wherein a particular condition of the one or more conditions is associated with the number of messages enqueued in the queue by the publisher after enqueueing the last message that was consumed by all consumers of one or more consumers that are associated with the publisher.

48. A computer-readable medium carrying one or more sequences of instructions which, when executed by one or more processors, causes the one or more processors to perform the method recited in claim 20.

49. A computer-readable medium carrying one or more sequences of instructions which, when executed by one or more processors, causes the one or more processors to perform the method recited in claim 21.

50. A computer-readable medium carrying one or more sequences of instructions which, when executed by one or more processors, causes the one or more processors to perform the method recited in claim 22.

51. A computer-readable medium carrying one or more sequences of instructions which, when executed by one or more processors, causes the one or more processors to perform the method recited in claim 23.

52. A computer-readable medium carrying one or more sequences of instructions which, when executed by one or more processors, causes the one or more processors to perform the method recited in claim 24.

53. A computer-readable medium carrying one or more sequences of instructions which, when executed by one or more processors, causes the one or more processors to perform the method recited in claim 25.

54. A computer-readable medium carrying one or more sequences of instructions which, when executed by one or more processors, causes the one or more processors to perform the method recited in claim 26.

55. A computer-readable medium carrying one or more sequences of instructions which, when executed by one or more processors, causes the one or more processors to perform the method recited in claim 27.

56. A computer-readable medium carrying one or more sequences of instructions which, when executed by one or more processors, causes the one or more processors to perform the method recited in claim 28.