Buffered message queue architecture for database management systems

• US 20040024774A1
• Filed: 05/21/2003
• Published: 02/05/2004
• Est. Priority Date: 08/01/2002
• Status: Active Grant

• Current Assignee: Oracle International Corporation (Oracle Corporation)
• Original Assignee: Oracle International Corporation (Oracle Corporation)
• Inventors: Meiyyappan, Krishnan, Mishra, Shailendra, Surlaker, Kapil, Jain, Namit, Kaluskar, Sanjay, Bhatt, Neerja
• Granted Patent: US 7,181,482 B2
• Time in Patent Office: Days
• US Class Current: 707/102
• CPC Class Codes: G06F 16/21 Design, administration or maintenance of databases
• CPC Class Codes: Y10S 707/99931 Database or file accessing
• CPC Class Codes: Y10S 707/99957 Garbage collection

Abstract

A buffered message queue architecture for managing messages in a database management system is disclosed. A “buffered message queue” refers to a message queue implemented in a volatile memory, such as a RAM. The volatile memory may be a shared volatile memory that is accessible by a plurality of processes. The buffered message queue architecture supports a publish and subscribe communication mechanism, where the message producers and message consumers may be decoupled from and independent of each other. The buffered message queue architecture provides all the functionality of a persistent publish-subscriber messaging system, without ever having to store the messages in persistent storage. The buffered message queue architecture provides better performance and scalability since no persistent operations are needed and no UNDO/REDO logs need to be maintained. Messages published to the buffered message queue are delivered to all eligible subscribers at least once, even in the event of failures, as long as the application is “repeatable.” The buffered message queue architecture also includes management mechanisms for performing buffered message queue cleanup and also for providing unlimited size buffered message queues when limited amounts of shared memory are available. The architecture also includes “zero copy” buffered message queues and provides for transaction-based enqueue of messages.

Claims

1. A method for managing messages in a database system, the method comprising the computer-implemented steps of:

creating a message queue in a volatile memory of the database system;

storing a first message in the message queue;

determining whether the first message satisfies subscription data for a first subscriber process; and

if the first message satisfies the subscription data for the first subscriber process, then providing, to the first subscriber process, access to the first message in the message queue.

2. The method as recited in claim 1, further comprising the computer-implemented step of in response to providing, to the first subscriber process, access to the first message in the message queue, deleting the first message from the message queue.

3. The method as recited in claim 1, further comprising the computer-implemented step of if data has not been received to indicate that the first message should not be deleted from the message queue after access to the first message is provided to the first subscriber process, then in response to providing, to the first subscriber, access to the first message in the message queue, deleting the first message from the message queue.

4. The method as recited in claim 1, further comprising the computer-implemented step of if an amount of available space in the message queue is less than a specified amount and if access to the first message has been provided to all active subscriber processes for which the first message satisfies corresponding subscription criteria, then deleting the first message from the message queue.

5. The method as recited in claim 1, further comprising the computer-implemented steps of:

creating a persistent message queue in a database of the database system; and

storing a second message in the persistent message queue.

6. The method as recited in claim 1, further comprising the computer-implemented steps of:

storing a second message in the message queue in a specified order relative to the first message; and

determining, based upon the specified order, whether the first and second messages satisfy subscription data for a second subscriber process.

7. The method as recited in claim 1, wherein the specified order is based upon when the first and second messages were stored in the message queue.

8. The method as recited in claim 1, further comprising the computer-implemented step of generating last message data that indicates that the first message was the last message received by the first subscriber process.

9. The method as recited in claim 8, further comprising the computer-implemented steps of:

storing a second message in the message queue;

determining whether the second message satisfies the subscription data for the first subscriber process; and

if the second message satisfies the subscription data for the first subscriber process, then

providing, to the first subscriber process, access to the second message in the message queue, and

updating the last message data to indicate that the second message was the last message received by the first subscriber process.

10. The method as recited in claim 1, wherein the message queue is created in response to receiving a request to store the first message into the message queue, wherein the message queue did not exist when the request to store the first message was received.

11. The method as recited in claim 1, further comprising the computer-implemented step of dynamically changing over time an amount of the volatile memory allocated to the message queue.

12. The method as recited in claim 1, wherein the step of providing, to. the first subscriber process, access to the first message in the message queue includes providing a copy of the first message to the first subscriber process.

13. The method as recited in claim 1, wherein the step of providing, to the first subscriber process, access to the first message in the message queue includes providing, to the first subscriber process, an address of the first message in the message queue.

14. The method as recited in claim 1, wherein the step of creating a message queue in a volatile memory of the database system includes creating a message queue in a shared volatile memory of the database system, wherein the shared volatile memory is accessible by a plurality of processes.

15. A computer-readable medium for managing messages in a database system, the computer-readable medium carrying one or more sequences of instructions which, when executed by one or more processors, cause the one or more processors to perform the steps of:

creating a message queue in a volatile memory of the database system;

storing a first message in the message queue;

determining whether the first message satisfies subscription data for a first subscriber process; and

if the first message satisfies the subscription data for the first subscriber process, then providing, to the first subscriber process, access to the first message in the message queue.

16. The computer-readable medium as recited in claim 15, further comprising one or more additional instructions which, when executed by the one or more processors, cause the one or more processors to perform the step of in response to providing, to the first subscriber process, access to the first message in the message queue, deleting the first message from the message queue.

17. The computer-readable medium as recited in claim 15, further comprising one or more additional instructions which, when executed by the one or more processors, cause the one or more processors to perform the step of if data has not been received to indicate that the first message should not be deleted from the message queue after access to the first message is provided to the first subscriber process, then in response to providing, to the first subscriber, access to the first message in the message queue, deleting the first message from the message queue.

18. The computer-readable medium as recited in claim 15, further comprising one or more additional instructions which, when executed by the one or more processors, cause the one or more processors to perform the step of if an amount of available space in the message queue is less than a specified amount and if access to the first message has been provided to all active subscriber processes for which the first message satisfies corresponding subscription criteria, then deleting the first message from the message queue.

19. The computer-readable medium as recited in claim 15, further comprising one or more additional instructions which, when executed by the one or more processors, cause the one or more processors to perform the steps of:

creating a persistent message queue in a database of the database system; and

storing a second message in the persistent message queue.

20. The computer-readable medium as recited in claim 15, further comprising one or more additional instructions which, when executed by the one or more processors, cause the one or more processors to perform the steps of:

storing a second message in the message queue in a specified order relative to the first message; and

determining, based upon the specified order, whether the first and second messages satisfy subscription data for a second subscriber process.

21. The computer-readable medium as recited in claim 15, wherein the specified order is based upon when the first and second messages were stored in the message queue.

22. The computer-readable medium as recited in claim 15, further comprising one or more additional instructions which, when executed by the one or more processors, cause the one or more processors to perform the step of generating last message data that indicates that the first message was the last message received by the first subscriber process.

23. The computer-readable medium as recited in claim 22, further comprising one or more additional instructions which, when executed by the one or more processors, cause the one or more processors to perform the steps of:

storing a second message in the message queue;

determining whether the second message satisfies the subscription data for the first subscriber process; and

if the second message satisfies the subscription data for the first subscriber process, then

providing, to the first subscriber process, access to the second message in the message queue, and

updating the last message data to indicate that the second message was the last message received by the first subscriber process.

24. The computer-readable medium as recited in claim 15, wherein the message queue is created in response to receiving a request to store the first message into the message queue, wherein the message queue did not exist when the request to store the first message was received.

25. The computer-readable medium as recited in claim 15, further comprising one or more additional instructions which, when executed by the one or more processors, cause the one or more processors to perform the step of dynamically changing over time an amount of the volatile memory allocated to the message queue.

26. The computer-readable medium as recited in claim 15, wherein the step of providing, to the first subscriber process, access to the first message in the message queue includes providing a copy of the first message to the first subscriber process.

27. The computer-readable medium as recited in claim 15, wherein the step of providing, to the first subscriber process, access to the first message in the message queue includes providing, to the first subscriber process, an address of the first message in the message queue.

28. The computer-readable medium as recited in claim 15, wherein the step of creating a message queue in a volatile memory of the database system includes creating a message queue in a shared volatile memory of the database system, wherein the shared volatile memory is accessible by a plurality of processes.

29. An apparatus for managing messages in a database system, the apparatus comprising a memory carrying one or more sequences of instructions which, when executed by one or more processors, cause the one or more processors to perform the steps of:

creating a message queue in a volatile memory of the database system;

storing a first message in the message queue;

determining whether the first message satisfies subscription data for a first subscriber process; and

if the first message satisfies the subscription data for the first subscriber process, then providing, to the first subscriber process, access to the first message in the message queue.

30. The apparatus as recited in claim 29, wherein the memory further includes one or more additional instructions which, when executed by the one or more processors, cause the one or more processors to perform the step of in response to providing, to the first subscriber process, access to the first message in the message queue, deleting the first message from the message queue.

31. The apparatus as recited in claim 29, further comprising one or more additional instructions which, when executed by the one or more processors, cause the one or more processors to perform the step of if data has not been received to indicate that the first message should not be deleted from the message queue after access to the first message is provided to the first subscriber process, then in response to providing, to the first subscriber, access to the first message in the message queue, deleting the first message from the message queue.

32. The apparatus as recited in claim 29, wherein the memory further includes one or more additional instructions which, when executed by the one or more processors, cause the one or more processors to perform the step of if an amount of available space in the message queue is less than a specified amount and if access to the first message has been provided to all active subscriber processes for which the first message satisfies corresponding subscription criteria, then deleting the first message from the message queue.

33. The apparatus as recited in claim 29, wherein the memory further includes one or more additional instructions which, when executed by the one or more processors, cause the one or more processors to perform the steps of:

creating a persistent message queue in a database of the database system; and

storing a second message in the persistent message queue.

34. The apparatus as recited in claim 29, wherein the memory further includes one or more additional instructions which, when executed by the one or more processors, cause the one or more processors to perform the steps of:

storing a second message in the message queue in a specified order relative to the first message; and

determining, based upon the specified order, whether the first and second messages satisfy subscription data for a second subscriber process.

35. The apparatus as recited in claim 29, wherein the specified order is based upon when the first and second messages were stored in the message queue.

36. The apparatus as recited in claim 29, wherein the memory further includes one or more additional instructions which, when executed by the one or more processors, cause the one or more processors to perform the step of generating last message data that indicates that the first message was the last message received by the first subscriber process.

37. The apparatus as recited in claim 36, wherein the memory further includes one or more additional instructions which, when executed by the one or more processors, cause the one or more processors to perform the steps of:

storing a second message in the message queue;

determining whether the second message satisfies the subscription data for the first subscriber process; and

if the second message satisfies the subscription data for the first subscriber process, then

providing, to the first subscriber process, access to the second message in the message queue, and

updating the last message data to indicate that the second message was the last message received by the first subscriber process.

38. The apparatus as recited in claim 29, wherein the message queue is created in response to receiving a request to store the first message into the message queue, wherein the message queue did not exist when the request to store the first message was received.

39. The apparatus as recited in claim 29, wherein the memory further includes one or more additional instructions which, when executed by the one or more processors, cause the one or more processors to perform the step of dynamically changing over time an amount of the volatile memory allocated to the message queue.

40. The apparatus as recited in claim 29, wherein the step of providing, to the first subscriber process, access to the first message in the message queue includes providing a copy of the first message to the first subscriber process.

41. The apparatus as recited in claim 29, wherein the step of providing, to the first subscriber process, access to the first message in the message queue includes providing, to the first subscriber process, an address of the first message in the message queue.

42. The apparatus as recited in claim 29, wherein the step of creating a message queue in a volatile memory of the database system includes creating a message queue in a shared volatile memory of the database system, wherein the shared volatile memory is accessible by a plurality of processes.